PDA

View Full Version : DC_READ_FREE issue



srini_31
10-16-2008, 02:51 PM
If I have a .csv file like below, DC_READ_FREE will not be able to parse the file correctly. It will read the value 212.3 into the previous column. How ever if I put an additioanl delimiter in that line, it works fine.
Is there any way I can get it to work if there are delimiters before the data.

700.5,12,64,0,212.3,0,41
699.5,13,64,0,212.3,0,41
674,16,129,0,212.3,1,56
854.5,17,129,0,212.3,1,100
1012,18,129,0,212.3,1,99
1080,19,129,0,212.3,1,100
,,,,212.3,0,99
,,,,100.3,1,90
,,,,210,0,88

Thanks,
Srinivas

donb
10-16-2008, 05:32 PM
Hi Srinivas,

These lines seem to read your example data:


fname = 'tst.dat'
a = 0.0 & e = 0.0
b = 0 & c = 0 & d = 0 & f = 0 & g = 0
s=DC_READ_FREE(fname, a, b, c, d, e, f, g, $
/Column, $
Delim = [','], $
Resize = INDGEN(7)+1, $
Miss_str = [''], $
Miss_val = [-9999])

There were mixed data types so I predefined the variable types. That isn't needed if you want them to all default to floating point data or if you expect to cast the results of the read.

The Delim keyword was used to specifically identify commas as the delimiter within the file. The Resize keyword tells DC_READ_FREE to continue reading the file and growing the results of the array sizes until the entire file is read. And the Miss_str, Miss_val keywords were used as a substitution pair; anytime DC_READ_FREE encounters a null string in the file it loads the value -9999 instead. This helps you easily idenify missing data within the file, otherwise DC_READ_FREE loads a zero and it's hard to tell if that's a real or a missing value from the file.

Hope this helps,

Don B.

hcrisp
11-26-2008, 03:04 PM
I am having the same problem. Using version 8.00. When I try Don's suggested code, I get the following:


WAVE> pm, [[a],[b],[c]]
700.500 12.0000 64.0000
699.500 13.0000 64.0000
674.000 16.0000 129.000
854.500 17.0000 129.000
1012.00 18.0000 129.000
1080.00 19.0000 129.000
0.000000 0.000000 0.000000
0.000000 0.000000 0.000000
0.000000 0.000000 0.000000
WAVE> pm, [[d],[e],[f]]
0.000000 212.300 0.000000
0.000000 212.300 0.000000
0.000000 212.300 1.00000
0.000000 212.300 1.00000
0.000000 212.300 1.00000
0.000000 212.300 1.00000
212.000 0.000000 99.0000
100.000 1.00000 90.0000
0.000000 210.000 0.000000
WAVE> pm, [g]
41
41
56
100
99
100
88
WAVE> info, a
A FLOAT = Array(9)
WAVE> info, g
G INT = Array(7)

Clearly, something is wrong.
(a) The missing characters are not getting replaced by -9999.
(b) The lines which begin with ",,,," are being shifted left one column. (Because the intial comma doesn't get read as a character or because double commas are treated as one delimiter?)
(c) The last variable, g, is two elements short, 7 instead of 9.

Does DC_READ_FREE have a bug? Or is there some other way to call it to get the right data?

brian
11-26-2008, 05:09 PM
As a quick test I ran the code in my home window and it seems to function as Don indicated. Is it possible one of the arguments has been omitted? You may want to cd to the directory with the data file then cut and paste the following code in to the home window or console:



fname = 'tst.dat'
a = 0.0 & e = 0.0
b = 0 & c = 0 & d = 0 & f = 0 & g = 0
s=DC_READ_FREE(fname, a, b, c, d, e, f, g, $
/Column, $
Delim = [','], $
Resize = INDGEN(7)+1, $
Miss_str = [''], $
Miss_val = [-9999])
pm, [[a],[b],[c], [d], [e]]
pm,[ [f], [g]]


Results:



WAVE> fname = 'tst.dat'
WAVE> a = 0.0 & e = 0.0
WAVE> b = 0 & c = 0 & d = 0 & f = 0 & g = 0
WAVE> s=DC_READ_FREE(fname, a, b, c, d, e, f, g, $
- /Column, $
- Delim = [','], $
- Resize = INDGEN(7)+1, $
- Miss_str = [''], $
- Miss_val = [-9999])
WAVE> pm, [[a],[b],[c], [d], [e]]
700.500 12.0000 64.0000 0.000000 212.300
699.500 13.0000 64.0000 0.000000 212.300
674.000 16.0000 129.000 0.000000 212.300
854.500 17.0000 129.000 0.000000 212.300
1012.00 18.0000 129.000 0.000000 212.300
1080.00 19.0000 129.000 0.000000 212.300
-9999.00 -9999.00 -9999.00 -9999.00 212.300
-9999.00 -9999.00 -9999.00 -9999.00 100.300
-9999.00 -9999.00 -9999.00 -9999.00 210.000
WAVE> pm,[ [f], [g]]
0 41
0 41
1 56
1 100
1 99
1 100
0 99
1 90
0 88



--brian

donb
11-26-2008, 08:51 PM
Hi hcrisp,

I don't have 8.00 on my machine, but I do have 8.01a; that produces the correct results.

I also looked at the Update_Notice file (a list of all Change Requests addressed) for version 8 and did not notice any fixes related to DC_READ_FREE.

As you state, clearly something is wrong, or at least different between what you see and what we see. Can you post the exact code you used? And can you provide the results of a 'print, !version' command in your v8 PV-WAVE session?? We might be dealing with an oddity that is happening on a specific platform.

Cheers, Don B.

hcrisp
12-03-2008, 09:45 AM
I did as you said and got the same results. A co-worker ran it on both 8.00 and 9.01. 8.00 got the same error I had, but 9.01 worked. I would say it is a version problem. Is there a patch for 8.00? We cannot move to 9.01 just yet.


WAVE> info, !version, /str
** Structure !VERSION, 5 tags, 20 length:
ARCH STRING '80x86'
OS STRING 'Windows-NT'
RELEASE STRING '8.00'
REVISION STRING ''
PLATFORM STRING 'i386nt'
WAVE> fname = 'tst.dat'
WAVE> a = 0.0 & e = 0.0
WAVE> b = 0 & c = 0 & d = 0 & f = 0 & g = 0
WAVE> s=DC_READ_FREE(fname, a, b, c, d, e, f, g, $
- /Column, $
- Delim = [','], $
- Resize = INDGEN(7)+1, $
- Miss_str = [''], $
- Miss_val = [-9999])
WAVE> pm, [[a],[b],[c]]
700.500 12.0000 64.0000
699.500 13.0000 64.0000
674.000 16.0000 129.000
854.500 17.0000 129.000
1012.00 18.0000 129.000
1080.00 19.0000 129.000
0.000000 0.000000 0.000000
0.000000 0.000000 0.000000
0.000000 0.000000 0.000000
WAVE> pm, [[d],[e],[f]]
0.000000 212.300 0.000000
0.000000 212.300 0.000000
0.000000 212.300 1.00000
0.000000 212.300 1.00000
0.000000 212.300 1.00000
0.000000 212.300 1.00000
212.000 0.000000 99.0000
100.000 1.00000 90.0000
0.000000 210.000 0.000000
WAVE> pm, [g]
41
41
56
100
99
100
88
WAVE> info, a, g
A FLOAT = Array(9)
G INT = Array(7)

donb
12-04-2008, 02:33 PM
We do have a PV-WAVE 8.0b patch on our website (here) (http://www.vni.com/products/wave/pvwave/patches.php). Unfortunately when I installed the patch the results were the same as you see in 8.00.

If you are open to using 8.01a, please contact PV-WAVE Technical Support (support@vni.com or 720.407.4201) and we can find a way to get this patch to you. That version returns the correct results.

Cheers, Don B.