PDA

View Full Version : Restoring XDR Files Via PV-Wave GUI



solargus
09-13-2011, 01:27 PM
I'm trying to build a GUI (my first) that will restore a .sav file, which is written in PV-Wave's XDR format, and plot the variables. Those variables are stored as 2-dimensional arrays and structures, the latter of which contains my date/time structure and an assortment of 1-dimensional arrays.

Here are the two related call backs:

PRO FileCB, wid, data
file = WwFileSeletion(wid, 'FileOkCB', 'FileCancelCB')
END

PRO FileOkCB, wid, shell
COMMON widg, top, rad
COMMON my, specific, variables
file = WwGetValue(wid)
restore, file
END

The restore command would appear to be failing. The only indication of a problem is one message per variable at the WAVE prompt stating, "identifiers can only be added or deleted at the main level: <variable name>"

I've seen this message before in other programs but it did not prevent successful execution of the code, so I'm not sure it's really meaningful here or not. It is interesting to note that Wave was able to report the names of all the variables contained in my .sav file, so it's curious why they're not being established and made available to my radio button call back for selection and plotting in the drawing area.

You'll also need to know that I'm working on a Solaris system running PV-Wave v9.1 with 8GB of available memory.

I can find nothing useful in the documentation, so I hope you can help.

Cheers!

donb
09-22-2011, 08:41 AM
Hello solargus,

The message you receive is quite correct - PV-WAVE SAVE files can only be RESTOREd at the $MAIN$ level.

This might not be optimal for your needs, but if you can pass the SAVE file name into a command file or as a optional aurgument when starting your PV-WAVE session, RESTORE the file, and *then* start up the Widget application, the variables can be reference as part of the Widget app.

Hope this helps,

Don B.

solargus
09-22-2011, 12:18 PM
Don,

Thanks so much for your reply. I really appreciate you tackling this one with me.

I have to admit that this comes as quite a surprise, if I understand you correctly. Are you saying that the GUI can't restore (to my novice point of view, a fancy read statement) a native PV-Wave format???

It did *partially* succeed in restoring the file, but instead of getting my arrays, what I got was a single constant, which was the first value in my arrays. It threw out the remainder of each array.

donb
09-23-2011, 10:05 AM
The SAVE/RESTORE file format is designed and implemented to be used only at the $MAIN$ level. The rationale here is that you're saving some or all (based on Keywords used in the SAVE command) information, variables, system settings, etc. of your current PV-WAVE session. Due to the types of things involved it doesn't make sense to do a RESTORE any place other than the $MAIN$ level.

While the format and content of the file is well known to PV-WAVE, and we *could* read it at some lower level routine within the call stack, variables created within a particular routine are deleted when that routine exits. $MAIN$ level variables however are persitent to the PV-WAVE session; and by using COMMON blocks or the UPVAR/ADDVAR routines, all other routines have access to the $MAIN$ level variables.

I understand what you're trying to do - within a PV-WAVE Widget GUI, allow the user to specifiy a file that contains some number of variables that s/he wants to use during the GUI execution. That being the case, the best recommendation is to use something other than a SAVE command when you create the file. The DC_WRITE_FREE or DC_WRITE_FIXED are easy to use; they create ASCII files, and the corresponding DC_READ_FREE/DC_READ_FIXED routines are then used in the GUI File Open callback routine.

If you're dealing with very large data sets, then a binary file format is more reasonable. For that, use the OPEN/WRITE routines, and if you have to have platform indepenence (create the file on a Solaris, and then read it on a Windows or VMS machine for example) then use the /XDR keyword. This ensures that the file is created in the XDR format (similar to SAVE) files and can be read by any PV-WAVE-supported platform. Then in the GUI File Open callback, use the OPEN/READ routines to make the data available.

Now, the routines mentioned above do require a little more knowledge of the type of data, and names of the variables if you need them, when you go to read the routines. A very typical technique is to create an associated file that contains this information, or you can add the information at the start of the file during the write.

I can post a simple example later that uses a combination of READ/WRITE and DC_WRITE_FREE/DC_READ_FREE to store and retrieve both variable names and data.

Cheers, Don B.

solargus
09-26-2011, 06:17 AM
Don,

You've hit it exactly. I'm working with very large data sets, some of which can exceed 1 GB and 10 million data points per file; Thus the need to store it in XDR binary format.

The simple example you mention would be much appreciated.

Cheers!

donb
09-29-2011, 02:31 PM
solargus,

Sorry for the delay. I’ve been spending a fair amount of time thinking about what to post (more time than I should :-)), and what to provide as an example. I’ve also been chasing down some clarification about when and how you can use SAVE files for such a concept. The conclusion is: this is a highly data-dependent exercise, and while a simple example is rather easy, the real world application requires more coding then is normally allowed in this type of exchange (this User Forum as opposed to working directly with our Technical Support group).

So here’s the basic concept for creating XDR files, and then reading them again. Let’s say you have 2 variables created in a PV-WAVE session; or the data exists elsewhere and then you read that data into PV-WAVE. The next step is to create a binary file, written in XDR format, that can be later read into your GUI application. The size and type of the variables don’t matter for the file creation, but matters greatly when you go to read the file later.

So to create the file, these are the commands:

OPENW, lun, ‘filename’, /Get_Lun, /XDR ;;;; creates the file AND specifies that you want the format to be XDR
WRITEU, lun, v1, v2 ;;;; This is an unformatted write (binary) where you provide the list of variables to be written
FREE_LUN, lun ;;;; This closes the file and releases the 'lun' back to the system pool of luns

If you know these variables are a 1024x768 byte array (an image for example), and then a one million element floating point array, then the read is equally simple. Once you get the filename from your user within the GUI, pre-define the size and type of the variables and do the read:

OPENR, lun, fname, /Get_Lun, /XDR ;;;; make sure you add the XDR keyword
v1=BYTARR(1024, 768) ;;;; create the image array
v2=FLTARR(1000000) ;;;;; create the floating point array
READU, lun, v1, v2 ;;;; read the 2 variables from the file
FREE_LUN, lun

The real catch is that for any binary file, you must know *exactly* what is in the file, and how many variables (both type and size) you need to predefine. You can use another text file, created at the same time as the binary file, to save this information. You’ll then have to parse that file and use the appropriate variable creation routines in the READ area of your code prior to reading the file itself. Specifically within a GUI application, and when the file might contain practically any type and size of data, the EXECUTE command is the way to go. You build up a string which is the variable creation command [[[[[cmd=“v2=” + STRING(type(0)) + “(“ + STRING(size(0)) + “)”, where type(0) is ‘FLTARR’, and size(0) is ‘1024,768’ read from the text file, and the final string for cmd is ‘v2=FLTARR(1024,768)’)]]]]], and then send that string (cmd) directly to the PV-WAVE parser using EXECUTE.

So, now on to the SAVE findings….it was nagging me that RESTORE can only be used at the $MAIN$ level. And I re-read your comments which had me studying the docs for both SAVE and RESTORE. There is no such defined restrictions in the doc. The next step was to spend some time with the developers asking why. I also knew that we have other areas of PV-WAVE, and other applications built with PV-WAVE, that take advantage of SAVE files.

It turns out that you CAN use RESTORE outside of the $MAIN$ level; so I was wrong in my previous post! The details of how this works goes very deep into the PV-WAVE kernel, and has a lot to do with temporary variable creation, and the complexity of all of that is the most feasible explanation for why this has not been added to the documentation. That said, I’ll file a documentation Change Request and see what we can add in a future release.

The key point in using RESTORE somewhere other than the $MAIN$ level is that the name of the variable to be read (or names of the variables to be read) MUST exist at the $MAIN$ level before you execute the RESTORE command. The size or type of the variable doesn’t matter – it’s only the name. For instance, if there are always 3 variables in you SAVE file, and they are called v1, v2 and v3, you can add this line to your PV-WAVE startup file:

v1=0 & v2=0 & v3=0

The names are now defined. When you do the RESTORE in a lower-level routine, it reads whatever is in the SAVE file into that variable. It doesn’t matter if v1 is actually a 1000 element array of date/time structures, or v2 is an associative array of 500 frames of an animation. As long as the names exist at $MAIN$, the RESTORE works.

We take advantage of this fact in some of those applications I referenced above. In our case, we have a HUGE data structure that contains every attribute associated with a multi-page, multi-plots on a page program that is all maintained in a single PV-WAVE variable. So when we SAVE the file, we write out ‘very_complex_structure’ as the name, define that name at the $MAIN$ level when the application begins, then use the RESTORE.

Of course, when you execute the RESTORE, the variable in read and available only as a local variable. You have to use a COMMON block to pick up the real variable(s) content OR you can use ADDVAR to send the real variable(s) content back up to the $MAIN$ level. If you don’t do one of these two things, the variable is deleted when the local routine exits…..

So, quite the long post! I’m happy to answer follow-on conceptual questions, but really do recommend that you contact our Technical Support group (pvwavesupport@roguewave.com) as you start working through the suggestions or ideas in this post.

Cheers, Don B.

solargus
09-30-2011, 06:09 AM
Don,

I think you're on to something with your point about establishing variables at the top level, first. That should be easy to do, because I know before hand what will be inside a particular .sav (XDR) file.

Thanks for being so tenacious on this one! I think I can take it from here.

Solargus