PDA

View Full Version : Memory leak?



hcrisp
05-14-2009, 01:27 PM
I'm running several analyses in batch mode (no interactivity) and getting the following errors:



90% of the maximum number of temporary variables is allocated. Save user data, close and reopen all active GUI tools. To improve performance exit and restart PV-WAVE.

95% of the maximum number of temporary variables is allocated. To save the current session enter the PV-WAVE comand SAVE, /All. To restore the saved session enter the PV-WAVE command RESTORE. *** Continued working can result in loss of user data.


I think it is coming from deep within PV-WAVE (not a .pro file). What temporary variables is it talking about?

I'm afraid calling SAVE and RESTORE is not an option. This appears to be a memory leak of some sort. Is there anything I can do to pinpoint where the leak is happening?

rwagner
05-18-2009, 11:13 AM
Hi Hcrisp,
Try taking a look at the documentation for the .locals, .size, and ..locals executive commands to resolve this problem. Increase the memory allocation for them above the system defaults.

The temporary variables are usually created during execute statements that can occur either in your code or in system PV-WAVE procedures.

<doc snippet>
.LOCALS
The syntax of the .LOCALS command is:
.LOCALS local_vars common_symbols
The .LOCALS command is similar to the .SIZE command, in that it resizes the data area (the data area is described in the previous section, Using .SIZE). The .LOCALS command, however, lets you specify the data area size in terms of local variables and common block symbols rather than in bytes. This command affects the size of the data area for the $MAIN$-level (commands entered from the WAVE> prompt), and the initial size of the data area for compiled procedures and functions.
The two parameters are positional, but not required. If you execute .LOCALS with no parameters, the data area is set back to its default value, which is 500 local variables. If you want to use 700 variables at the $MAIN$ level, enter:
.LOCALS 700
.LOCALS clears and frees the current $MAIN$ data area and code area. It then allocates a new code area of the same size as the previous one and a new data area of the specified size.
For compiled procedures and functions, the compiler initially allocates code and data areas of the same size as those that $MAIN$ is currently using. If you get compiler error messages stating that the code and/or data area of a procedure or function is full, you must first make the $MAIN$ code and/or data areas larger with the .SIZE or .LOCALS execu tive command. Then when you recompile the procedure or function, the compiler starts with the larger code and/or data areas.


..LOCALS
The syntax of the ..LOCALS compiler directive is:
..LOCALS local_vars common_symbols
This command is useful when you want to place the EXECUTE function inside a procedure or function. EXECUTE takes a string parameter con taining a PV-WAVE command. This command argument is compiled and executed at runtime, allowing the possibility for command options to be specified by the user. Because the data area is compressed after compila tion, there may not be enough room for additional local variables and common block symbols created by EXECUTE. The ..LOCALS command provides a method of allocating extra space for these additional items.
The ..LOCALS compiler directive is similar to the .LOCALS executive command, except:

..LOCALS is only used inside procedures and functions.

Its arguments specify the number of additional local variables and com mon block symbols that will be needed at ?interpreter? time (when the already-compiled instructions are interpreted).

It is used in conjunction with the EXECUTE function, which can create new local variables and common block symbols at runtime.


.SIZE
The syntax of the .SIZE command is:
.SIZE code_size data_size
The .SIZE command resizes the code area and data area. These memory areas are used when PV-WAVE programs are compiled. The code area holds internal instruction codes that the compiler generates. The data area, also used by the compiler, contains variable name, common block, and keyword information for each compiled function, procedure, and main program.
After successful compilation, a new memory area of the required size is allocated to hold the newly compiled program unit.
By default, the size of the code area is about 800 bytes, and it grows dynamically as needed to accommodate the activity of your session. The initial size of the data area is 8,000 bytes (enough space to hold 500 local variables).

NOTE
Resizing the code and data areas erases the currently compiled main program and all main program variables.
For example, to extend the code and data areas to 40,000 and 10,000 bytes respectively:
.SIZE 40000 10000
The upper limit for both code_size and data_size is over 2 billion bytes.

</doc snippet>


-Ryan

donb
05-19-2009, 07:06 AM
Ryan is correct in that using .SIZE, .LOCALS and ..LOCALS can increase the system limits. In particular ..LOCALS is very important when creating variables at execution time (as opposed to compile time) in your routines with the EXECUTE command.

I suspect however that increasing these limits will simply prolong the messages you are seeing about temporary variable usage.

Temp variables in PV-WAVE are created, manipulated and generally freed with practically very command. For instance, the assignment "a = 5" causes a temp variable creation for the constant '5'. Likewise, the command "x = y(5:10) + sin(z)" creates 3 temp variables; one for the result of sin(z), one for the results of the array subset of y [y(5:10)], one for the results of the addition of the two previous results, then the final result is transfered into x, and then all three of the temp variables are freed.

The messages you are seeing are not the results of a memory leak in PV-WAVE. They are generally the result of something going on in your code that is causing PV-WAVE to hang on to some intermediate results (temp variables), and the code is continually asking for more temp assignments that also don't get released.

We'll look through our Tech Support data base to try to find some simple code examples that demonstrate this behavior and post them here as a follow-up.

Don B.

donb
05-19-2009, 03:43 PM
Below are comments from a Ticket in our Tech Support database that explains temporary variables quite nicely:


PV-WAVE allocates temporary variables 100 at a time, up to 100,000 temporary variables (hard-coded limit). When the initial 100 free temps allocated at startup are used up, more memory is allocated for a new set of 100, but this shouldn't happen too often; you should always see the number remain stable.

The MEMORY() function returns a four-element array. The first three elements match "INFO, /Mem" output, but the fourth element is the number of free temps.

m = MEMORY() returns:
m(0): Dynamic memory usage
m(1): Heap memory in use
m(2): Number of calls to Malloc
m(3): Number of free temp variables

Therefore, add the following PRINT statement before and after where you believe the temporary variables are being gobbled up.

PRINT, "Free temps available: ", (MEMORY())(3)

If the number of temporary varaibles does not remain stable before and after the function call, then there probably is a problem with the function not freeing temporary variables.

This technique can be used to examine your code as well as individual PV-WAVE kernel routines. For example, if you wanted to check if the MAX function is releasing temp variables then here's a very simple test case:


PRO tmptest, n
IF NOT PARAM_PRESENT(n) THEN n=20
PRINT, "Free temps available before the call: ", (MEMORY())(3)
FOR i=0L, n DO BEGIN
m = MAX([0,1,2], Dim=1)
ENDFOR
PRINT, "Free temps available after the call: ", (MEMORY())(3)
END

Add these print statements to various areas within your PV-WAVE application - and then narrow the scope to something reasonable. If you can get this down to a few PV-WAVE routines that are suspect, we'll gladly take if from there and dig into the underlying PV-WAVE kernel code.

Let us know what you find out,

Don B.

hcrisp
05-27-2009, 11:47 AM
Thanks, Don, that appears most helpful. I will have to probe into my code to find the culprit.

I take it that there is no function which can return the total number of temporary variables used (out of 100,000)? I can forsee problems if a call consumes more than 100 temporary variables. For example try the with your code; it makes it look like it frees memory:



tmptest, 180
; Free temps available before the call: 53
; Free temps available after the call: 72


Lastly, why does the addition of a keyword in MAX consume temporary variables? If you drop the DIM keyword, then tmptest.pro becomes stable.



...
m = MAX([0,1,2])
...
tmptest
; Free temps available before the call: 72
; Free temps available after the call: 72

donb
05-28-2009, 11:10 AM
As far as I know, there is no function call or system variable available to check on remaining number of temp var space. And using more than 100 temp vars in a single routine is not an issue; when the routine terminates the temp vars are released back to the pool.

As for the last question, as mentioned previously, almost everything in PV-WAVE results in allocation of temp vars. Even before MAX is called, the keyword definition itself (Dims=1) results in a temp var created for Dims, and one created for the constant '1'. In fact, the use of the "[0,1,2]" syntax in the call to MAX results in at least 4 temp vars (one for each constant and one for the array to hold the constants). And once inside MAX, temp vars are created to hang on to the value of Dims as the incoming local keyword vars. That's why you see an increase when the keyword is added.

Temp vars are usually a non-issue - they get created ALL the time and released ALL the time. It's a normal and rather important byproduct of the PV-WAVE interpreter.

Thanks for the question, and let us know when you find areas in your code (or in PV-WAVE code for that matter!) that you'd like us to examine for temp var usage and potentially not being freed/released.

Don B.

hcrisp
05-28-2009, 12:43 PM
Don,
Thanks for the explanation, but I'm still confused. You said, "when the routine terminates the temp vars are released back to the pool." Why does MAX not release the temporary variable when the DIM keyword is used?



PRINT, "Free temps available: ", (MEMORY())(3)
; Free temps available: 175
m = MAX([0,1,2])
PRINT, "Free temps available: ", (MEMORY())(3)
; Free temps available: 175
m = MAX([0,1,2], DIM=1)
PRINT, "Free temps available: ", (MEMORY())(3)
; Free temps available: 174


Is not this an instance of memory leak? I think my problem is many small memory leaks like this adding up to 90%+ leak of available temporary variables.

donb
05-28-2009, 02:09 PM
hcrisp, what version of PV-WAVE are you running, and what platform? Are you executing any other code prior to the code in your last example?

I've run the 3 PRINTs and 2 MAX commands on version 8, 8.5, 9 and 9.01; all on Windows. The PRINTs return the value 98 in every case...

Don B.

hcrisp
05-28-2009, 02:43 PM
I am running 8.00. I have a lot of initialization code that runs when I start WAVE. If I run the commands without the initialization code, it returns 98 every time like yours. What could my initialization code be doing to affect the release of temporary variables?

donb
05-29-2009, 01:58 PM
We can't really provide a good answer without being able to examine your initialization code. Can you send that to Support (support@vni.com) so we can take a look?

hcrisp
06-01-2009, 09:43 AM
I'm afraid I can't do that without an NDA since the code is proprietary. Can you explain in general terms what possible scenarios would cause a temporary variable not to be returned?

donb
06-02-2009, 12:17 PM
We've spent a fair amount of time trying to come up with a simple answer to your question ('general terms' and 'possible scenarios'). Unfortunately we didn't arrive at anything concrete. Practically everything we tried returned the temporary variables as expected. We did find something that looks suspicious in the VDA Tool area that will be filed as a CR (Change Request) to be checked in the next release; however you stated your application is batch-based so that doesn't seem to be a match for what you've reported.

So next possible steps include:

- In lieu of seeing your code, tell us what functional areas within PV-WAVE are included in the initialization code. (XML, WaveHandles, plotting, mapping, images, etc.)?
- You're running v8.0. We've had 4 key releases since then with a big focus on addressing possible memory leaks. If you have access to v9.01, run your code with this latest version to see if you get the same results.
- Add the PRINT statements in various places within your initialization code to help us determine areas that might not be releasing temporary variables.

We really want to help figure this out and we'll need your assistance.

We also have to remember the purpose of this Forum - to assist with general questions related to VNI products. The Forum is not meant or intended to be a replacement for Technical Support access through our paid annual maintenance program, the Software Update and Subscription Service (SUSS).

Our final recommendation is that you email (or call) Support (support@vni.com) and log a Ticket for this issue. We ask that you provide the results from the bullet list above to our Support Engineer. If you prefer, you can email me (donb@vni.com) or Margaret Journey (margaret@vni.com) to provide the results.

For other people reading this thread??If the end result turns out to be an issue internal to PV-WAVE, we?ll submit a post to this thread to close the loop and inform the other readers.

Best Regards,

Don B.

hcrisp
06-02-2009, 02:22 PM
I believe the problem is coming from WwMenuBar. I spoke of using batch earlier, but I discovered the loss of temporary variables while testing the MEMORY function in interactive mode. So, after doing painstaking work to isolate the error, here is how to reproduce the problem:

Create the following functions:



FUNCTION getmenu
..LOCALS 2000
str = STRARR(2)
st = DC_READ_FREE('menu3.txt', str, RESIZE=[1], DELIM=['|'])
strconcat = STRJOIN(str, '')
st = EXECUTE('m = ' + strconcat)
RETURN, m
END

PRO ttest
PRINT, "Free temps available before the call: ", (MEMORY())(3)
m = MAX([0,1,2], DIM=1)
PRINT, "Free temps available after the call: ", (MEMORY())(3)
END


You will need unzip the attached file, 'menu3.zip' to 'menu3.txt'. This is my actual menu structure with the menu names obfuscated to 'junk'.

This code, executed at MAIN, will reproduce the error:



ttest
; Free temps available before the call: 98
; Free temps available after the call: 97
ttest
; Free temps available before the call: 98
; Free temps available after the call: 97
w_top = WwInit('App', 'App', w_workarea, /FORM, $
TITLE='Title', LAYOUT_NAME='Main', CONFIRMCLOSE='AppCB')
ttest
; Free temps available before the call: 98
; Free temps available after the call: 97
ttest
; Free temps available before the call: 98
; Free temps available after the call: 97
m = getmenu()
ttest
; Free temps available before the call: 98
; Free temps available after the call: 97
ttest
; Free temps available before the call: 98
; Free temps available after the call: 97
w_menu = WwMenuBar (w_workarea, m, $
Menus=w_panes, /Top, /Left, /Right, $
Layout_Name = 'Menu', /HelpJustify)
ttest
; Free temps available before the call: 59
; Free temps available after the call: 58
ttest
; Free temps available before the call: 58
; Free temps available after the call: 57
; Note the call to ttest does not return the temporary variable now!
ttest
; Free temps available before the call: 57
; Free temps available after the call: 56


As to the use of the Forum: I do not mean to waste space with posts that are about problems with my particular code. However, I post in the hope that someone in the user community or support group has run across the type of problem I am facing and will be able to help me. Plus, it may enable you to identify bugs you otherwise would not.

donb
06-03-2009, 10:10 AM
Hi hcrisp,

We see the same results when we run PV-WAVE v8.0.

More recent versions of PV-WAVE (v8.5, v9, v9.01) do not show this behavior and the temporary variable count is stable.

Cheers, Don B.

hcrisp
06-03-2009, 10:59 AM
Thanks, I am glad to hear that the problem has been dealt with in WAVE 9.01. The goal is to migrate our application to that version in a couple months, so now we just have extra incentive to do so.

Just to close the loop on the initial post -- I debugged the batch code using the same slow technique of using PRINT, (MEMORY())(3) and found the offending line. There was a FOR loop that called WHEREIN to check if a scalar existed in another array. Something like this:



PRO memtest

t = 64.D
m = INDGEN(37) + 60
PRINT, 'before', (memory())(3)
FOR i=0L, 20 DO BEGIN
PRINT, 'in loop: before', (memory())(3)
id = WHEREIN(m, t)
PRINT, 'in loop: after', (memory())(3)
; This leaks 1 temporary variable!
ENDFOR
PRINT, 'after', (memory())(3)

END


The loop was running 6000+ times each for 50+ files and thus had the potential to leak up to 300,000 temporary variables. I solved the problem (in 8.00) by changing the WHEREIN call to a WHERE call which does not result in the loss of temporary variables. (I can't say whether WHEREIN usage would be stable in 9.01, but it could be tested to see.)



...
id = WHERE(m EQ t)
...

Margaret Journey
06-12-2009, 06:48 AM
We tested your sample code using wherein with PV-WAVE 9.01. Unfortunately there does appear to be a memory problem with this function. We will look at this in more detail and let you know when we have a resolution.