PDA

View Full Version : Problems with Data Execution Prevention



utesch
06-23-2008, 01:57 AM
Hi,

we stumbled over a problem with RWUXTheme dll's in combination with the Data Execution Prevention.

You can reproduce it with the ThemeColor sample from the Stingray Studio 10.0. The only thing which has to be done is to turn on Data Execution Prevention for all programs and add the ThemeColor executable to the Exception list (so that it should not get protected by DEP)

Now it's impossible to startup the ThemeColor executable. If you run it in a debugger you will get an access violation in the uxtheme.dll

The really interesting thing is if you switchback the DEP to only protect essential Windows programs without removing the exception the problem still exists. :confused:

Regards,
utesch

Terry
06-24-2008, 04:51 PM
The problem is that UXTheme.dll is being loaded during static initialization. This happens because there is a static SEC_AUX_DATA object whose constructor calls SEC_AUX_DATA::UpdateSysColors(), which calls RWResetO2K3Colors(), which calls RWIsAppThemed(), which calls RWInitUXThemeFunctionPtrs(), which attempts to load the UXTheme DLL.

To fix this, you can comment out the call to UpdateSysColors() in RWUXTheme.cpp, line 778. You will need to rebuild the RWUXTheme library.

This was fixed in Stingray Studio 10.0, but this part of the fix was inadvertently merged out. The other changes necessary for this to work are in place.