I'm constantly in a similar situation even though I never get that exact same message, and have done a bit of testing around.
To my understanding, the core resource that is under stress here, is the GPU memory. But this doesn't necessarily indicate that you're running out of this resource in general. It might just mean the Desktop Window Manager has detected that you're running so low that one specific feature of this service could be disabled to free up more memory.
To test this, I started opening up GPU memory intensive applications:
This is somewhat above average use on my triple-screen setup with a few key players running [Visual Studio 2012 [hardware acceleration enabled], PhpStorm, Aptana Studio, Chrome, Firefox, IE, ...]. So, yeah, if you only have a 1 GB card and this would be your usual usage scenario, you'd already have a problem.
I had to push a bit further and start a couple more Visual Studio instances...
...until it was nearing the 1.5 GB mark and...
SNAP! This caused Windows to kill desktop compositing completely [and free up some precious resources].
Now, when I'm already at a critical level, and I start an application that uses excessive amounts of GPU memory in fullscreen, I can even go beyond this critical limit. Here's the result after running Black Mesa for a while on 2560x1440 at max details:
So, two things can be deducted from this. While going over the 75% mark on the desktop can force
Windows to disable desktop composition, when reaching the same limit in a fullscreen application [and, optionally exiting that application] desktop composition is not disabled.
The second thing is, while you're in your game, you can get the idea "Hey, I have enough resources to run this game, why don't I have enough resources for the desktop?". The reason is, both require memory at the same time.
Windows might only be able to tell you about the memory situation after you exited the game. So, when I start another Visual Studio after exiting Black Mesa...
So, what can we do about this?
Get more GPU memory
Sweet and simple.
Disable Desktop Composition [per process]
As was already suggested, you can disable desktop composition for a single executable. This has the effect that desktop composition is temporarily disabled while the executable is being
executed. This drastically reduced the overall memory consumption while the application is executed in my tests:
The bounty notice mentions that this is undesirable, as this is potentially a lot of work.
Disable Desktop Composition [globally]
I wouldn't consider this a solution as desktop composition is usually desired. But this is where to disable it:
I just want to get rid of the annoying message!
Just because you remove the warning "Your battery is almost empty! You can only talk for 10 more minutes!" does not mean you can talk for longer than 10 minutes. In fact, your phone will most likely simply shut off and that's it. Now how's that for an improvement?
I never assumed the message can be turned off and I don't see how that could be considered a benefit.
You might think you know better, but you don't. If the system is telling you it's running out of resources, it is.
But it isn't! I know!
Ok, let's assume Windows is just not smart enough to detect the special situation you're in and the warning message is simply an annoying inconvenience. What now?
The thing is, me personally, I'm also affected by this and it's annoying the hell out of me. Because I don't even get that warning message. Windows simply switches my color profile and that's it. And I like that actually.
When it happens I usually quickly run a script that calls
net stop uxsms & net start uxsmsHere is the full script. It will pop open a UAC prompt and restart the service so it can be triggered from anywhere. I have it pinned to the start menu.
@if [1==1] @if[1==0] @ELSE @echo off&SETLOCAL ENABLEEXTENSIONS >nul 2>&1 "%SYSTEMROOT%\system32\cacls.exe" "%SYSTEMROOT%\system32\config\system"||[ cscript //E:JScript //nologo "%~f0" @goto :EOF ] NET STOP uxsms NET START uxsms @goto :EOF @end @ELSE ShA=new ActiveXObject["Shell.Application"] ShA.ShellExecute["cmd.exe","/c \""+WScript.ScriptFullName+"\"","","runas",5]; @endThis restarts the Desktop Window Manager and brings me back to my composited desktop [and it frees up lots of resources in the process, yay].
Knowing this, you can also construct yourself a special gaming environment where you stop the service before starting the game to combat this whole behavior. However, this will cause identical behavior to disabling desktop composition for a single executable through the file properties.