Hey everyone!
Until now, I knew the basic nature of how this problem occurs. We noticed and investigated it during SketchUp 2013 development, even going so far as to implement a fix we thought would prevent it from ever happening again. Imagine my surprise at seeing it once or twice during the beta period and now continuing to get support inquiries about it. With no apparent pattern to who sees it or why, I just couldn't figure out its random persistence...until now.
After thinking about it some more this afternoon, I just had an epiphany about exactly what's causing this behavior. Although the only method for preventing this is not something I can readily implement, I'm thrilled to have figured out the root cause of this problem's continued appearance, despite our previous countermeasures.
A (typically) lengthy explanation follows.
Background
When Windows Vista was released, Microsoft changed enough of the Windows internals to break backward compatibility with many legacy programs. This compelled them to create a "compatibility mode" to help users install and run programs made for older versions of Windows under Vista. When in compatibility mode, Windows changes its behavior to mimic an older operating system in an effort to play nicely with software that would not work otherwise. This feature continues in Windows 7 and 8 today and can be very helpful when dealing with software that wasn't explicitly written for newer OSes. An important component of compatibility mode is the "Program Compatibility Assistant" (PCA), a system process which operates in the background to discover and correct incidents of program incompatibility. Think of it as a troubleshooter that keeps an eye on things and gets involved to automatically fix perceived compatibility issues in legacy programs when they arise.
However, if a program was actually written for compatibility with the OS you're running, it's important that the PCA never get involved because enabling legacy compatibility mode for a program that doesn't need it might cause more problems than it solves. An example of such a problem is that which started this thread, where our perfectly Windows-7-compatible installer is being mistakenly launched in compatibility mode and behaves incorrectly as a result. In order to prevent the PCA from interfering unnecessarily, a newly created EXE must include a "manifest" file that explicitly declares which operating systems it was targeted to support.
How the PCA works
The PCA works by monitoring certain attributes of a program's execution. If you launch a program that does not include a manifest stating compatibility with the OS you're running, then the PCA will monitor that program and try to determine if it ever misbehaves. If it detects misbehavior at any time while monitored, then when the application terminates, it will pop up a dialog to ask if you might like to try re-running it in compatibility mode instead. If you ever happen to choose "yes", then the file and path name of the application in question is added to the registry in a section marking those files which should always be run in compatibility mode and then it sets the flag on the file with a default value. After that, unless the "compatibility mode" option is manually disabled, the program will forevermore be launched in a context where Windows masquerades as an older operating system than it actually is. In the cases that concern us, the PCA chooses a compatibility mode for Windows XP SP2, which is older than the SketchUp 2013 installer allows, thereby causing it to abort installation.
Here's the curious part
The SketchUp 2013 installer was explicitly written to include "Windows Vista" and "Windows 7" compatibility statements in the application manifest. Therefore, the PCA service ignores it and will never get involved with our installer on those OSes. That means the Windows 7 PCA isn't responsible for having enabled compatibility mode on that file. Given also that we have yet to find a case of a user going out of their way to invoke compatibility mode and also cannot find any group policies that could be responsible for blindly enabling compatibility mode for the entire system, why then do we see that the tell-tale compatibility setting is enabled and causing installation problems for some of our users?
And now the twist
This is the part I just realized this afternoon that solves the mystery.
None of this is actually a problem with SketchUp 2013. It's actually SketchUp 8's fault.
SketchUp 8 vs. PCA
The SketchUp 8 installer does not contain the application manifest to declare its compatibility with Windows Vista or Windows 7. It isn't that the SketchUp 8 installer is incompatible, but just that the need for a manifest was not known to us back in the time of SketchUp 8, so we never took the proper steps to include it. As a result, when running the SketchUp 8 installer, it's monitored by the PCA and as explained above, if any "misbehavior" is detected, the PCA will pop up a dialog to tell you that it thinks the installer may have failed prematurely due to a compatibility problem and ask if you wish to possibly try it in compatibility mode instead.
Why does the PCA think the SketchUp 8 installer misbehaved? Well, the PCA detects that the program is an installer and therefore assumes if it completes correctly, the "Add/Remove Programs" (aka "Programs and Features") menu will necessarily be changed if the installer runs correctly to completion. Therefore if for any reason the "Add/Remove Programs" menu remains unchanged when an installer exits, the PCA thinks it misbehaved. Upgrade scenarios and even launching the installer and clicking "cancel" are both circumstances in which the installer will exit without changing the "Add/Remove Programs" list, so those are cases where the PCA will probably show up, even though nothing actually went wrong. Then if you happen to dismiss the dialog without reading it (such as by clicking too fast or hitting "Enter" for some reason), the default choice is to affirm that the installation was a failure. Then the file gets marked as needing to run in compatibility mode in the future.
I've saved the best part for last
Here's where it all comes together. You may have noticed that whenever we launch new versions of SketchUp, they're always available at the same web address and are always named something generic like "SketchUpProWEN.exe" or "SketchUpMFR.dmg". There are numerous reasons why we do this which I won't get into, but this is a critical factor in explaining how the SketchUp 2013 installer is being sabotaged by SketchUp 8.
If at any time you downloaded and ran a SketchUp 8 installer by a name like the one above (aka SketchUp 8 M4 or M5, before we launched 2013) and for any reason had the PCA dialog pop up and then ever clicked "yes" when asked if Windows should use compatibility mode for that program, the PCA recorded that file's name in the Windows registry with a note to say that it should always be opened in a default compatibility mode for (Windows XP SP2).
Some time later, you realized your download folder was getting full and confusing, so you either deleted the old SketchUp 8 installer, or renamed it so you could better tell what it was.
Fast forward to today.
If you download SketchUp 2013, you're going to get a file named exactly like the one you got for SketchUp 8, saved to the exact same place as the old one--your downloads folder. Now, when you double-click to launch it, the PCA rears its ugly head and BOOM, you're stuck in Windows XP SP2 compatibility mode for the SketchUp 2013 installer, even though none of us intended it to be so.
It doesn't matter that this is a brand new file that you've never had on your computer before.
It doesn't matter that the contents of this file are nothing like the old SU 8 installer that caused compatibility problems in the past.
It doesn't matter that we've included the correct application manifest inside of this installer, declaring that it's perfectly Windows 7 compatible.
It doesn't matter that you either deleted or renamed the old SketchUp 8 installer and it might not even be anywhere on your computer anymore.
You see, Windows couldn't care less that we've done so much to differentiate this new installer from the trouble-making SU8 installer. The only thing it cares about is that there's a registry entry for a file by that name which stipulates that it must run in compatibility mode. That fact alone puts the scarlet letter on your newly downloaded installer, forcing you into an undesired compatibility mode that SketchUp 2013 doesn't support. It's absolutely ridiculous. If this "feature" were integrated directly into NTFS (imparted at the filesystem level instead of in the registry), or at least if the registry were updated to track any time a tagged file is moved or renamed, neither the installation problem nor this thread would ever have existed)!
Mitigation
If your registry contains a compatibility mode flag for a given file, there are only two ways to prevent it from causing a problem. Either you can follow the procedure mentioned in the previous post and turn off compatibility mode for the file, or you can rename it to fool the registry into forgetting about the compatibility mode.
Really, that's all? Just rename the file Yep.
Correction
I've come back to edit my original post on this point in order to correct an invalid assumption I made regarding the rename procedure. When I first made this post, it was entirely theoretical; I hadn't been able to verify any of the points I stated above. It just suddenly made such undeniable sense of what was previously such a baffling problem that I was certain this newly discovered explanation was correct. Well, I miscalculated one point.
I expected that if I were to tag file as needing a compatibility mode and then rename it, the registry would be updated to follow the file by the new name, meaning that simply renaming a file would never clear the flag. I likewise thought deletion of a file should clear the compatibility flag and assumed the observed behavior of persistence was merely an unintended bug. After all, hadn't anyone considered what would happen if a generic file like "setup.exe" ever got tagged with the compatibility flag? It would affect every file by that name thereafter and cause anarchy! It just seemed obvious that the only way to implement a reliable strategy for implementing compatibility mode would involve updating the flag updated with filesystem changes. But evidently Microsoft and I define "obvious" differently.
Once I finally had a chance to directly test all of these theories, I found that renaming a file actually does clear the flag, which means rename and delete are both working as intended and my guess on that point was wrong. To prevent confusion for future readers, this section is updated to remove those misunderstandings and explain things as they really are.
I contend that Microsoft failed to fully consider all of these possibilities in designing the compatibility mode features. But then again, I'm biased by the fact that this entire problem would never have existed if not for the failure to make the compatibility flag follow the file across renames and deletes.
Download Improvements
Now that I've taken you through the whole explanation about how this problem came about, there's just one last point to make about how we should handle this in the future. A straightforward solution on our end is to change the download name of the files we serve so that the name is different from anything it was in SketchUp 8, thereby preventing the sabotage. Then the next problem to tackle becomes the fact that for numerous reasons that shall remain unexplained, despite being straightforward, it's nevertheless difficult, time-consuming and un-fun to make changes in all the places that have to be coordinated to keep our whole download system working correctly. Yuck.
That's all, Folks!
I hope this helps anyone who runs across this problem again. And know that we will try to find a good solution to this so it won't keep happening forever.
Comments and questions are welcome.
Cheers!
Andrew
