In the first part of this series, I have introduced you to some of the fundamental changes in Windows Phone 8 on the operating system level. A lot of the improvements however go far beyond that and affect not only the local situation on devices, but the surrounding infrastructure and phone ecosystem as a whole.
With this, I not necessarily mean the great improvements that have been introduced for the Windows Phone Store [1] or in the field of business opportunities – there's also a lot in the box on a very technical level that's not obvious to a quick passing view. One of these things is what I want to talk about today: Compile in the Cloud.
A Quick Look Back
To understand what has been changed and accomplished by that new feature, we first have to take a look back at how things are done on Windows Phone 7, or for that matter, on all other .NET platforms. When you use Visual Studio to build e.g. your C# code, the compiler does not produce a binary that contains machine code. Instead, your source code is translated into something called Intermediate Language (IL). Only when your code is about to be executed will a so-called just-in-time (JIT) compiler do the final step and translate the IL code into something that can be understood by and executed on the target machine architecture. This alone is a pretty fascinating detail, as the JIT compiler is invoked on a per-method basis as soon as the method in question is invoked. For subsequent calls to the method, the previous results are cached and reused, but only until the application is shut down! Then, all the compiled code is simply thrown away and the whole process starts over the next time the application is launched (we like to refer to this as the "Groundhog Day" effect :)).
Now the first question that pops into ones mind is: why would you want to do something like that? It seems like a lot of unnecessary extra work at first, does it? The truth is that this has quite some advantages in a lot of situations. First of all, Intermediate Language is a CPU-independent instruction set. This means that you can pass around the IL code as is, and as long as there's a JIT compiler that knows how to translate it to machine code, it'll work on all platforms without modification. Second, the fact that compilation happens every time the application is executed also means that it is built against the current version of installed dependencies (namely the .NET runtime itself) from scratch on each run – meaning that changes to that runtime will not break the application as long as the public API stays compatible. There's also some inherent optimization to the concept of just-in-time compilation: code that is not needed during execution also will never be compiled. And finally there's some other features that IL and the included metadata enable, for example enforcing code security and code validation. If you're interested in it, you can easily take a look at what the IL version of your code looks like using tools like ILdasm.exe [2]:
If you're scared by what you see, don't worry. If you never learned assembly language it's quite natural that it looks all gibberish.
One of the drawbacks of JIT compilation is that it adds some overhead at execution time. As you can imagine, things like parsing metadata, enforcing code validation, setting up dummy methods during load time (stubs) and compiling them into real code on the fly all come at a price. On desktop computers, this price usually is easy to pay, as the computational power is sufficient to do all these things in a blink of an eye. However, with limited resource like – let's say – on a mobile device, these penalties suddenly become very relevant, and no matter how optimized the mechanism is in terms of performance, we're suddenly talking about seconds instead of milliseconds. Additionally, we are even facing some kind of vicious circle here: on a smart phone, the JIT compiler should perform as fast as possible. However, fast usually means producing less optimized code, which on the other hand would result in additional benefits for applications. All in all, the clever concept of IL and JIT compilation is one of the reasons on Windows Phone 7 why we see relatively long start times of applications, sometimes in the range of several seconds. Something had to be done here, and Microsoft did :).
A Solution?
A seemingly obvious solution to the problem is to skip the IL and JIT part altogether and simply produce machine code, like it was and still is done in other languages today. That would eliminate issues like long start times of apps immediately, and even result in a performance boost at runtime too, because no code would have to be compiled on the fly. The idea of this is nothing new to the .NET world. In fact, a tool named NGEN has been around since the very early days of .NET [3]. NGEN generates platform-specific machine code from your IL assemblies that then can be used without the need for a tool like the JIT compiler at all.
There are however some problems with this. One is that generating those native images is slow even on powerful machines. A prominent example of software that actively makes use of NGEN is the free image processing software Paint.NET [4] which incorporates an optimization step with NGEN as part of its installation. In some situations it's possible that this optimization takes several minutes to complete. Can you imagine the time it would take to do the same on a phone?
A more subtle, yet even more important detail can be found in the above linked page with the NGEN documentation:
"Images need to be regenerated when the original assembly or one of its dependencies is serviced."
The first part is somewhat obvious; when I update the original application, the optimized image needs to be recreated. However, the crucial part is the second one: when a dependency (read: .NET runtime components) are changed, the image needs to be recreated also. As a consequence, if we compiled apps to native code, updates to the runtime on the phone inevitably would require to recompile all installed apps on that device too. If optimizing one app potentially could take dozens of minutes, then recompiling your collection of a hundred apps would probably make you go berserk…
A Better Solution!
After this somewhat lengthy introduction on the topic I think you have already guessed that the real solution to all this must be the "Compile in the Cloud" feature that is mentioned so prominently in the headline already. However, simply moving the compilation of IL to native images (NGEN) to the cloud would be a weak solution that introduced all sorts of new problems. For example, even if we used the power of the cloud to do the heavy lifting, we still would need to download that newly compiled image to our phones. Re-downloading a hundred apps after a platform update? I don't think so. So, if you were a clever Microsoft employee and faced all these problems:
- JIT is too slow, we can't (continue to) do that
- Compiling to native images locally is way too slow, people would hate us
- Re-downloading pre-compiled native images all the time also is impracticable, no chance!
Then what would you do? The correct answer is: simply create yet another language that we can compile to and that solves all these problems :). Ladies and gentlemen, please welcome MDIL – Machine Dependent Intermediate Language. Here is the process for this new concept:
To understand why the solution here was to develop a new language, we first have to fully understand the reason why native images need to be regenerated their dependencies are changed. In the process of creating these images all kind of information about the physical layout of references is baked into the result, in particular address offsets. That data potentially becomes stale when those dependencies are changed, for example when new members are added to them. So despite the fact that an API may stay very similar and fully compatible to its previous version, the physical layout of that code can change and cause all sorts of problems up to very severe data inconsistencies and even corruption. This is when MDIL comes into play. It is a format that somewhat resembles the final machine code, but it has placeholder tokens in all the places that usually would contain the hard-coded literals that potentially cause problems. This concept allows to have an "almost done" native image that can be finished in a final, very lean step directly on the phone. Once any dependencies change, that final step can easily be repeated in little time. The actual creation of the MDIL file (which is the heavy lifting in this process) can be performed in the cloud. When a client installs an app from the Phone Store, he already receives that prepared MDIL image from Microsoft's servers.
How You Can Benefit From This
What do you as a developer have to do for your app to support this? Literally nothing. That is one of the beautiful details of this feature: the initial compilation to MDIL is performed the moment when you upload the application to the Store. You don't have to do anything to make this work, it happens automatically behind the scenes for you. Also, you don't have to worry about problems with users who still run Windows Phone 7.x. Supposed your app supports these platforms of course, those users receive an unaltered version of your app package that is JIT compiled on the device just like before. Only users of Windows Phone 8 are provided with the optimized, pre-compiled MDIL version of your app.
The result of this new feature can immediately be seen when you run an app on a Windows Phone 8 device and compare in particular the startup time to its behavior on Windows Phone 7 devices – improvements of 50% and more are absolutely possible.
More Details
If you want to test your app as an optimized native image before submitting it to the Windows Phone Store, you can do that by following some simple steps. It requires a physical device though and cannot be performed in the emulator. Details on this can be found in the Windows Phone 8 SDK documentation "How to test the retail version of your app for Windows Phone 8" [5].
A lot more information on the topic can be found in both the Build session "Deep Dive into the Kernel of .NET on Windows Phone 8" [6] as well as the Channel 9 Going Deep show "Inside Compiler in the Cloud and MDIL" [7]. The latter one features Peter Sollich, who also is listed as the inventor of MDIL in Microsoft's patent application for it [8 (PDF)].
Summary
Compile in the Cloud is one of these hidden gems in the set of new features of Windows Phone 8 that most likely won't be noticed and talked about a lot outside of a rather small, interested community. I'm pretty sure that a lot of developers even will never learn about this and frankly, they don't need to – which is why it is such an elegant solution. To the marketing people, it's probably simply "hey we've improved startup times so much it'll drop your jaw". To me, exploring the details of the technology once again was one of these magic moments that make me realize how much effort and thinking goes into pushing forward the platform and evolving the concepts that have been around for ten years now. Fascinating, isn't it?