All New Code?

Why is it that people believe that every release of Windows is entirely new code? I’ve never seen anyone from Microsoft ever claim any such thing, but every release I see people talking about the claim. Having said that, in every OS release almost every component gets touched if just to fix potential security vulnerabilities found by automated tools. That’s the advantage of a full OS release, you get the most complete testing cycle Microsoft can manage (internally and externally). Let’s see if I can introduce a lexicon for people to talk about OS release changes. Here are some categories to count and measure:

  1. Absolute Development Time – Each release only has so many developer resources for a period of time, so even if it’s just cleaning up almost invisible implementation issues, or major new features there is a an absolute amount of effort put in to each OS release. While people talk about vista in terms of 5 years since XP, the reality is that most of the windows organization for a bunch of that time was focused on the first and especially the second XP service pack.
  2. Subsystem Replacements – Instead of incremental changes to a couple components, this implies major rewrites and replacements. Windows ME to XP involved replacing the the windows 9x OS with the Windows 2000/NT codebase especially at the lower levels of the OS. Much of that code had been shipped and tested as Windows NT and Windows 2000, so for the development team this was incremental work, but for the consumer OS customers is was a new code base with all the pain involved. IIRC a decent amount of Windows ME was getting the driver ecosystem compatible with the Windows 2000 codebase so that Windows XP wouldn’t be as painful of a switchover. (There is a lesson here, you got to ship an OS which will get a negative reputation to move the market whenver making major changes that affect drivers, 64bit Vista is playing that role right now for future 64bit Windows OS versions). In Vista, there were at least three major subsystem replacements, the video, audio and networking stack each got rewrite/replacement level changes. The primary motivation for a subsystem replacement is to provide an better foundation for new features, but often pulls in a couple new features themselves (like IPV6 getting all the features the IPV4 stack had). This type of change is the most exciting and also the most likely to break existing drivers and applications.
  3. Architectural Rewiring – This is where we restructure existing code for modularity and potentially new release possibilities. Server Code and MinWin fall into this category of changes. To the upper layers of the OS (applications) it looks like nothing has changed, but you now have the ability to more easily release a super stripped down version of the OS, or let different parts of the OS evolve independently.One of the sins of Windows was the circular dependencies between some components, and we are in the middle of multi-release work to clean it up. A focus of Vista was to map out the system and put in controls to make sure we never introduce more. As a OS Geek, this is exciting stuff, as a OS user, this is something that is sucking up development dollars without apparent affect.
  4. UI Changes – For a user of the OS this is what they typically use to judge how much an OS has changed. Sometimes this implies a lot of work, sometimes this isn’t so much work. Because of the attention, every product typically has some UI change for the sake of change alone, and that change is usually one of the most protected secrets about the OS. There is a balancing act between holding these changes secret, and testing the OS as a final product. Often a ugly theme that utilities the same features as the final theme/UI is introduced to help mitigate the risk. (Therefore pre-release builds shouldn’t be judged on ascetics).
  5. New Features/Components – These are the functional improvements in the products. I think people have a pretty good grasp of this type of change.
  6. Changing Defaults – Relatively simple code/setting changes might make drastic changes to the user experience. Turning off old protocols, making new users non admin by default, etc.
  7. Bake Time/Cleanup – This is the relatively boring but critical process of fixing bugs, incremental performance tuning and just general "make things better" that takes of the majority of a development cycle and extends post release into service packs and the next release. It’s healthy to occasionally have a release that the majority of it is in this category, specifically targeting the things that were too risky for a service pack, but isn’t really a new feature. Unfortunately this type of changes tends to not sell new copies of the OS. This type of next release time is getting institutionalized at Microsoft in the form a Quality Milestone done during product planning when the development team doesn’t have much to do yet.
  8. Platform development – This is the type of work done that might be in the OS, but doesn’t really have any exposure or use until a corresponding server release, or other product takes advantage of it. For example: Windows XP had a feature for restoring automatic backups of previous file versions that only showed up when attached to a server that supported it. Vista (and XP via a separate download) has an amazing new GUI support for applications called the windows presentation foundation, but nothing in the OS itself takes advantage of it. It usually takes a while before we see application developers get used to the new libraries and choose to develop for it (normally a developer doesn’t want to develop for an OS version that users aren’t using in bulk).

Looking forward, we already know that some Architectural Rewiring is happing in the next Windows release with MinWin and with such major Subsystem Replacements in Vista and the compressed schedule for the next release, I can’t imagine too many Subsystem Replacements happening, but I guess we’ll have to wait and see.

Advertisements

They are just more critical these days

Since the launch of Vista, I’ve simply been amazed and the frequency and severity of criticism Vista has received. I humbly accept the places where the complaints make sense to me (Performance/Compatibility; and in many cases I grok the reason compatibility was broken), but much of it, like the DRM hype is just astonishing to watch. Worse, there are many features and improvements that I’ve yet to see Vista get credit for. Anyhow, I’ve been collecting theories of what happened:

  • Security trumped compatibility in this release. (Most of the things that Windows could do without breaking stuff was done in XPSP2)
  • We didn’t focus on compatibility like we did in Windows 95
  • We shipped new Networking, Audio and Video stacks in Vista, and that will cause application compatibility issues and it’s going to take a while for drivers to catch back up to the level of optimization we had before.
  • Too many little features, not enough big ones.
    • Broken planning, dependency tracking, etc.
    • Ship everything at once mentality, instead of incremental improvements
  • There wasn’t enough architectural oversight of the product
  • Too many shifting and impossible to follow through “Basics” (Don’t worry if you don’t get this one)
  • Vista wasn’t selfhost-able until way too late in the product cycle
  • Since the product shipped late, expectations were set to negative by default
  • XP brought the reliability people were screaming for, XPSP2 brought the security people were screaming for. Vista just meet a fundamental need the way XP did.
  • The big stuff people were promised didn’t show up (WinFS and ???)
  • This is really the same thing XP went through
  • ABMs (Anything But Microsoft) people are more are listened to more and more effective with FUD then in the past.
  • They are just more critical these days

I must admit, I didn’t get the last one when I was told it, but I’ve been warming up to it. Enough people are computer savvy now that they no longer blame themselves when things break, they blame the hardware and software people. Well actually, most people just plain blame Microsoft, but give it a couple another decade and people will get better at blaming individual hardware/software manufacturers. While none of the the list is self sufficient as a reason, the recent criticism around Apple’s Leopard release is giving more and more credit to the theory.

Live Maps: The Little Things

While Live Maps haven’t copied the eye-popping drag and reroute feature of Google maps, there is a number of really cool improvements in the latest release. The Live search blog has a list of ten improvements that haven’t got the press of other features like the “model your own 3d building on a map” one. My personal favorites from this list are the End/Start details where you filter down driving instructions to allow you to ignore the stuff you don’t need to see (like how to get out of your neighborhood), the new black color on the traffic map for dead in the water congestion and some of the extra business information they now have.

Vista

So I’ve started to see some press getting way down on Vista about things I haven’t experienced and decided to go and see if I could figure out what was going on. First off, let me summarize my house’s trip to vista.

  • Machines
    • 3 older machines
    • 1 brand new nice 64 bit box
  • Issues
    • Memory
      • Most of my machines needed a memory upgrade to be happy on vista.  Where I Couldafford it machines went to 2Gb.
    • RAW photo support for my camera on 64 bit windows
      • Canon was in no rush to release it and I still don’t have RAW support on 64bit (which is where I do photo stuff)
    • Media Center on 64 bit
      • It was either the 64bitness or trying to also use the machine as a desktop while it was a media center, but this led to a lot of crashes of media center.
    • Loud machines
      • Since vista supports sleep better then previous versions, I started used it for my desktops. I then started to notice the noise difference between on and off.

 

And while I’m at it, the BS issues that people complain about, but I don’t get the issue.

  • DRM
    • Everyone gripes about it and it’s the default reason people give for anything that is broken, but it probably has nothing to do with anything since I’m not aware of anyone using it’s new features yet. It’s a passive, when the application asks for it, feature not an active (lets look for violations) system.
  • UAC
    • When you get a okay/cancel UAC prompt, you are running as an administrator and if you weren’t you would have been asked for administrator account and password. Even when you run as administrator with UAC, you are not administrator. The prompt authorizes a process to run as true administrator. There is a reasonable amount of security value here. The main question is “Should this require administrator rights to run?” whenever you see a prompt. Frankly I don’t get prompted often, and when I do, I find it’s appropriate. The notable exception is when I want to see details of what driver is loaded for my network card or video card. The UI for viewing and setting the settings weren’t separated and so you get a prompt even when you don’t want to change anything.
    • If you think UAC is annoying, the question is, what did it prompt you for that it shouldn’t have?
    • It’s very amusing when people comment about UAC and get recommend another OS that does the same thing, except requires you type in a password.

Okay, so now that I have that out of the way… I’ll next write on where and what I’ve learned

Finally, some good arguments against OpenXML

Stéphane Rodriguez has an article about issues one hits when trying to implement or use OpenXML. They don’t have the idiotic and artificial type of arguments that lists like groklaw has created, but some of his examples feel a bit extended to make a good story.

Lets see what the summary of his issues are with my bottom line comments. Also note I’m no expert at this stuff, I’m a geek, not a word processing file format geek and I certainly don’t speak for Microsoft on these issues.

  1. Self-exploding spreadsheets
    • Removing formulas from a spreadsheet is non trivial because there are other files with references to the forumla to update, such as the calculation chain
    • You can’t rebuild the calculation chain without going through the whole document.
    • While the calculation chain can be excluded it is non optimal to do so because some one who does need to understand the whole spreadsheet will have to recalculate it.
    • Some ZIP libraries don’t deal efficiently with doing the sort of operations needed to manipulate these zip based documents structures
    • Bottom Line 1: Invalidating the Calculation Chain should be automatic, so that simple manipulation tools work better
    • Bottom Line 2: Classic engineering tradeoff, you can precalc stuff if you want, but then you have to be able to precalculate it and keep some sort of invalidating state.
  2. Entered versus stored values
    • The intuition that what you type in excel is what is stored is incorrect. Excel does magic to make it more user friendly like automatically adjusting to local convention (like , instead of . in number formatting) and auto converting to a type instead of treating everything as a string or forcing the user to be explicit
    • The stored number values are affected by IEEE rounding rules
    • Stored values are not locale dependant (This is a bad thing?)
    • Bottom Line: It’s not clear how this affects the usability or usefulness of the format to me. Maybe a different example where values that aren’t in this format (generated by a third party tool) fail in excel?
  3. Optimization artefacts become a feature instead of an embarrasment
    • Worksheet shared forulas are listed as “copy from Cell X” instead of having a neutral non cell reference that everything uses
    • This leads to a lot more work to change a formula in one place if others reference it.
    • Bottom Line: Sounds like a valid complaint to me
  4. VML isn’t XML
    • VML is supposed to be deprecated but gets used in some places like comments
    • 10 year old memo from Gates that has little to no bearing on the world or Microsoft today
    • Bottom Line: I’m not familiar enough with the spec to know if this is an issue or not, but it sounds like comments in Excel is hard to work with and that’s bad.
  5. Open packaging parts minefield
    • You can’t delete a part and know who relies on it without parsing through everything in the file
    • Bottom Line: sounds sucky
  6. International, but US English first and foremost
    • The functional things in the format for excel is in english (like the SUM() function)
    • VML and DrawingML have a number of encoding notes to help with localization which aren’t documented well
    • Applications on top of OpenXML have to localize everything themselves
    • Bottom Line: Maybe I’m missing it, but this seems like a feature, my spreadsheet manipulator doesn’t have to be aware of all the possible language encoding of the word “SUM”

I’m going to cut off this post here for now (wife wants my attention 🙂 ) and maybe continue it another day

Major themes from the list so far:

  • The excel format seems to be not well designed for targeted modification of existing files. You have to load an understand the whole thing and then write it all back out again. (unless you are using the custom schema stuff, but that is out of scope)
  • VML interacts with parts of openXML is not well describe ways

— Ari

Windows Security Boundaries

I was reading Raymond’s post on Escalation of Privilege bugs that don’t actually escalate your privilege and then quickly read the earlier episode of the series. There I saw a lot of commenter rebilling against the concept of post by drawing new security boundaries which the hypothetical exploit would cross. This crystallized a concept for me that there are certain security boundaries in windows that are harder then others and there is much confusion in this area. Since I haven’t seen this information in one place anywhere, I’ll try to consolidate my understanding of it here.

Security Boundaries control the flow of information and execution between two distinct environments. We consider a boundary breached when arbitrary data or execution is no longer prevented from occurring. Most of the time we consider one of the environments a superset of the other, for example, going from executing as a single user to controlling the entire Operating System. However any attack that gives you more privileges then you currently have can be considered an escalation of privilege.

  • Primary Security Boundaries
    1. The Remote Boundary (is there a better name?)
      • This boundary separates things executing off your computer and on your computer. When an attacker can remotely make your computer do arbitrary things in a security context that would be crossing the remote/machine boundary.
    2. The User Principle Boundary
      • This refers to the security boundary created by executing code under a security principal and the ACLs that details which user has access to which resources. This is what keeps one user from snooping on another user’s files. If untrusted code manages to run in your user account, it’s not really your user account any more. This can also refer to non user accounts such as services.
    3. The Administrator/Kernel vrs Not Boundary
      • This is the boundary between a normal user and running as administrator or executing code in the kernel. Once untrusted code is running in either administrator or in the kernel, it is not your box anymore.
    4. Privileges
      • These carve out boundaries like ACLs.
    5. The Operating System Boundary
      • This boundary refers to the ability to read files and execute when it is allowed to execute outside the context of the operating system normally in control of the resources. If the OS isn’t running it can’t protect secrets. Technologies like bitlocker and the one-way encryption of passwords are attempt to deal with breaches of this boundary. Vitalization is making this area more interesting.This is also the point of Immutable Law #3.
    6. Managed Code (CLR/Java) sandboxing
  • Mitigation Boundaries (These are bypass-able, have uses and may be put together to make something stronger but alone do not form a primary security boundary, see Mark’s blog)
    1. Power User/Administrator/Kernel/System
      • You can switch between these without much difficulty.
    2. Vista Admin account UAC
      • The split token helps but doesn’t make a full boundary
    3. Session boundaries
      • Different user sessions have different named object namespaces ACL’d to them, however one user could reach over and mess with then session of another instance of the same user.
    4. Restricted Tokens
    5. IL Levels
    6. Software Restriction Policies
    7. UAC elevated processes in a user session
    8. Kernel Driver Signing
    9. NATs/Most Firewalls
    10. Kiosk style, certain applications only hacks/setting changes
    11. System File Protection
    12. Windows Data Protection – DPAPI
    13. Code Signing

Much of the confusion occurs from “breaching” a Mitigation Boundary instead of one of the Primary Security Boundaries. Aside from some nice new Mitigation Boundaries, the main thing that Vista does is move most users from the Administrator/Kernel side to the rest side or the primary boundaries #3, and that is a big deal.

Google decides to be evil

According to a number of articles like this one, Google is the source of an antitrust complaint against Windows Vista because of a change of the default implementation of Desktop file search. In Windows XP, when you searched for files it would do a actual, go scan your harddrive search, and at the end of the search you got an option to turn on indexing to make your search faster. This would search anvista indexingd make notes about your harddrive in advance so that the requested search became much faster. I’m guessing that it was off by default in XP because it wasn’t really optimized for a desktop both in performance, the type of data it indexed about the files and it wasn’t something people did a whole lot so it wasn’t worth the weight on the system. Enter Vista and the world has changed, indexing is the standard approach to search on the desktop as demonstrated by the improved indexers shipped in MacOSX, Google desktop and MSN one. So the good old xp indexer gets a lot of attention, a nice upgrade, some very nice usability improvements and, Oh yeah, the indexer is now on by the default instead of just for power users. Well, that last step is one step too far according to Google.

According to the article they are worried about interactions between their indexer and the vista one. While a lot of people, on digg at least, are calling BS. It is especially weird to me since a number of applications that I’m running these days are busy indexing the harddrive. The photo gallery software and all three music applications are going at it. They manage to coexist in vista, what’s wrong with Google’s indexer? This sounds like a technical limitation in their product they wish to use to harm vista’s indexer.

They have plenty of business reasons for such a desire, they used XP’s deficiencies in this area as a big reason to get people use the Google toolbar (which includes their desktop indexer). This is important to them because it has all sorts of tie backs to Google services where they make money. It was a good gig, the MSN team developed and did the same thing. The Vista indexer doesn’t have any such ties, but now people have lost a huge reason to install the Google toolbar (and the MSN toolbar for that matter). So they have a business problem, and from their complaint a minor technical problem. Business model problems don’t make good complaints to the DOJ, but maybe they could make hay with their technical issue. Unfortunately most techies would predictably call BS if they heard the complaint (I guess that’s why it was a confidential complaint) which leads back to the premise, It appears that Google has unabashedly decided to be evil.

On the other hand, indexers are programs that are not just running all the time, but constantly trying to do work. Smart applications attempt to do more and more stuff when the user wouldn’t notice, such as checking for and downloading updates or pre-creating image thumbnails so they don’t have to be generated at run time. On a logical level there is some theoretical maximum to how much time a computer has for such background tasks. Google seems to be implying that there is not enough room for anyone but them. Even in this worse case, this is something that a years worth of Moore’s law will fix faster then any legal remedy. Oh and I should point out it has been years since the first of this generation of indexers were downloaded and used on computers.

It’s going to be interesting to see the arguments on the other side of this one.

Disclosure: I work in windows networking, I don’t have anything to do with the indexer technologies except complain about how slow the early versions of it in pre-reset longhorn were.