Doubt's Log

I really enjoy Joel's writing. He does a nice job explaining the state of affairs: Martian Headsets - Joel on Software

Reading the original IE Blog Article and the /. Discussion on the X-UA-Compatible markings, I have reached a couple of conclusions.

• There is a camp of people who think that standards are an end to themselves free from nitty gritty details like solving real world problems.
• Some people understand the issues but are happy to give the finger to billions of lines of working HTML pages and HTML generating code because it didn't have the honor of being standards compliant years ago when it was written. (I think these people are secretly interested in donating their time to a  Y2K style effort of fixing all these old sites)
• Many commenters don't understand the constraints of this particular problem

For the sake of the last camp, I will attempt to make the issues clear up the problem constraints (and fail).

1. It is unacceptable to break existing pages. If a person's favorite site doesn't work, they will avoid the upgrade or downgrade back to the old browser.  Assuming that all browser upgrades brings us closer to interoperable web standards, non adoption of the latest browser version is a very bad thing.
2. Most existing content is immutable. There is too much of it and too much work to fix all the HTML and HTML generators which originally produced it.
3. Web Standards and Implementations are not instantaneously mature, which means that all implementations will ship with bugs. While this is painfully obvious in IE, it is also demonstrated elsewhere: Firefox 2 doesn't pass ACID 2, Firefox 3 will. What happens to pages depending on the bugs in Firefox 2 fixed in Firefox 3?
4. There is an awful mess of pages out there that will forever be in Quirks mode and IE6/7 "standards" mode. Any solution that doesn't deal this issue is broken.
5. It can't be a one time fix (debatable?). Something like a one time doctype change is not sustainable and leads to the same problem over and over again (since there will always be the latest new standard). Any solution should be good to handle this type of problem again and again for every browser vendor.

So now that you understand the constraints and you still have issues, make the world a better place and figure out a better solution then Microsoft did.

Stéphane Rodriguez has an article about issues one hits when trying to implement or use OpenXML. They don't have the idiotic and artificial type of arguments that lists like groklaw has created, but some of his examples feel a bit extended to make a good story.

Lets see what the summary of his issues are with my bottom line comments. Also note I'm no expert at this stuff, I'm a geek, not a word processing file format geek and I certainly don't speak for Microsoft on these issues.

1. Self-exploding spreadsheets
• Removing formulas from a spreadsheet is non trivial because there are other files with references to the forumla to update, such as the calculation chain
• You can't rebuild the calculation chain without going through the whole document.
• While the calculation chain can be excluded it is non optimal to do so because some one who does need to understand the whole spreadsheet will have to recalculate it.
• Some ZIP libraries don't deal efficiently with doing the sort of operations needed to manipulate these zip based documents structures
• Bottom Line 1: Invalidating the Calculation Chain should be automatic, so that simple manipulation tools work better
• Bottom Line 2: Classic engineering tradeoff, you can precalc stuff if you want, but then you have to be able to precalculate it and keep some sort of invalidating state.
2. Entered versus stored values
• The intuition that what you type in excel is what is stored is incorrect. Excel does magic to make it more user friendly like automatically adjusting to local convention (like , instead of . in number formatting) and auto converting to a type instead of treating everything as a string or forcing the user to be explicit
• The stored number values are affected by IEEE rounding rules
• Stored values are not locale dependant (This is a bad thing?)
• Bottom Line: It's not clear how this affects the usability or usefulness of the format to me. Maybe a different example where values that aren't in this format (generated by a third party tool) fail in excel?
3. Optimization artefacts become a feature instead of an embarrasment
• Worksheet shared forulas are listed as "copy from Cell X" instead of having a neutral non cell reference that everything uses
• This leads to a lot more work to change a formula in one place if others reference it.
• Bottom Line: Sounds like a valid complaint to me
4. VML isn't XML
• VML is supposed to be deprecated but gets used in some places like comments
• 10 year old memo from Gates that has little to no bearing on the world or Microsoft today
• Bottom Line: I'm not familiar enough with the spec to know if this is an issue or not, but it sounds like comments in Excel is hard to work with and that's bad.
5. Open packaging parts minefield
• You can't delete a part and know who relies on it without parsing through everything in the file
• Bottom Line: sounds sucky
6. International, but US English first and foremost
• The functional things in the format for excel is in english (like the SUM() function)
• VML and DrawingML have a number of encoding notes to help with localization which aren't documented well
• Applications on top of OpenXML have to localize everything themselves
• Bottom Line: Maybe I'm missing it, but this seems like a feature, my spreadsheet manipulator doesn't have to be aware of all the possible language encoding of the word "SUM"

I'm going to cut off this post here for now (wife wants my attention :) ) and maybe continue it another day

Major themes from the list so far:

• The excel format seems to be not well designed for targeted modification of existing files. You have to load an understand the whole thing and then write it all back out again. (unless you are using the custom schema stuff, but that is out of scope)
• VML interacts with parts of openXML is not well describe ways

-- Ari

My first reaction to the news was, ah so that's how they will allow people to develop and test their apps for the iphone. Then we loaded it up on a test box and I had three reactions. First: Why does the window frame look like crap? Second: Why is all their web page text so fuzzy to the point I felt sick? Third: How the heck does one open a new tab? It seems to be the pattern that whenever apple ships software for windows it looks much uglier then a default hello world message box type app. Hopefully they will someday improve upon their porting kit and make something that doesn't look so awful. I can also understand apple's hostility to windows, if I had to use/test apps that looked like that all day I would be hostile too. :)
Oh and a couple more quick usage notes:

• The back button on my mouse doesn't do anything in Safari
• Not having an edge of the window to use for resizing is pretty annoying
• I can't find any way to add wikipedia to the search box
• If you don't have any binary legacy support to worry about, why are you going 32bit only? Get the extension market used to 64 bit now before it becomes a legacy hassle.
• Drag and drop customization of the UI elements is pretty cool
• CFNetwork.dll? This could be fun to play with...

Overall, this has a serious case of portcitus, when your app looks or acts lame because you are more focused on a compatible source tree and exact rending with the other platforms then taking advantage of the platform you are porting to.

Update: Oh yeah... and do some security testing :)

I saw a slashdot article this morning about Apple releasing more vuln fixes. In the comment section, discussion broke into the usual "why do people think Macs are safer then Windows" arguements. The two major points of "it has less of a market" and "it's just more secure" went back and forth. I happen to think both are an oversimplification of the subject.

Vuln finding is a function people of going after whatever is currently easiest. Many attackers have broaden their horizons to other platforms once Windows became significantly more secure and harden against attack. Oracle was the next major target and Apple might be the one after. I admit that I love the irony of the switch after both companies choose to market on how they must be more secure since people weren't finding vulns in them.

Exploits on the other hand is based on the business case these days. The vulns are available but Windows didn't have the magnitude of the problem it did until there was a profit motive to create bot networks.

So to put it together, vulns found help you tell about the security of an area, exploiting tells you about how profitable a particular OS is to attack. The corollary of this rule is that as a random host you are as profitable as the OS, as a specific host with specific data or rights you are as valuable to attack as that data or rights. The result being that if your data is valuable is doesn't matter that there are few exploits for your box when there are plenty of vulns.

An attempt to respond to the latest thing I've read and stake out my feelings on ODF/OOXML.

From what I understand of the market, you have a number of (free) add-on ODF plugins for Microsoft Office. This means that the simple requirement being able to read and write the format will be satisfied to the level of quality of the plugins and the ability of the interoperable aspects of the ODF standard to handle office semantics. I feel that the blogoshpere has made it clear that the only way ODF will be able to handle the body of existing office documents (Bugs and features) at full fidelity is for there to be a large number of extensions that would render ODF something not ODF anymore, especially from the standpoint of other ODF implementations. It might be in the vaguely "right" looking container, but it would not be interoperable. Any movement in this space would (rightly?) be branded Embrace, Extend, Extinguish.

I believe it is clear that users want something like OpenXML. We've seen that previous movements in this direction by office in the 2003 products are never used because of the loss of fidelity. I'm just not going to migrate my spreadsheets to ODF format if my formulas are going to break, and that is the type of user complaints that you will start to get when you tell your customers you must move over. If you don't get how complex this type of thing gets, you should start reading Raymond Chen's blog. It is quite obvious how hostile the ODF crowd appears to be to backwards comparability with the amount of hoopla generated around supporting the 1900 excel/lotus 123 date issues in OOXML.

Could all the the technical issues been worked out in ODF? Maybe. I think the hostile environment, the time required to work on modifications to ODF in an open way and the timeline for the politics and government mandates pretty much precluded that option for the short term. On the brightside, ODF folks can take the out there and free OOXML spec and decide how they want to absorb it for future versions of ODF. Thus somday the promised nirvana of ODF being the native interoperable format of all office suites that it's supporters want might be realized. In the here and now, there is a pretty cool creative energy that both formats competing right now has created. In an attempt to score points in some insane "Who is Right" contest both sides are pointing out the flaws in the other, and the pragmatists will pick up the real stuff and just make thier stuff better. This is a good thing no matter how ugly the process is to get there.

In the background of this debate, It appears that their are two camps in the world when it comes to this stuff, purists who believe that future technology should be clean slates not marred with the real world and those who muck around in the complex world of user demand and prior work. I have to admit out of college I was very much in favor of the purist view of the world. This little debate is making me realize that I've now firmly landed in the other camp. The purist typically ends in the worst hacks and/or low adoption. There are a lot of people out there who use software and just don't care about the religious battles. It doesn't matter what your standard is or how you architected the code is, if it doesn't solve the user's needs.  Put simply, users are more important then you or I and placing requirements down that are tangential to their needs is just a speedbump for them to roll over. The coders who love and support these users are going to have to help carry forward whatever hack someone came up with to get around the artificial speedbump. The sooner one grok's this concept the better the world might be.

If ODF solves a user's needs, they will use it, if OOXML solves it better it will be used regardless of which of them have ISO certification. There is already ECMA certification and good IP promises for OOXML. (The inability to use without IP considerations a file embded in either format is a red herring). It appears that Microsoft is supportive of having OOXML ISO certified, which sounds great to me. If there are considerations unrelated to ODF then they should be fixed, but the notion of which sausage factory produced the 1.0 spec or that you can't have both formats be standards seems silly to me. Both are too new on the scene to have proven that they are going to be the end all. If anything, office via market share and caring about backwards compatability has a huge leg up.

Disclaimer: I work for Microsoft but nothing to do with office.

There a pretty reasonable podcast about Vista DRM in Security Now #75. Key points:

• Worse case is that you can't play content that demnands a super secure path.
• No known media is requesting the super secure path. It is very questionable if anyone will ever want to take the PR hit of actually using it.
• Constriction or "fuzzyness" is for the high quality content; not everything on your screen and only if the content requires it.
• The main device you are probably playing HD-DVD's on is laptops who have onboard graphics and are exempt from a number of things that people are concerned about.

Update: Just to be fair, there are a number of legit concerns that the Gutmann paper talks about, but even in that paper there are examples that people have let thier imagination run away with. The legit concerns include: side effects to how open hardware is when hardware needs to authenticate to the driver (They should do a public key thing here IMHO), Hardware/CPU costs in dealing with encrypting content across an open pci bus, potential cost for splitting out drivers to mitigate potential protect content trust revocation, the potental for hardware manufactures to destablize a PC when creating an implementation of tilt bits and IP/Licencing costs for the content protection hardware. To me these are pretty minor or requires assuming the worst for a true bad effect.

It's easier then ever to get a continuous stream of Windows Vista FUD. In the past you had slashdot, but had to ignore the pesky rated 5 comments which often would point out the obvious stuff. Now however we have the BadVista blog, which is FSF new foray into the world of pure unadulterated BS. Some of which the press runs away with because there aren't enough people actually using the software to call BS loud enough. Let's look at some of today's news stream:

1. Microsoft Vista is not an option
   This link is about the "Licensing and Activation" hurt hobbiests meme. We have a writer who switches out the hardware inside his case once a week and is using XP. To my spider senses, Something doesn't add up here. XP already has activation. Even If they tightened up the requirements (which in practice remains to be seem) he should already be tripping over activation left and right. In some ways activation has gotten less onerous, especially in cases where you buy a computer from a OEM like Dell or HP. Personally, I have built all the machines in my house, and swap around components regularly (although I guess I'm too busy and poor to swap things weekly on a single machine). I've been bit by re-activation and had to call the activation help for my home machines twice since 2001. I told them that I was moving around components and things worked within 5 minutes both times, this is hardly social engineering. I don't expect Vista to be any different, and I've already moved some hardware around. If the author hasn't been through this already with XP, the the worst I would expect is that he will make the 5 minute phone call once a couple years. If the fear of that potential phone call and not even a real experience is bad enough to make him switch to another OS, then I not sure Vista is the real issue.
2. DRM behind lack of Windows Vista drivers.. and fear new content protection.
   This is based on the Gutmann FUD, which spells out a worse case scenario for the implementation DRM in Vista based on random bits of documentation and conjecture. The basic problem here is that the worst case scenario he envisions isn't  how anything was implemented and causal checks confirm it. There are still class drivers for video. Non-protected content (which is most of what I have) plays unmolested, even while I play DRM'd music and video.  There is an example in the paper of expensive optical system computer in a hospital going fuzzy because the user is playing music. The first question a reader should ask is, even if the hospital bought into that sort of DRM and the system was designed that way (which from casual observation it isn't) why would the hospital not buy a computer system to view the imagery that supported a DRM path in the hardware. It's like buying a CAT scan system and not buying a compatible display to see the results.
   In reality there is more drivers and compatibility for Vista pre launch then there was for XP (probably because of continuation of the move to class drivers and early frequent public releases). Inherent to the whole arguement is a bet that you will have pervasive protected content you want. This is the same bet that iTunes Music Store makes. If you want to watch such content, then you won't want to run an OS where you can't watch such content, and only systems with these protections will be allowed to decrypt it. Back in the early days of DVD's, Linux had zero players until the protection scheme for DVD's was broken. These days the new formats won't be cracked that easily as they have learned a couple lessons since then. (They can remove support from all future media for a player's decryption key once it's known to be cracked and the general purpose cracking is probably much harder)
   There are some real stuff to the story, supporting DRM through hardware is not free, and if you want that feature you will pay for it (similar to how we all pay for DVD support), but none of this is Vista specific. The main thing with Vista that you might complain about is that it supports it at all, or that Microsoft hasn't done enough to fight DRM. Of course if you buy PowerDVD for BlueRay or HD-DVD you are getting pretty much the same thing from a different vendor. This type of stuff really annoys open source purists because licensing and securing implementation runs counter to the basic philosophy, but it's not a showstopper as companies that actually build commercial products using pieces of open source don't have such issues.
   I'll also note that I'm not in love with DRM, but that's a topic I'll save for a different post.
3. Vista: Why Bother?
   This starts with the insufficient hardware meme. If you asked me right after beta 2 shipped, I would be wholeheartedly agreeing. What I have discovered is that a) they fixed much of that between beta and release and b) more RAM fixes the rest . Ironically the RAM part was exactly what I was sitting around realizing when XP shipped. The end rule is if you bought it in the last two years new, get it up to say 1 Gb RAM, it'll be fine.
   The actual piece plays a bunch of games with the facts. First it talks about video editing, which is demanding in general and nothing specific to Vista. Even looking at the Mac's that advertise high end video editing you are looking at some seriously powerful machines. Processor, RAM speed and file system speed are the things I've noticed are the big deal, not OS. Next there is the 94% figure, which pulls a double whammy, first it is a survey of corporate machines, which since they tend to do simpler less CPU/ram intensive things compared to consumer PCs. The more realistic numbers are the CPU replacement numbers (replacing the CPU, especially in older machines usually means a new PC), here we see 84% of corporate PCs will be ready from a CPU standpoint (I suspect many of the 84% will need more RAM, but the numbers aren't in the article). The other little trick done with that number is using the premium level of readiness instead of the minimum. For corporate PCs, the difference between the premium and the Min are features that won't be missed doing day to day work, like the flashier GUI.
   Next in the piece is software compatibility. This is a harder area, although three of his examples are now bogus. The Zune software for Vista is already released on zune.org, I'm running the Vista Powershell (it comes as an OS update, so it's mostly an issue of packaging, not compatibility as people running the old msi versions of Powershell can attest). The new Virtual PC has hit RC status. OpenGL is supported in the major graphic vendors drivers. I've found that most of the real issues with compatibility are from deeply integrated software using unpublished interfaces who aren't in a rush to put the vista versions out and UAC related issues. For many of these companies the clock didn't start until we RTM'd Vista. The latter is a price we will pay for the security it brings, but will be lessened as compatibility updates come out. On the anacdotal side, I'm mainly feel pain with x64 versus x86 rather then Vista versus XP.
   Also in the piece Start Menu issue. My start menu has two options for "shutdown" and a somewhat hidden advanced menu. The two options on my box are: low power mode and lock session. Ironically, I don't even use either of them on my home machine. I just push the power button on my case to go to low power mode. At work I only use the Lock one (assuming I don't just press Windows-L). So it appears they choose the right two.
   For bonus points the author then compares upgrading a point release of openSUSE to upgrading Vista from XP. A fair comparison would be to a service pack update, although I would guess that even that would be more then the dot release.
   Which finally concludes in the classic, why update? If you need a single compelling reason to go to vista, it would be security and maybe the flashier GUI, after that it just feels better, the sum of a thousand little things. This is not great for marketing, but pure addicting goodness as a user and home admin. This should become quite apparent after the OS is actually out there, but you can see it in people like the TWIT crowd who has talked about their experiences since they first installed it and now really like it (oh and they are heavy Mac users). My suggestion is to find someway to use it for a week or two and decide for yourself.

Disclosure: I am a Microsoft Employee who works on Windows, but these views are my personal ones and are not my employers.

The New York Times has developed a pretty cool windows client using the avalon Windows Presentation Framework in .Net Framework 3.0. The pp downloads the articles so that you can browse the news paper offline, it re-flows the content to intelligently display articles based on the size of the windows, has great search features, keeps read state, lets you annotate the articles save it off, etc. It's a pretty powerful demonstration of what new windows applications can be like.  Right now it uses the free NyTimes registration. Check it out!

My Favorite unadvertised benefit of working for Microsoft is access to the various speakers that come through campus and give talks to MSR. It is great to here that many of them have made it to the public on ResearchChannel. I've bookmarked on del.icio.us a number of talks I've seen and enjoyed and a couple that I haven't seen and want to. Special notice to the two MSPAC ones, David Brin, Mandelbrot, Bruce Campbell, Malcolm Gadwell and Cory Doctorow.

He then takes a sweeping amount of ignorance and combines it with a single datapoint from a conference to get that vista won't be any better regarding security and malware. As someone who has gone through the culture shift here and seen so many features thrown on the back burner to make room for UAC, protected mode IE, theat models, security audits (internal and external) and more. I just have to guess he isn't following this area at all.

Then he makes a prediction that Apple's Bootcamp will just make Microsoft more money because apple users will buy Windows retail. I think a good analogy for this is naked pcs, where people buy a machine without an OS or one with Lindows and just pops in a Window XP CD they have around or borrowed from some one else. I'm less worried about retailers doing it or businesses, but it remains to be seen how much these channels will want Windows on their boxen.

I watched the PDC keynote yesterday.. or at least everything up to the code walkthroughs. I was very happy with some of the improvements comming in office 12, but not much in the Vista demos excited me (probably because I've been see'ng the stuff for while). The UI improvements promise to be good, I'm especially the quick “What's my next meeting?” on the mail page. What I'm estatic about is the LUA features, or the “How we don't always run as admin“ features.

Other things for me to check out:

• Lighting Up on Windows Vista
• RSS Support in Windows Vista 
• Linq Q&A - Channel 9 Interview
• WorkFlow Support - Channel 9 Interview - Community Site
• Tons of Channel 9 Interviews

Better Living through Software points us at NPR's Podcast page. 130 podcasts ready to go.

The Code Project has a article playing with IPV6 in C# on the 1.1 CLR. (BenjaminGay)

• Npr has a story on Donald Knuth
• Nytimes' Kristof laments about the state of environmentalism, he is looking for a less alarmist more responsible movement
• In spite of lower value of the dollar, the trade deficit hits it's second highest figure ever
• Scheneier points out that the Unicode community is thinking more about security in this draft report

Larry Osterman has a great article about Coding Styles.

I finally got a real understand of what DTrace can do by this blog entry. It's a sweet form of instrumentation.

Netcraft has a article about some of the security issues that are showing up in non-IE browsers.

Internet Explorer isn't the only browser vulnerable to spoofing, as evidenced by the discovery of security holes in Firefox, Mozilla and Opera.

They also talk about the dangerous trend for IE.

Thus far, services tracking browser usage report only incremental gains for Firefox, Mozilla and Opera, with some suggesting IE has lost about 1 percent of its 90 percent-plus market share. But Firefox in particular seems to be catching on in some quarters, as was evidenced at the recent BlogOn2004 conference for weblog aficionados. During a Microsoft presentation about its Channel 9 blog outreach, a presenter asked "Show of hands...How many of you use Internet Explorer?" Not a single hand went up.

Just a quick heads up on a products a lot of my work friends on that has been announced publically. It's Windows Network Access Protection. The basic idea is that a client connecting to a network is put on a restircted network till it has proven that it's a safe machine (up to date with patches, running anti-virus, AV signatures are up to date, etc). This restricted network has enough access to do things like update the AV signatures and so forth. It's a pretty nice idea for all thoose corperate laptops that slip around the firewall and screw your internal network.

Steve Butler has an blog entries on using the WMP SDK to parse Library information and even better has a project about building a UPNP Media Server using some stuff Intel provides to make UPnP usable from managed code.

Don Box points out that you can now download the Comega compiler that contains Xen (X#) and Concurrency extensions. Xen is what happens when you take databases and xml and make it fundimental to a OOP language.

The problem facing the programming language designer is not only identifying the "heavily used APIs" and "common programming patterns" mentioned earlier but also tastefully realizing them as first-class language features or constructs. Bolting on random features leads to language bloat, potentially destroying any coherence the language may have had. In addition one hopes that by promoting a feature from an API it will be better supported given its existence in the language.

...

In our opinion three areas that are ripe for liberation from their lowly API status are (a) data-access, (b) concurrency and (c) security. [Programming with Circles, Triangles and Rectangles]

This week I'm taking some training by OSR on advanced driver development. I've been testing drivers since I started at Microsoft, but rarely from the kernel directy. As a result I have a fisheye's view of the kernel that I hope to correct.

In the process of explaining how PnP works they showed off thier tool Device Tree, which is also distributed as part of the DDK. I couldn't find a copy of the DDK for immediate download (although MSDN has a online copy of the docs), one could order it for just the shipping and handling. One of the cooler things about the recent DDK is that is comes with prefast, a nice tool finding some types of bugs in both drivers and normal code.

It was suprising to hear that people have trouble setting up a kernel debugger for windows kernel development. After noticing a prety good description of how to do it by Jolyon Wright, I feel I can only add two points and quick and dirty step-by-step guide. You can find the debugger on microsoft.com and the builtin “bootcfg” command line tool is a nice way to avoid the whole attrib thing when editing the boot.ini.

Here is the minimum setup:

1. Connect the two machines via a NULL modem cable. Notice which serial ports you plugged the cable into.
2. Choose a machine to be the debugger. Install the debuggers on this machine. Run windbg. Hit ctrl-k and put in the serial port you are using on that machine with a speed of 115200.
3. On the other machine the debugee, run bootcfg. Run something like this: BOOTCFG /Debug ON /PORT COM1 /BAUD 115200 /ID 2
4. Reboot the debugee and wait for debug spew to show up in windbg.
5. hit the break button in the debugger (ctrl-break). Type .symfix and hit enter. Wait for the symbols to load from the internet's symbol server and hit g to allow the debugee to continue running.

BTW: This item is a test post, which might get picked up by the Microsoft Community page.

I wrote a little peice on the WebTransports blog to help answer some questions Jon Udell asks about Windows XP, IE and credential management.

Cory Doctorow gave a talk today at Microsoft regarding why he feels that DRM is bad. There are three overall themes, first is that DRM is an end run around copyright law, and that all these schemes plus anticircumvention laws allows content distributers to invent new “law“. The second is that DRM is harmfull to pretty much everyone. The third is that copyright law has always in the past adapted to the new technological reality, not the otherway around.