New Features

A full breakdown of all the methods and functions of 2.0 are available on GitHub, but these are some of the new features.

• Array.sample
• Array.histogram
• Array.countByType
• Array.percentile
• ps.math.even
• ps.math.odd
• ps.math.product
• ps.math.randomBetween
• ps.math.randomNormal

Array.sample

Probably the biggest new feature is the ability to do sampling for any of the Array methods using Array.sample. It returns a randomly sampled array of any length less than or equal to the source array length. You can use it to quickly do calculations over large datasets (1MM+ rows) very quickly while sacrificing only a small amount of accuracy.

Here is an example:

// This will take a long time (many seconds) to run.
// Makes the browser become unresponsive
tenMillionRowArray.stdDev();

// Takes only a few milliseconds to run, and the
// returned value is almost exactly the same.
tenMillionRowArray.sample(20000).stdDev();

Suggestions

As always, if you have any other useful math or statistics functions you’d like to see implemented just drop me a line at Twitter or GitHub with some details on how to perform the calculation. Even better, just send me a pull-request with your implementation. ;)

Ellis’ law is simply this: (Scope * Quality) / Resources = Duration. This ‘law’ isn’t something I have decreed to be true in projects I’ve been a part of. It is something I have observed; more like how Newton observed an apple falling on his head. There is immutable relationship between each component of the equation. If you change one component, then at least one other will change. Let me break this down further by sharing my definitions of each component.

Components of the law

Scope is simply the list of stuff to be built. It could be a marketing requirements document (MRD), a product requirements document (PRD), a Scrum product backlog, a BDD behavioral spec, or any other form of ‘requirements’ even if it’s just a list in the founder’s head.

Quality in this equation is expressed as an oversimplified 0-100% of the theoretical quality you could intend to achieve for a project. QA is always the first place people, especially management, want to trim when a project timeline gets tight. They never explicitly accept that they are allowing the quality to drop of course. In their mind the quality should stay the same; we usually do a lot of superfluous tests apparently. But even if you are comfortable with the resulting quality of the project, you will never be as certain the quality is as high as you would like.

Resources has a very broad meaning in this context. It represents anything that can be used to complete the project. It can be more team members, longer hours, improving the talent level of the team, faster computers, more monitors, you can even increase the resources by lowering the overhead (meetings, process, etc.) that takes up the team’s time. Once QA has been trimmed to the bone, the next step is always stealing resources from another team even though your team wastes 1/3^rd of their time in meetings.

Lastly, the result of all of this is the duration of the project, which is pretty self-explanatory. In the primary form of the equation expressed above it is how long the project will take. It can also be the duration you desire if you are solving for one of the other components though.

Permutations of the law

You don’t need to exactly calculate the law but I like the equation form of this idea. Using basic algebra concepts you can ‘solve’ for each of the different components. If you know your scope, quality, and resources then the duration is decided for you.

If you know your scope, quality, and how long the duration can be then you know what kind of resources you will need to complete it on time.

If you know your scope, duration, and resources then you know what quality you will end up with.

If you know your duration, resources, and quality target then you know how large your scope can be.

tl;dr

So next time someone says you need to ‘accelerate’ some project because it has to be done sooner you know what you need to do. You need to figure out a nice way to either 1) ask for more resources, 2) ask them which features they want to drop, or 3) explain to them why you are lowering the quality bar. The choice is theirs.

Waterfall

In a waterfall process the development flows sequentially downward through each phase of the model: requirement definition, design, implementation, verification, etc. Each of these steps are really a form of what I call “intellectual inventory”. The phases in the model change from company to company but this is basically what happens:

1. An inventory of requirements is created before the product can be designed
2. An inventory of design is built before engineers can code
3. Engineering produces an inventory of untested/buggy code before QA can test
4. QA builds up an inventory of defects for engineering to go back and fix
5. Maybe user experience then builds a list of usability defects to go back and fix

Keep in mind that each phase in this process will be at least one month and usually quite a bit longer than that depending on the project. That means when you get to step 5 you have two choices:

1. Don’t fix any issues and ship anyway
2. Ship very late and go back to step 2

Neither option is very good for you, your customers, or your company. There is another industry that used to produce their products this way that we should learn a lot from: the auto industry.

Lesson’s from another industry

Cars used to get produced in huge batches as it was seen as inefficient to produce a variety of cars at the same time. Large caches of parts were held in inventory every step of the way to minimize fluctuation in supply and demand. This resulted in greatly reduced flexibility. In fact the Model T famously came in any color you wanted “so long as it is black.” This was the Ford way, and Ford was seen as the epitome of success with their assembly line production.

Then Toyota came along with their Toyota Production System (TPS) in the 1950s and changed the whole game. TPS is now more generically referred to as Just-In-Time (JIT) Production. In a JIT world inventory equals waste, by definition. Consider this paragraph from Wikipedia on a TPS principal known as Heijunka (production leveling):

“To prevent fluctuations in production … it is important to try to keep fluctuation in the final assembly line to zero. Toyota’s final assembly line never assembles the same automobile model in a batch. Production is leveled by making first one model, then another model, then yet another. In production leveling, batches are made as small as possible in contrast to traditional mass production, where bigger is considered better. When the final assembly process assembles cars in small batches, then the earlier processes, such as the press operation, have to follow the same approach. Long changeover times have meant that economically it was sound to punch out as many parts as possible. In the Toyota Production System this does not apply. Die changes (changeovers) are made quickly and improved even more with practice. In the 1940s it took two to three hours, in the 1950s it dropped from one hour to 15 minutes, now it takes three minutes.“

TPS actually does the exact opposite thing to minimize fluctuations in production. Even though this minimizes fluctuations it actually improves responsiveness to customer and market demands. Most cars are available in at least a dozen colors and have so many possible options (sunroof, satellite radio, GPS, power seats, engines, transmissions, etc) that there are potentially hundreds of thousands or even millions of possible versions of a very common car such as a Toyota Camry.

Hidden benefit of Scrum

Of the many advantages to Scrum which are commonly cited the most important benefit is implied but never really articulated and that is reducing what I call “intellectual inventory”. Software development can have many forms of this type of inventory. It can be any artifact (requirement/PRD, design document, untested code, etc) that is completely created before being passed off to another functional role.

The key steps of the waterfall process are actually all forms of Intellectual Inventory: requirement definition, design, implementation, and verification. In fact the waterfall model is basically building up inventories in large batches.

Creating requirements takes a lot of “non-development” time from team members like the Product Owner/Manager, User Experience, Design, etc. The more complete the requirements are the more investment you are making. It takes development time to implement something that doesn’t get tested. Even if it does get tested but the feature is too buggy to pass QA you have still invested in something with zero return; even though your engineers were “done” with their phase of the waterfall.  Make no mistake about it, intellectual inventory is something you spend investment on.

Intellectual Inventory = Waste

In an Agile/Scrum model intellectual inventory = waste as well. Requirements that are twelve months from being implemented don’t need to be refined to the point that the feature could be implemented today. A lot can change in a year and that feature could get dropped, expanded on, or maybe it will be delivered in a different way (iPad app instead of desktop client, HTML5 instead of Flash, etc) because users and the market change. Clearly any time spent refining the requirements that are discarded would have been a complete waste. Of course in practice it isn’t possible to completely eliminate inventory, but it can be greatly reduced.

Let’s go back to that quote from Wikipedia and see what it looks like when hypothetically applied to software now (changes in bold):

“To prevent fluctuations in production it is important to try to keep fluctuation in the final assembly line to zero. Facebook’s final assembly line never assembles the same product features in a batch. Production is leveled by making first one feature, then another feature, then yet another. In Scrum, batches are made as small as possible in contrast to waterfall, where bigger is considered better. … Long changeover times have meant that economically it was sound to create out as many features as possible. In Scrum this does not apply. Requirement changes are made quickly and improved even more with practice. In the 1980s it took two to three hours, in the 1990s it dropped from one hour to 15 minutes, now it takes three minutes.”

I have personally seen feedback from our user experience (UX) team make it into a shipped product in less than a month from when they did user testing on the product that was in development. That type of responsiveness is impossible in a waterfall model because the UX would be done in a batch that would then lead to a batch of requirements, and so on. Just as in automobile production, reducing batch production of intellectual inventory allows much more responsiveness to consumer demand and reduces waste.

Waste is probably actually worse in software development than the auto industry. Even if a batch of cars is built that ends up having low demand they probably won’t be sold for less than the marginal cost to produce them, so the inventory still retains much of its value. If a requirement or design is never built because consumers don’t want it then the value of that inventory is zero. All of that investment was lost.

Just one similarity

Using JIT concepts to reduce intellectual inventory is of course just one aspect of Scrum that is similar to the principles of TPS. Here are some others:

• Self-organizing teams with daily status meetings and weekly retrospectives are just a form of Kaizen.
• Ability for any team member to bring any issue to light at any time in the development process is also a similar to the role of an Andon system.
• Using a “pull” system (as opposed to waterfall’s “push”) is basically Kanban.

So the next time you are trying to convince someone in your company of the virtues of adopting a “radical” process like Scrum point out how well these same principles worked for a plucky up-start named Toyota.

Full disclosure: I have an MBA (that’s how I learned about MBA-ish stuff like the Toyota Production System) and am a Certified Scrum Product Owner. ;)

Update: Version 2.0 is out now. Check it out here.

This deficiency became apparent to me while coding something for work where I wanted to use JavaScript to calculate some basic math and statistics: mean, variance, standard deviation, etc. There weren’t any good libraries or code snippets I could find to do these functions easily.

I ended up throwing something together that got the job done but later decided that I could make it a lot more simple and reusable. Turns out that JavaScript’s prototyping capability is perfect for these types of operations. It turns Math.max.apply(Math, myArray) into myArray.max().

These are the methods I have implemented so far:

• Array.prototype.sum(): returns sum of all array values
• Array.prototype.min(): returns the lowest numeric value of an array
• Array.prototype.max(): returns the highest numeric value of an array
• Array.prototype.mean(): returns the arithmetic mean of an array
• Array.prototype.median(): returns the median of an array
• Array.prototype.sortNumber(boolean decending): returns array sorted ascending, or descending if sortNumber(true)
• Array.prototype.variance(): returns the variance of an array
• Array.prototype.stdDev(): returns standard deviation of an array
• normsinv(p): returns lower tail quantile for standard normal distribution function

These are just some basic Excel-type of functions. I am open to adding more functionality to the library, so if you would like to contribute some code or just have a suggestion for a useful function then please leave comment.

Download the script and examples here. You can also link against the minified version here if you would always like to use the latest version.

BTW, I really didn’t want to post anything pro-Microsoft or anti-Apple today, but this was the news I was dealt. :)

A Palm Refugee

Basically you could say that I am a long time Palm user that is growing increasingly impatient. I like the ease of use and efficiency of the PalmOS UI, but the under-pinnings are really starting to show their age. This has been made very apparent by adding a data plan to my phone recently.

I like having the access a lot more than I would have expected; Opera Mini is a great browser but the Java VM that runs it isn’t so much (it crashes regularly). Add on the lack of native Bluetooth A2DP (which my car’s audio system does support), a so-so email client, and Palm’s tardiness with a new OS and you can see why I’m looking for something better.

Honestly I have to admit that the iPhone is probably the best device right now for what I want (strong multimedia, great web browsing, good email client, decent form factor), although it is far from perfect (the phone part isn’t amazing, no built-in search, short battery life with 3G on, no A2DP, etc). So why am I not buying it?

My Apple Epiphany

I must confess that I generally don’t like Apple, and that I think their products are over-hyped most of the time (“Apple is reinventing the home stereo with the new iPod Hi-Fi” –Steve Jobs) but they generally make some good products. The iPod, Macbook Pro, Macbook Air, or Mac Pro are all legitimate top-of-the-line competitive products that most companies’ products do worse than. I realized what my real issue with Apple is though: their business practices.

This is further exacerbated by the fact that when you go Apple your choices are mostly dictated to you by Apple (aka Steve Jobs). Why will Adobe’s CS4 suite be 64-bit only on Windows? An Apple business decision. Why is the iPhone only available on AT&T? An Apple business decision. Why couldn’t .Mac users wait until MobileMe was stable to switch their e-mail over? Again, an Apple business decision.

The problem is particularly pronounced on the iPhone as it is an insanely closed platform (without jailbreaking it). It is like the iPhone is nothing but a DRM device, because basically it is. Lock down my music, check. Lock down my videos, check. Lock down my service provider, check. Lock down my choice of applications, check. Pretty much anything you can do with it is locked down.

Open Platform != Open Source

Don’t confuse an open platform with open source. Windows, PalmOS, Symbian, and even Mac OS X are all basically open platforms (but clearly not open source). You can run any app designed for the platform whether it is specifically blessed by the developer of the platform or not. If Windows or Mac were closed platforms you couldn’t make a third-party application like Firefox because Microsoft and Apple both already have competing web browsers. Look on the iPhone though and you’ll see that Apple won’t let any developer make a competing media player. See the difference?

I have numerous third-party apps on my Treo 680: Google Maps, Opera Mini, Gmail, Pocket Tunes, Facebook, a dictionary, etc. It may seem funny, but it would really bother me to have Apple deciding what I can and cannot use. Simple things like the program I use to track my gas mileage are switching costs to me if there isn’t a viable alternative on a new platform. After Apple’s trend of pulling Apps from iTunes lately I really can’t say I trust them.

Technically there are Windows Mobile 6 phones that have all of the features I want (A2DP, Opera, 3G, wifi, real multitasking) but I just don’t think I could stomach the stodgy UI. So I guess I’m left waiting to see whether Android materializes into something good, Palm can finally bring out their new OS, or hope that Windows Mobile 7 has a new UI, because those will all happen before Apple truly opens up the iPhone.

What is up with the “blogosphere” on this one? Some of the titles would make you think that Microsoft lied about what the software could do when really the only “lie” they told was what the name of the OS is. So I don’t know how Microsoft lied to make them like it. The people in the videos obviously really liked it.  I personally like Vista, but I was genuinely surprised by how much some of these people just fawned over it. They were that impressed.

My History With Vista

I was really skeptical of Vista at first myself. I had tried out the beta versions and hated every single one. Literally the only reason for why I switched my desktop over to Vista was so I could do Media Center on my Xbox 360. My desktop is basically a file and print server for our laptops so I didn’t really care if it wasn’t that great so long as that stuff worked. However, within about a month of having Vista on my desktop I switched over my laptop, and a couple of months later my wife’s got switched too.

I should mention that I didn’t switch over until Vista had been out for about six months, so I missed out on the launch-day issues, but I never switch over to a new operating system when it comes out. No matter who makes it, new OSes always have some somewhat significant bugs or quirks. I would have probably hated Vista in January of 2007, but Vista in August 2008 is a different story. When I saw ExtremeTech defending Vista I knew the tide was turning for Microsoft.

For more of my ramblings on Windows Vista and XP check out this post.

Just look at their 1.0 product short comings that a subsequent version fixed:

• iPod: firewire only, no iTunes for Windows, no service to replace old batteries, mechanical scroll wheel
• iPhone: no 3G, no GPS, no third-party software, no contacts search, no corporate e-mail/contacts/calendar sync, 4GB model, couldn’t easily use third—party headphones, no music ringtones, etc
• iPod Touch: pretty much the same list as the iPhone but you have to pay for each update even though they are free for the iPhone and new iPod Touches
• AppleTV: couldn’t purchase or download content on the AppleTV, measly 40GB hard drive, no support for Dolby Digital 5.1, had to be connected to a computer to do anything, apparently people are still unhappy with the state of AppleTV
• Mac OS X: launch version had almost zero software, ran very slowly, no DVD playback, no CD burning, no Windows/Samba file-sharing, no built-in search
• MacBook: palm rest discoloration, cracking plastic, low quality 6-bit LCD panels, you could only order it with 512MB of RAM, had draft-N wireless support but you had to pay $5 to use it, excessive heat made Apple label it a notebook instead of a laptop (because it is too hot for your lap apparently)
• And now MobileMe: “1%” of users couldn’t access their mail for weeks (as of this writing I’ve seen reports that some still can’t), Apple’s idea of PUSH technology isn’t actually a PUSH at all, exchange contact and calendars don’t sync, and now they are adding MobileMe software onto non-MobileMe users’ PC via iTunes without asking or even telling them

Basically, within a year or two of each product coming out a new revision/version comes out that fixes the glaring bugs and notably missing features, and sometimes even costs less. If you ask me, it really does seem like 1.0 is more of a paid public beta test for Apple. The thing that amazes me is how they can get their users to forget about all of this. That is some amazing marketing…

autoHideStatusbar: Quite simply AHS hides the status bar unless you hover over a link or move your cursor to the bottom of the browser window. It is great for maximizing vertical space within the browser window. I don’t know about you, but most of the time I don’t need the status bar but sometimes I do.

Image Zoom: This is another simple extension that can be really useful. It adds a context menu item for zooming in and out on an individual image on a page. It makes it a lot easier to see details in smaller images when you can easily enlarge it by 200%. I would like to see a bit more friendly UI for this one however. The context menu approach works, but some sort of hover over pop-up may be more intuitive.

Speed Dial: I’m sure all of the Opera users out there recognize this one; it is a blatant copy of the speed dial built into Opera. Imitation is the sincerest form of flattery right? When you open a new window or tab this extension will populate it with a three-by-three (although you can adjust the row and column count) set of tiles of sites you selected. I have mine set for five columns with three rows so that the most common fifteen sites I visit are easily accessible. I know I could (and do) have them in my bookmarks, but this is really just quicker and easier. The tiles also refresh regularly so for certain sites I can tell if there has been an update just from the tile.

Personal Menu: This is a must have extension for me, but I may just be crazy about maximizing my screen space. This extension will remove the regular menu bar and make it a drop down menu from an icon. That way I can eliminate an additional toolbar and save myself some vertical space. Something like this, or perhaps even the method IE uses of pressing the Alt key to bring up the menu would be a welcome addition.

Update Notifier: This extension is one that I can’t believe wasn’t added to Firefox years ago. When an update for Firefox (it works in Thunderbird too), an extension, or a theme is available the icon will turn blue and pop-up a small notification window listing the available updates. You can then install all of the updates right from the drop down menu.

I first thought up this concept about four years ago, before the term “cloud computing” was en vogue. Basically I was thinking how great it would be to be able to pool and leverage the CPU resources of the three computers (two laptops and a desktop) I had within my house. In particular I wanted to be able to use the desktop’s faster CPU to complete workloads for my laptop in a relatively ad hoc manner.

Some video applications I had used could render their output on more than one machine using their own clustering software, but why should it be limited to niche programs like that? I already share disk space, printers, or even TV tuners (via Orb or Media Center Extender), why not my CPU? You can push your computing out on the Internet cloud all you want but a lot of people have a micro cloud of resources already within their home, or workgroup.

At the time it seemed like something that might be esoteric enough to be implemented in Linux so I tried to find out how feasible it was in some Linux developer forums. In my mind I thought it could be implemented as a virtual CPU that the scheduler would only send jobs to if the physical CPU was at 100% for more than ten seconds or so. It would also have to be aware of the bandwidth and latencies of the connection between machines. Obviously network accessed CPU resources wouldn’t be incredibly efficient but any additional processing cycles gained would help.

As the number of cores continues to increase (particularly on the desktop) and devices like netbooks with limited resources become popular this could have been a huge boon. Unfortunately nobody took me seriously. After all, my kernel-level C programming is severely lacking. :)

If the documents SDTimes has received are accurate it would seem that Midori is all about cloud computing. Or put more accurately, heavily abstracting away hardware from software to enable remote or local resources to be used for any given task. It isn’t just for the CPU either. Imagine being able to easily pool the disk storage available on the various nodes of your network into one huge fault-tolerant distributed storage volume. All I can say is that on paper this all sounds incredible.

Microsoft is obviously pretty quiet on the purpose of Midori. The PR people have admitted its existence but they it is just an incubation project; only time will tell. Hopefully it will see the light of day before I die or Duke Nukem Forever comes out, which ever comes first.

*Update: Must be a pretty big story if even the BBC is running it.

This isn’t the first time I’ve heard mostly false statements like this, but I expected more from The Economist. This isn’t going to be a post about why you should or should not upgrade to Vista though, it is about the truth surrounding XP’s adoption. The facts after the jump.

Name That Windows Upgrade

If I told you that I had been reading about the “slow, tortoiselike” adoption of a Windows OS, where people were complaining about a slow UI, how some really old peripheral or software wouldn’t work, and where businesses felt the required investment in upgraded hardware “could be cost-prohibitive“, which version would you guess it was? Vista? Well I have news for you, I was reading about Windows XP; Vista’s now (apparently) beloved predecessor.

I have to admit it is a pet peeve of mine when people act like Windows XP was a blockbuster from the get-go because the fact of the matter is it wasn’t. About the news that Intel isn’t switching to Vista yet, guess what? They were really slow to switch to XP too, and it really shouldn’t have been news back then either.

The Speed of Corporate IT

Anyone who has worked in any sort of large scale IT environment knows that businesses do not rush to change the latest version. In 2005 I was working in an IT environment where all 2,000 PCs we had were finally switching over to Windows XP from Windows 2000 after XP had been out for four years. Windows XP’s adoption was a lot like Vista’s is now, primarily people buying new machines, with businesses slowly embracing the upgrade.

When XP launched in 2001 Gartner said that Windows 2000 Pro would “continue to be the leading business version until 2003.” In 2005 AssetMatrix did a study that concluded “that Windows 2000 is installed on 48 percent of all corporate PCs as of the first quarter of this year, only falling four percent since the last quarter of 2003.” In fact, at that time 10% of companies were still running Windows 95 for some reason.

Even once Windows XP had significantly started to penetrate corporate IT, they were really slow on implementing Service Pack 2. In fact E-commerce Times said that “a substantial number of companies have yet to decide whether to accept or embargo Windows XP SP2.” (emphasis added) Seriously people thought about skipping SP2 somehow?

This is how corporate IT works. If it ain’t broke don’t fix it. Nobody wants to lose their job over a hasty upgrade. Windows XP did have a ton of bugs when if first came out. In fact I personally had a lot more issues (hardware, software, compatibility, stability, etc) with the first version of XP than I did with pre-SP1 Vista.

My Two Cents

Honestly, Vista is a better upgrade from XP, than XP was from 2000. I held out on Windows 2000 for quite a few years, but Vista got me to upgrade within 6 months. I am not about to say that Vista doesn’t have its warts, but on decent hardware it runs really well and I can’t imagine going back to XP.

Vista’s Explorer UI is way better (breadcrumb navigation, extra large thumbnails for photos, preview pane, etc), the integrated search is literally the only one I’ve ever liked (and I’ve used F-spot, Beagle, Spotlight, Google Desktop Search, Windows Desktop Search, etc), it is more secure, Media Center is awesome, the network location management is great for laptops, I’m a fan of the application specific audio mixing (ala BeOS), even just simple stuff like how clicking on the time brings up a calendar and multiple clocks is nice.

When XP came out it had very few benefits over Windows 2000. It supported USB better (but that was added into Windows 2000 by a service-pack), it had a slightly different (but not better) UI, it had a worthless broken firewall, and it had fast user switching. It was a lot more stable than Windows 98, but so is a three legged dog, so is that really saying anything?

Ultimately, I think a lot of people don’t like change. People don’t want to learn a new way to do something even if it is way better. It is just really hard to push this much change. By the time Windows 7 comes out everyone will be used to Vista and XP will be long forgotten. My prediction is that Windows 7 will be to Vista what Windows 98 Second Edition was to Windows 98, and it will be very popular.

In my humble opinion I think it is foolish to disregard the effects of piracy with the argument that the people who share copy pirate (argh me maties!) wouldn’t pay anyway. This argument isn’t just used for Microsoft software either, it is a common argument used with pirated music and PC gaming as well.

Copyright

In each example it really boils down to copyright though. The dictionary defines copyright as “A grant of an exclusive right to produce or sell a book, motion picture, work of art, musical composition, software, or similar product during a specified period of time.” (emphasis added)

It is the copyright holder’s exclusive right to choose how their work is distributed. For some reason, whether it is Microsoft or Metallica there are people who think that these exclusive rights are somehow irrelevant because they already made their money. Copyright holders really shouldn’t have to prove that the piracy of their works (whether proprietary or open-source/GPL/etc) could have been turned into actual sales; that is beside the point.

A Different Kind of MSN

Without taking away from what I have already said, I don’t know why Microsoft chooses (it is their choice after all) to be so aggressive on their licensing enforcement and policies. For me, it all points towards Metcalf’s law on network effects. The value of the Microsoft “network” or ecosystem of software is arguably highest when everyone (paying customers and pirates) uses it.

There is a segment of users, especially in certain regions, who will never pay much or anything for their software; and not because they care about open-source or libre software. There could be many reasons why they don’t care to pay, but the reasons don’t really matter. As Microsoft really pushes their pricing and licensing enforcement, they will push these users toward free OSes like Ubuntu (or other Linux distros).

Ultimately this will diminish the value of the Microsoft “network” and increase the value of the alternatives. Indeed a big part of the reason why Linux (or even Mac to a lesser extent) isn’t more popular (i.e. valuable) on the desktop is that the network is too small.

Microsoft’s time and money could be better spent exploring reduced pricing in various regions; and no, a gimped “starter edition” doesn’t count. Yes, this would open up gray-market issues, but I’d rather deal with some people not paying the right price than some people not paying at all, or even worse, some not even using the software.

Note: I am not in any way condoning or justifying piracy.

Remember XYZ OS? It Was Great

I’ve heard statements like this about every new operating system or office suite for many years now. Oddly enough some people are particularly fond of DOS, Windows 3.11, or especially Windows 95. In their mind’s eye the software didn’t have viruses, didn’t crash (well, DOS usually didn’t), wasn’t “bloated”, and ran on a measly 33MHz CPU with 4MB of RAM just fine. Sounds pretty great right? Wrong.

The software also didn’t do very much (relatively speaking). There was little or no multi-tasking (I upgraded to OS/2 from Windows 3.11 solely to download from BBS‘s in the background). Then there was that arcane 640KB memory limit in DOS you had to deal with.

There was also no Internet (at least what we consider the Internet today); so no WWW, email, blogs, instant messaging, VoIP, or online gaming. Computers also couldn’t do all of the multimedia (music, video, photo editing, record TV, etc) we take for granted today. Here is the real kicker though, they also cost more in nominal and real terms. I remember my first hard drive cost $300-400 and it was only 20MB! I just bought a 750GB drive for ~$100 or so.

Trust Me, It Wasn’t As Good As You Remember It

Here is how I know it wasn’t better back then, if it was, we would all be downgrading back to a 286 running Windows 3.11. A lot of the problems people have with computers today have more to do with the Internet than their OS. If you really want to be as safe from viruses/malware as you were then, don’t connect your computer to the Internet, ever. That will take care of about 99.9999% of your computers problems…and about 90% of its functionality too.

I do have my favorite programs/OSes from back in the day, but they were really only great relative to their contemporaries. That is why I run Vista instead of BeOS or OS/2 (arguably my two favorite OSes of all time). Sure BeOS could boot in 5 seconds on a Pentium 200MHz, but once it was up what would I do with it?

Standards behind the “free and open Web”

This isn’t the first time I’ve heard this statement, but using a term like “free and open” is such utopian propaganda. After all how could you be against “free and open” right? A brief look at the web standards groups might illustrate the real root of the problem though.

The W3C (World-Wide-Web Consortium) is the main standards body for the web. To say that they have a reputation for being slow is an understatement; their last XHTML/HTML recommendation (XHTML 1.1) was in 2001. That was seven years ago, or almost eternity in Internet or dog years.

Eventually it got so bad that some people from Apple, Mozilla, and Opera forked off into their own group called WHATWG (Web Hypertext Application Technology Working Group) in 2004. They started, and are still working on, the draft of HTML 5 which has finally been adopted as the starting point for the W3C’s new HTML working group. Unfortunately, according to the WHATWG editor for HTML it doesn’t look like HTML 5 will be done until 2012; eight years after the WHATWG was formed, and eleven years after XHTML 1.1. That sounds like a rapid pace of innovation to me.

The real culprit

This may seem like a forgone conclusion to many of you after seeing the W3C’s development timetables, but the real reason Flash and Silverlight exist is because the “open web” people dropped the ball. HTML simply can handle what Flash and Silverlight can do. It has become increasingly stale for modern web development needs.

Here is some perspective, HTML5 has finally added a tag for handling video. Flash 6 came out with video support in 2002! Where is the HTML version of Line Rider? It is in Flash and Silverlight now. If you want to see something really interesting check out Hard Rock Cafe’s memorabilia page (Silverlight 2 required) and tell me if you’ve ever seen something like that with HTML. (Here are some other interesting examples)

I actually hate Flash, but I’m not going to blame Adobe for the fact that so many people and companies have decided to use it. It isn’t like Adobe is paying people on MySpace or bloggers to use Flash widgets. Youtube could have really only happened using Flash too.

AJAX to the rescue?

What about AJAX and all of those Web 2.0 sites though? They seem pretty sophisticated right? In short AJAX is a kludge of various technologies that were never intended to work together in this manner. It can work, but AJAX development is a pain. It gets even more complicated when you start to mix in other aspects of the “free and open” Web like SVG or CSS. It is anything but a cohesive set of technologies.

The real weak spot is in the development tools for “free and open” technologies. There are no AJAX development environments that can compare to the tools available for Flash and Silverlight, and the latter has only been out for one year. It is so bad that people made a big deal over a framework to make AJAX development a little easier.

Honey and Vinegar

I’m not against the idea of a “free and open” web, but obviously there is an increasing demand for a richer experience than that offered by the W3C’s dated technologies. After all there isn’t just one, but two major competitors to them.

If the web is going to steer clear of these proprietary environments the proponents of the standards will need to create the technologies that enable innovative new online experiences instead of just copying implementing features that have already been done before elsewhere. Complaining about the proprietary web won’t do anything, after all you catch more flies with honey than with vinegar.

The browser wars have been heating up lately with the recent releases of Firefox 3 and Opera 9.5. As a long-time Phoenix Firebird Firefox user, and recent convert to Opera Mini (which is excellent btw) on my Treo, I thought I’d run these two through their paces to see what they are made of. The hits and misses after the jump.

Opera 9.5

Opera 9.5 actually came out about a week or so before Firefox 3 so it was the one I tried out first. Opera takes a kind of “kitchen sink” approach, and is arguably the most feature packed browser out there. For the most part this approach works really well. Most browsers couldn’t duplicate the functionality of Opera, and it would take dozens of extensions (which often don’t work when new versions of Firefox come out) to try to pull off the same effect in Firefox.

Here are some of the notable features built-in to Opera:

• Built-in browser sync (even to Opera Mini on my Treo!, try that with Weave)
• An excellent download manager
• Built-in ad content-blocker
• Session management
• Wand (auto-form filler on steroids)
• Web development tool
• A unique trash can approach for recently closed tabs
• A novel speed dial start page

The best, and most notable, “feature” of Opera is its speed however. It really is incredibly fast. It starts almost instantly; actually everything happens pretty much instantly. It is easily the fastest full service (i.e. not K-melon) browser I’ve ever used. Opera also has a pretty good security track record and is even more obscure than Firefox (smaller target for hackers).

While I really appreciate that I don’t have to basically roll my own browser (I’m looking at you and your extensions Firefox), Opera may suffer from a few too many features. Does a modern web browser really need an IRC or email client? Both clients are just average implementations, and the really odd part is how they show up as tabs in the browser right next to web pages. That said, you don’t have to use or enable them if you don’t want to.

The biggest problem I have is with the rendering in Opera 9.5. While most (>95%) of the sites I visited rendered fine, quite a few blogs and Netflix didn’t look right. Even though most of the blogs probably suffer from poor HTML coding and lack of compliance to web standards, it is still something that was a bother. Firefox has always rendered quirky sites well.

Firefox 3

Now onto the reigning “alternative” browser champ, Firefox; here is the quick take on its latest installment. The good: all of the features of Firefox 2. The bad: not really anything new since Firefox 2. The speed has improved a bit and it hasn’t crashed on me yet (should that really be a feature?), but I really have to ask myself what else Mozilla has been doing during the almost two years between Firefox 2 and 3.

Here are the only features I’ve found notable in Firefox 3:

• Scales/zooms images using a higher quality bilinear (or maybe bicubic, see image to the right) method instead of using a low quality “nearest neighbor” approach (finally someone did it!)
• Download manager back-end has really been improved (resume actually works now), it is too bad they ruined the front-end UI for it however

There really aren’t many other new features but most of them fall under the “different, but not necessarily better” category for me. Yeah, yeah, the bookmarks use a database now, but I’m still trying to figure out how that benefits me or why I should care, the “AwesomeBar” is a bit short on awesomeness, and the one-click bookmarking is only easier if you never organize your bookmarks.

The Verdict

The verdict? I’m still using Firefox. The page rendering problems and something about the Opera UI just doesn’t suit me well. Other than that though it is clearly the better browser; I will definitely be following its development. I am probably just too used to Firefox really; especially the keyboard shortcuts for tabs. If I was already an Opera user I could not think of a single reason why I would switch to Firefox though. If you aren’t happy with your current browser you should definitely check Opera out.

…one more thing about Firefox

Back in the day one of my favorite web browsers didn’t really improve much for a long time, remember it? It was Netscape 4 and they weren’t even the underdog. If Mozilla wants to maintain their momentum they need to bring their A-game, there is some stiff competition these days (Opera, IE, Safari) who are all actually innovating.

Money and Self-Preservation

Of course the reason Mozilla doesn’t make you choose a search engine (like Internet Explorer does) is because if they did, they’d lose almost all of their revenues. Mozilla’s number one (and nearly only) source of revenue is an advertising revenue sharing program it has with Google. Of course Internet Explorer used to default to MSN Search, but they were forced to ask users after companies like Google started suing them over it.

Corporations are just like people; they are extremely interested in their own self-preservation (read: money, for corporations). It is easy for Google and Mozilla to talk about building an “open web platform” when that is in their best interests. You can see that they aren’t always for openness and choice, though.

Case in point, Google sued Microsoft because Windows Vista’s search can only use Microsoft’s own built-in search (Service Pack 1 changed that because of the lawsuit), but does Gmail let you use a different search? Nope. Does Google Talk natively support any other IM networks? Nope. Does Google Earth allow third-party search results? Nope. But I thought they were all about choice and openness?

Mozilla doesn’t even list Live Search (which I recently switched to and actually prefer now) as an option. If they were truly for openness then surely the number three web search would be included above “Creative Commons” (who knows why that is there) right? Firefox also makes itself the default web browser when you install it; again Internet Explorer makes you choose.

I’m not saying Google and Mozilla are the devil, just that their motives are the same as Microsoft. In truth, it could be argued that Microsoft’s products are now more open to choice than either of these other companies. Google is at the stage where they are following all of the big bad anti-competitive things (exclusive OEM deals, suing companies to damage their products, etc) that Microsoft used to do.

When companies are still new and small(ish) it is easy to say you are truly for openness and the consumer. But as soon as they are established at all, the game changes. Firefox came out nearly four years ago, and no matter how plucky Mozilla tries to act, there are a lot of people who only truly care about protecting their jobs and the core product of the company they work for. If that means ensuring a Google (their #1 customer) search engine monopoly, then so be it.

Honestly, I wish everyone would stop suing and just compete on the quality of their software. One of the main weaknesses of Vista versus Mac OS X is that it would be illegal for Vista to have that level of integration. Everywhere you turn in Vista it has to ask you which search engine you’d like to use, what music store, or if you want Windows Media Player to be your default music program. Do you ever see that in OS X? Nope.

Get this though, they are launching this “Download Day” promotion and they don’t even know when Firefox 3 will be coming out, “but it should be in June.” Rule #1 for any sort of promotional day, figure out when it will be before you tell people about it. There also isn’t any existing record so they want to “outdo the number of Firefox 2 downloads on its launch day” which was only 1.6 million, but they throw out 5 million as a sky’s-the-limit goal.

Now I don’t have any of Microsoft’s server logs, but something tells me that with the Windows install base nearing one billion, they have probably had software with more that 5 million downloads in a day at Windows Update alone. It might even happen every month on patch Tuesday.

Mozilla should stick to focusing on the software and leave the gimmicks at home. But with it looking like Firefox’s only compelling improvement being that it doesn’t leak memory like a sieve (which is more of a bug fix right?), I guess they have to get what they can get.

You can follow their race for the record on various social networks, twitter, and even “pledge” on SpreadFirefox.com. How cute.

For all the Vista users out there, I recently stumbled across an awesome program for the Vista Sidebar. It is called Amnesty Generator, and basically it will take the code for any web widget (think Google Gadgets, ESPN, etc) and turn it into a Vista Sidebar compatible gadget. It is a simple program, but works great; I use it to display the NBA.com live sports scores on my desktop. Check it out. There is also a Mac OS X version, but I have not used it.

Next up on the block is Nintendo’s forthcoming Mario Kart release for the Wii. Forget Zelda or Metroid, Mario Kart may be the most important franchise Nintendo has made in the last decade or two. All three people Everyone who had a N64 or Gamecube had Mario Kart; it is probably the only game I really played on Gamecube. It single handedly kept Nintendo in the console business, IMHO.

The formula is simple, easy and fun racing antics for you and your friends. Really it is about playing with others and yelling at them when they use a power-up on you. “Damn you Carl! That was the luckiest turtle shell ever!” The problem is that Mario Kart for Wii will not have voice support for online play (of course you can still taunt in person :)).

Maybe it is just me, but I don’t get it. Why doesn’t Nintendo come out with a headset for the Wii (it uses Bluetooth after all)? They made a steering wheel for Mario Kart, and I would think that another accessory to buy would fit perfect with Nintendo’s current accessory fetish strategy.

Mario Kart online won’t be half of the fun it could be with voice support. I understand that they may be concerned about the unwanted jagged words you often hear on Xbox Live, but limiting voice chat to friends-only by default would take care of that. When I read posts like this of people using Xbox 360s to do voice chat while they play Wii games, you know something is wrong.

Lastly, and certainly not least, why do people care what Kevin Rose says or does? (This is a real question, feel free to comment). I know he runs a website that is almost as popular as TechConsumer (ha!), but really, why do people care? It was the Kevin Rose 3G iPhone rumors post about back-to-back webcams that really pushed me over the edge. It got coverage on AppleInsider, Gizmodo, Techmeme, and of course Digg.

Kevin Rose has had one good idea in his life so far (which appears to be easy to copy and make better), and that’s it. Digg initially thrived under the so-called “wisdom of crowds” but it is now languishing under “mob rule.” Interestingly enough, the mob has a fancy for Paris Hilton, all products from Apple, and stupid ugly cats (and squirrels). More people on Digg read what Kevin Rose says than what Ben Bernanke says (even with this “recession”). Rose’s predictions were way off on the iPhone 1.0, so why would I listen to him now? He doesn’t work with or for Apple (actually his business relationship is closer with Microsoft).

So again I ask, why does anyone listen to Kevin Rose? If you can actually give me a good reason why, I’ll be surprised.

I am going on record that, unless Mozilla changes the direction they are heading, Internet Explorer will push Firefox back to single digit market share within three years. I’m sure a lot of people will flame me that Microsoft could never pull that off, but try telling that to Netscape. Each incarnation of Firefox since at least version 1.5 has not been a significant improvement on the previous version. Yeah, they claim all sorts of new features, but really, what is going on with Firefox 3?

Sure it is supposed to use lower memory, but the user interface is seriously going downhill. They seem so insanely focused on the underlying technologies (which isn’t necessarily bad) that they don’t spend any time on the UI. Look at the new download manager; functionally it actually works a lot better, but I’ll be damned if my parents (or any other average user) would ever figure out how to use the new features it has.

Then there are the areas where they have changed the underlying system dramatically (database driven history and bookmarks) that really don’t work any better than the old way. I ran FF3 Beta 3 for about a month, honestly the new bookmarking/history features adds about 5% benefit (it does the search in the location bar instead of in the history sidebar) but at a cost that every extension written for bookmarks or the history don’t work. That is a major deal breaker for me. Maybe Firefox 3.5 will actually expose more useful functionality for this feature, but it isn’t that great right now.

My real problem is the direction that Mozilla is taking Firefox. Their handling of adding features and juggling extensions is a joke. They consistently seem scared of adding a new feature that would be genuinely useful to the average user. I have seen this first hand on Bugzilla with a feature request for save to PDF support. Firefox 3 has the built-in underlying technology (through Cairo) to save web pages to PDF, but they don’t want to add it as a feature. They just say that it should be delivered through an extension (which exists). The code is all there except for a user interface to expose the functionality to the user!

Every user has to go out and find the extensions though. Why can’t Firefox have official/recommended extensions (maybe weather, gmail, etc) that can optionally be installed with Firefox? Or why aren’t there different versions of Firefox? Just think, they could make a Firefox Developer Edition that would come with many common web developer extensions like Firebug, Web Developer, or HTML Validator.

The biggest problem with extensions is that they never work from version to version. Firefox is a terrible platform in this regard. It is ridiculous that by far most extensions won’t support FF3 it before it launches. But the Mozilla folks seem to believe that that doesn’t matter. Why is it that extensions constantly have to be redone for new versions of Firefox? Hell, even Firefox 1.0 extensions didn’t work on 1.5 but “add-ons” for Internet Explorer 6 still work on version 8!

This finally gets me to Internet Explorer 8 Beta 1. The bottom line? I kind of like it. They are actually going in a direction that I’m interested in with features like Webslices and Activities. Basically, Webslices allow you to subscribe to a portion of web page; similar to how you sign up for an RSS feed. It fits a very different usage scenario than RSS feeds though. A Webslice could be a eBay auction you are following, the status of a friend on a social network, or the latest news headlines. Here is a link to Microsoft’s page and a video on Webslices.

Internet Explorer Activities allow you to select text (anchors can be embedded in the page too) and get context sensitive options. The most obvious example is selecting an address to get a map. The cool part is that the activity can show information (like a map) without leaving the page (see image below). Any website can create activities for IE8 as well; it isn’t locked down to just Microsoft services.

The best thing about Activities and Webslices? They just come built-in to IE8. They aren’t some “great” extension that only one in twenty users of a browser with 15% market share have. So with Firefox the feature will be so uncommon (3% of web users) that no web developer can really target it. Within a year of IE8 coming out it will have more than 50% of the market. Consequently, websites will actually implement Activities and Webslices.

Another illustration of where IE8 is going is that it includes a Firebug-esque development tool built-in. The Mozilla people need to come to grips with the fact that a huge amount of the “value” of Firefox to users is found in the extensions. They try to position Firefox as an extensible base platform with a rich ecosystem of add-ons, but the add-ons break between every single version. That is, if the average user has even found or realized that they can add those add-ons.

Bottom line: I haven’t switched to Internet Explorer yet, but if Microsoft and Mozilla keep the trajectories they are on I can’t rule it out in the future.

*Disclaimer: I have been using a Mozilla browser as my primary browser for six years. First Mozilla (aka Seamonkey) v0.95, and then Firefox when it was known as Phoenix 0.6. I have been exposed to some of the development activities on Mozilla’s Bugzilla too. So I don’t want to hear that I’m just some Microsoft fanboy.

The Good

Lean: Overall Firefox seems so much leaner this time around. Even after hours of browsing with dozens of tabs open Firefox 3 is using about one-third less RAM than I typically see Firefox 2 use. The RAM savings didn’t come at the sacrifice of performance though, everything is notably quicker. Going back to previous pages, opening new ones, even the auto-completion when I typed in a URL seemed quicker. Even Google Maps seemed more responsive.

Features: While there is an entire list of changes in Firefox 3, Mozilla has added a notable one. The history and bookmarks have been combined into one database driven section called Places. Don’t be worried that the the UI has changed too much, on the surface most users won’t really notice the difference; it is more of a back end thing. They did add a new “Places” folder on the bookmark toolbar which can show recently viewed pages, tags, or starred pages. In addition, there is a completely revamped bookmarks organizer that will allow you to search your current bookmarks or history as well.

Some of the changes are much more subtle. The search box has been changed so that you can now resize it to any arbitrary size you want. When you scroll through tabs when there are more than can fit on the screen, they have added some animation to make it more clear what is happening.

If you zoom in or out on a webpage (ctrl-plus or ctrl-minus) you will notice that the whole page zooms now instead of just the text. While it is a nice feature in practice, the images look horrible when scaled up. I am still waiting for a browser that will do a smooth (read: bicubic/bilinear, not nearest neighbor) resize of a scale image. If the images looked good, this could be a major feature for those with old eyes that would just like everything to be bigger on the high DPI screens being sold today. It should be noted that version 3 also remembers your page-zoom settings on a site-by-site basis now too.

The Bad

Extensions: It can be summed up in one word, Extensions. While the extensibility of Firefox is a major feature (I probably like my set of extensions more than I like Firefox really), they are a huge problem when it comes to upgrades. Out of the eleven extensions I use, only one works with Firefox 3. That means, no weather, Gmail, Google Toolbar or Bookmark Sync, Image Zoom, Firebug, etc. Now I know some of these will probably be compatible by the time version three dot zero is released, but I’ll bet most of them still won’t. And until 95% of them work, I won’t be upgrading to Firefox 3.

The Verdict: Firefox 3 is a solid, but progressive upgrade. I won’t be adopting it though until at least six months after its release. I don’t know what the technical solution is for the Extensions, but Mozilla needs to figure out something with this. It is unacceptable that one of the biggest features of their product is incompatible from version to version. It happened when 1.5 came out, 2.0, and now 3.0. This is an area where Microsoft has typically excelled.

* Disclaimer: If you plan on checking out Firefox 3 for yourself, make sure you backup your Firefox user profile first.