efeefe

[NB: this could easily have been titled 'the software industry's innovation problem', since the problem applies broadly to all sorts of software development, and what I'm talking about as 'change' is often referred to as 'innovation', a word that has been twisted almost beyond recognition. I'd like to focus on this little corner of the industry today, so the linux desktop's change problem it is for now.]

A couple things recently (combined with writing this post the other day) made me think about the difficulties in bringing change to the linux desktop.

• talking with someone about distributor innovation brought up this post by Vincent. Solid post that I mostly agree with (bottom line: distributor changes done without upstream input are doomed to failure) but the comments raise a challenging and difficult-to-rebut counterpoint: distributor changes done with lots of upstream input have a history of being watered down, shouted down, and hence failing to make revolutionary change. Incremental or additive change, fine; revolutionary change, not so much.
• discussion of javascript in the desktop reminded me of Pyro. Obviously Pyro wasn’t perfect, but the speed with which it was shouted down, despite the advantages it might have brought (easy web integration, someone else doing optimization and a11y for us, etc.) was very troubling. To paraphrase what a friend said at the time, ‘developing an online desktop in GTK seems like flying a private jet to an environmental rally- it might be justifiable, but it suggests you aren’t really that serious about the benefits of the new system you’re claiming to embrace. If you really think the web is that great, you have to take Pyro seriously.’ What can we learn from the differences in their receptions?
• discussion in this bug about the Sugar filesystem is fairly typical of what happens when you try to implement radical change- people used to the old system focus intensely on the transition costs (it doesn’t work RIGHT NOW and my old system WORKS RIGHT NOW DAMMIT) and give varying levels of thought (usually little) to the potential upside of the change- maybe tagging and search really have vastly more potential than hierarchies now that our computers have more capabilities than they did in the time of Aristotle. Kudos to the Sugar folks for persisting despite that resistance.

This isn’t to say that resistance to change is always a bad thing. Plenty of people have brought fairly cracked out ideas to the GNOME table that deserve to be shouted down; Sugar would certainly have been helped by better communication about their vision, a strong dash of pragmatism and a better sense of how to pick your battles; and Pyro might have been better received if the presentation at GUADEC hadn’t been frequently described as ‘condescending.’ But there is a theme here- lots of shouting down of new ideas is being done. Sadly, you can even make a fairly good argument that this is the default reaction to new ideas.

For a long time, this problem didn’t matter that much to the linux desktop. There were plenty of problems to solve just to get usable at all, competitors put out shoddy products, innovation by others happened very slowly if it happened at all, and there was no disruptive change in the basic model for delivery of apps. This basic state of affairs lasted for nearly 20 years. But now things have changed- the linux desktop has matured to the point where the answer to ‘what next?’ can at least sometimes be something other than ‘fix bugs’, Apple is doing real user experience innovation, Windows is feeling price and quality competition (say what you will, but XP and Office 2007 are miles better than Windows/Office 98), and the web is simultaneously revolutionizing how people collaborate and how almost all modern end-user software is developed and delivered.

It seems like a critical question for the linux desktop, then, to figure out how to make radical change happen, since our competitors are eating away at our traditional advantages while making radical change themselves. It can’t be just incremental change (gradually improved usability) or additive change (bolt on a11y on top), because competitors are delivering all those things and then some. Freedom helps here, especially in the very, very long term, since end-user modification and individual control are not features that Apple and Microsoft can ever offer, and which web services can offer only with great difficulty. But that is a very long term solution. In the mean time, if you don’t want to just imperfectly clone everything they do and then wait for freedom to work its magic, merely being as easy to use as OS X or as accessible as Windows won’t help- something bigger and more substantive has to change.

The question is how to do this change, given the resistance I’ve already suggested is rampant? I’m afraid I have no magical bullet to resolving this resistance to change, but I have some suggestions:

1. The first one is a non-suggestion. The solution is not to do innovation in-house and closed off from the public. Do it that way, and you have all the difficulties that stem from being proprietary (lack of resources, lack of feedback, etc.) and you’ll never get any of the benefits of being open (free assistance in maintenance, free integration in other products, etc.), since odds of it being integrated upstream are extremely low (for good reasons). ‘I want the PR value’ is also not a good reason; the positive PR value you get from that sort of thing lasts only one release cycle, while the damage to your community reputation is permanent. Obviously some balances must be struck so that we can get ideas from as many sources (corporate and community) as possible, but too often pleas for balance are really ‘I don’t know how to do innovation any other way, so I’m doing it this way, sorry’ in disguise.
2. actively encourage incubation of new technologies. Anyone willing to work on changes in a branch should be encouraged with all the resources we have, and with a serious no-stop-energy policy. Constructive, non-bikeshedding feedback only- those running an incubator should feel completely empowered to ban or mute people who aren’t contributing constructively. There is a time for non-constructive feedback, but that time is at the last possible moment. Think of premature stop energy as premature optimization- the costs are almost always higher than the payoffs. To do it right, you have to understand the problem first, and the best way to do that is to encourage someone to do the design and prototyping and write the code.
3. Move development to git or bzr ASAP. Part of incubation of new ideas is to fork early and often, with tools that lower the barrier to creating and testing changes, and which make it as easy as possible to merge those changes back once it makes sense. (Great to see that Behdad is working on that.)
4. Make testing changes easy. Greg seems to be pushing hard for this in the Sugar space, which will be terrific for them in the long term. Some of the push for jscript in GNOME seems to be driven by the same concern- quicker cycles of iteration mean more small changes, quickly, which helps development of new ideas.
5. Encourage a JFDI culture. Part of the problem with bikeshedding is caused by developers who think they are required to talk about things at great length before doing them. A great developer should be transparent, by publishing as much information as they can while they work, but if you’re well trained and well prepared you need not wait to get public feedback on that information before going forward. The best feedback will still be there for you when you’re done- and likely it’ll be even better in reaction to your work. (Note that transparent JFDI implementation after some early internal design thinking is probably a solid feasible halfway point between ‘do it all in-house’ and ‘do it all in the open.’)
6. Encourage a serious innovation culture. We need to be able to tell the difference between serious innovation (what the web is doing to collaboration and development; changing how users experience games) and ‘innovation’ (ooh! shiny!) It would be great if development as a whole (and linux in particular) started to be consciously aware of the difference- that would go a long way towards resolving the imbalance between the two categories. (We did some of the same sort of thing in creating a GNOME culture around ease of use, so it isn’t impossible, though it has been less conscious of late- which is a definite problem, easily visible in some of the newest UI bits.)
7. develop sustainable design best practices: the practices used by designers tend to be optimized for small, tight-knit groups, and many designers strongly believe that too many cooks spoil the broth. Learning how to do iterative, long-term, expert-driven-but-not-dominated design that can respect and incorporate community-driven feedback would be huge. As best as I can tell, no one really knows what such a process would look like yet. Figuring that out should be an explicit goal for someone- perhaps Mozilla (now that they’ve purchased Humanized) or perhaps a university program in design.

It would be great if this post started some serious discussions of the problem and solutions; I certainly don’t pretend that I’m an expert in innovation or that I’d have all the answers even if I were such an expert. And at the end of the day, the #1 reason there isn’t much ’serious’ innovation- anywhere- is that innovation is really damn hard. That I can’t solve either… but some of the other issues should be resolvable as well.

[Ed. later: I removed some childishly snide KDE links, and apologize for the snarky tone therein. I do think that KDE 4's 'innovations' are a prime example of confusing 'shiny' with 'innovative', but I should have said that flat out rather than doing it the way I did. Note that that may well be a better way of failing to innovate than complaining about- and killing- new ideas before they ever get to the user; either way I think both GNOME and KDE have a real problem here, which is why I said 'linux desktop' and not GNOME in the title.]