1 post tagged “weblications”

BY:

http://ifindkarma.typepad.com/relax/2004/12/weblications.html

Yesterday I started reading Paul Graham's Hackers and Painters, and it is wonderful. It literally is changing my perspective about how I think about the world we live in, and where we want to go from here.(Thank you for the suggestion, Aaron and Evan.)

So far the chapter that has really resonated with me most is The Other Road Ahead, in which Paul writes,

With web-based software, most users won't have to think about anything except the applications they use. All the messy, changing stuff will be sitting on a server somewhere, maintained by the kind of people who are good at that kind of thing... Desktop software forces users to become system administrators. Web-based software forces programmers to. There is less stress in total, but more for the programmers...

One thing that might deter you from writing web-based applications is the lameness of web pages as a UI. That is a problem, I admit. There were a few things we would have really liked to add to HTML and HTTP. What matters, though, is that web pages are just good enough...

Because web-based software assumes nothing about the client, it will work anywhere the Web works. That's a big advantage already, and the advantage will grow as new web devices proliferate. Users will like you because your software just works, and your life will be easier because you won't have to tweak it for every new client...

You don't have to ask anyone's permission to develop web-based applications. You don't have to do licensing deals, or get shelf space in retail stores, or grovel to have your application bundled with the OS. You can deliver software right to the browser, and no one can get between you and potential users without preventing them from browsing the Web.

Interestingly, I heard this line of reasoning using different words in the summer of 1998 when I was working at Microsoft. The source was an internal document called Weblications written by Adam Bosworth. To my delight, I have found a public document from May 1998 that contains a lot of Adam's discussions, titled Microsoft's vision for XML:

What have we learned from the Web?

First and foremost, we have learned that it isn't enough for something to be possible. It must be easy, open and flexible. The web predates HTML of course, but until the advent of HTTP and HTML, it didn't really explode. Why? The answer, succinctly, is empowerment. Once HTML and HTTP arrived, more people could play more easily. The solutions were not necessarily optimal from the point of view of performance or even robustness. They were optimal from the point of view of ease of getting started. In short, they were drop-dead simple. Many people point out the deficiencies of HTML, especially because of the sloppiness of its grammar...

The lesson is that simplicity and flexibility beat optimization and power in a world where connectivity is key. There is a second lesson which is key. Applications need to be constructed out of coarse-grained components that can be dynamically loaded rather than single large monolithic blocks. In the HTML world, these components are pages. In the applications world in general, however, this lesson applies. The reason for this is simple. The application starts more quickly, only consumes the resources it really needs, and most importantly can be dynamically loaded off of the net. Why is this so important? It is important because of deployment. Applications that can be dynamically loaded in from a central place don't require some massive, complex and difficult installation process onto clients' machines. Note that Java per se doesn't give one this. It is easy, as anyone who has built a large and complex Java application can testify, to build one, which requires literally hundreds of classes to run. That is monolithic. HTML had the serendipitous effect of forcing application designs to partition the application. To repeat, the lesson is that applications should be loaded in coarse-grained chunks.

It comes as no surprise then that Adam Bosworth recommends Hackers and Painters as well, for it echoes in his own philosophy:

For much (although certainly not all) of the work IT does, IT is like children building sand castles on the beach and watching the tide roll in. That tide is highly customizable web based solutions, Salesforce.com today, perhaps Talaris tomorrow. Ask the average Salesforce.com customer (meaning a salesrep) if he is happier with the solution he has now or the one he had back when IT was building a customer CRM for him. I think the answer will surprise you. Web Services have helped immensely here because it has made possible the integration of these solutions with internal logic for those things IT should still be working on. This is the promise and the future in my opinion.

To me, the passages from Paul Graham and Adam Bosworth articulate well The Web Way. Interestingly, Adam's journey has taken him from a startup called Analytica to Borland, to Microsoft, through a startup called Crossgain, to BEA, and now to Google; all of them were platform companies -- and it's notable that he hopped from a startup to a languages company, to an operating system platform company, through another startup to an application server platform company, and (now) to a web platform company.

I believe Adam's journey represents the evolution of the software industry over the last two decades: from desktop applications running on single-machines that helped individuals with productivity through word processing and spreadsheets and email, to enterprise applications in corporate data centers that helped workgroups and companies with productivity through automating business processes... and now to collaborative applications available to anyone from anywhere on the Internet, leveraging an increasingly-connected and ever-faster world. The web is the platform that subsumes the others.

The software platform of choice evolved from desktop operating systems, to enterprise operating systems, to what Tim O'Reilly calls the emergent Internet operating system. Which reminds me of Jason Kottke's description of the Google operating system:

Google's money won't be made with search... that's small peanuts compared to selling access to the world's biggest, best, and most cleverly-utilized map of the web...

Google isn't worried about Yahoo! or Microsoft's search efforts... although the media's focus on that is probably to their advantage. Their real target is Windows. Who needs Windows when anyone can have free unlimited access to the world's fastest computer running the smartest operating system? Mobile devices don't need big, bloated OSes... they'll be perfect platforms for accessing the GooOS. Using Gnome and Linux as a starting point, Google should design an OS for desktop computers that's modified to use the GooOS and sell it right alongside Windows ($200) at CompUSA for $10/apiece (available free online of course). Google Office (Goffice?) will be built in, with all your data stored locally, backed up remotely, and available to whomever it needs to be (SubEthaEdit-style collaboration on Word/Excel/PowerPoint-esque documents is only the beginning). Email, shopping, games, music, news, personal publishing, etc.; all the stuff that people use their computers for, it's all there.

By the way, Rick Skrenta pointed out (via the New York Times) in April 2004 that Google had at that time 100,000 servers. Eight months later, Charles Ferguson points out (via his MIT Tech Review article, What's Next For Google) in December 2004 that Google has 250,000 servers. If they double roughly every nine months, Google will have a million servers in summer 2006. Ferguson writes,

Today, the wholesale search market has significant barriers to entry. Economies of scale have asserted themselves, secondary competitors have folded, and the creation of new search engines by startups is becoming prohibitively expensive. Consider: to crawl, index, and search more than eight billion pages -- still only a fraction of the Web -- Google now operates a global infrastructure of more than 250,000 Linux-based servers of its own design, according to one Google executive I spoke with, and it is becoming a major consumer of electrical power, computer hardware, and telecommunications bandwidth.

I'm still stuck on the notion that in less than two years Google will have a million-node computer operating as a single, optimized operating system for web-based applications. Google gets it. Most professional developers look at web-based applications and all they see is "the lameness of web pages as a UI", as Paul Graham called it. They grumpily say in Eeyore fashion that web apps are the legacy apps of the future. They don't see that the power of Weblications is that "simplicity and flexibility beat optimization and power in a world where connectivity is key", as Adam Bosworth put it.

It is no coincidence that in 2004 Google hired Adam Bosworth. It is no coincidence that in 2004 Google hired among-the-best-and-brightest-in-the-world web developers such as Chris Wetherell and Aaron Boodman -- and they are searching for more. Because 2004 is the year that Google led the charge in making applications that showcase The Web Way -- using just the simple and flexible mix of dynamic HTML, JavaScript, style sheets, and a DOM-capable browser -- respectable and cool and (most importantly:) useful enough to be considered a viable alternative to The Three M's Of UI Lock-in (Macromedia, Microsoft, and Mozilla), all of which aim to fatten up the client and lock some users in and lock other would-be users out because they don't have Flash or XAML or XUL or any other doesn't-work-on-some-platforms complex soup like Java Server Faces and Struts taglibs. When it comes to simple, ubiquitous, usable user interfaces, Dynamic HTML is one ring to bing them all.

It took a long time -- upwards of a decade since Netscape released the first widely-used, widely-deployed browser -- for The Web Platform to become stable enough to build complete, beautiful applications upon. In 2004, Google's applications like Gmail and experiments like Google Suggest demonstrate that we can have robust, interactive, useful web-based applications. Joyce pointed this out over the weekend in her essay, Google is good for webdev, in which she wrote,

Google is quietly hiring -- they probably have between 5 and 10 of the top front-end devs, which doesn't sound like much until you realize there might be only a couple dozen out there with significant experience. That decreases the supply of the remaining ones, which as we all know means you're gonna have to show them some serious love to enjoy their scarce favors.

Most importantly, Google seems to be putting some chips down on the DHTML side of the table instead of the Flash or XAML or XUL or Laszlo sides. Given how important a few technology leaders are as role models for all developers -- I dunno how many arguments I've had where the magic words, "But Yahoo/Amazon/Google does it this way!" work their incantatory magic -- this is very much a Good Thing. So I've got to thank Google for making it viable to spend money building newer, faster, more responsive, standards-compliant, user-centric, cross-browser interfaces. Whatever the opposite of "collateral damage" is -- collateral benevolence? -- Google is doing it for webdev now.

What's so significant about the development of Gmail? Jon Udell looked under the hood, and was enthused by what he saw:

I’m ready to join the chorus singing the praises of GMail’s user-interface technology. Its combination of HTML, JavaScript, and the DOM makes the browser do some remarkable tricks.

One of my favorite acid tests is address completion. When you begin typing an e-mail address, your mail program should immediately show you the matching addresses and then dynamically constrain the list as you continue to type. Outlook does poorly on this test; you have to type CTRL-K to invoke the address book in a separate window. OS X’s Mail does address completion in situ, just as I expect. So does Gmail. And here’s the shocker: Gmail does it faster.

Gmail’s spell checker is another amazing hack. When you invoke it from the message composer, misspelled words turn red. Click one and a list of choices drops down, ending with an Edit choice. Click Edit and the suspect word converts, inline, to an input box. When you’re done correcting one or more words they merge back into the text.

As early adopters discovered long before I did, there’s an architecture behind this JavaScript/ DHTML wizardry. The best description I’ve found is from Johnvey Hwang, who deconstructed Gmail’s JavaScript code and created a .Net-based Gmail API. As Hwang described in his July 5 write-up, Gmail loads a JavaScript "UI engine" into your browser at the beginning of each session. Oddpost, he noted, was the first Web mail application to perfect this technique. That was a prophetic statement: Just four days later, on July 9, Yahoo acquired Oddpost.

Because Gmail’s behavior is embedded in the UI engine, all subsequent interaction between the browser and the Gmail service is just an exchange of data. What Hwang calls the DataPack format is not XML, though; it’s JavaScript. When you make a request to the Gmail service, whether to refresh your inbox or to modify the list of labels you can attach to messages, the response is a minimal set of JavaScript function calls and associated data objects that the engine uses to update the display.

This is very geeky stuff, I admit, but here are two important points to take away. First, as I’ve often said, intelligent use of browser-based technology can accomplish more than most people realize. You can’t do everything — not by a long shot — but for many of the things that information workers routinely do, even ordinary Web UI is good enough. And now Gmail is proving that we don’t have to settle for ordinary.

Second, Gmail’s architecture is not limited to Web UI. Because it is protocol-driven, developers can create new tools that speak to the DataPack format. Many have done so already.

In Gmail's case, the weblication was better than the non-web-based applications. Gmail raised the entire discussion of what is possible with web-based applications. It didn't go it alone -- it built on the knowledge of web-based applications learned by those who had previously written state-of-the-art weblications. What's significant is that it was able to break through expectations, and proved that "we don't have to settle for ordinary" with web-based applications. Koranteng Ofosu-Amaah clarifies this point in his essay On GMail and DHTML architecture again, in which he writes,

GMail's architecture is actually very generic for a DHTML app. Everyone with a clue should be trying to leverage the browser and that, in essence, is all they are doing.

It is riskier to do more on the rich browser client because it has been a more brittle platform over this past decade. Companies that do middleware and server-side tooling take a while to move from their core competency. Architects that thrive in that environment are essentially conservative and for good reason... Four years later, I now hear mutterings about drag-and-drop and richer clients in our corridors...

This only underscores the point that Jakob Nielsen's predictions about browser adoption cycles have turned out to be pretty accurate. Even though web application developers have been quietly spreading unobtrusive javascript usage in the interim, it is only now that there's a critical mass of clients that can leverage them; when Amazon and Yahoo move, something must be happening.

The developer tools and resources have gotten (slightly) better and there's more experience with the DOM. Increased adoption of broadband also helps reduce latency for the average client so you don't have to fight the inevitable arguments about performance and can couch your advocacy in terms of user interaction. In any case if and when you do have the performance discussion you can always argue that caching as close to the client as possible is a good thing and what better cache than the browser itself. It just so happens that applications like GMail, Bloglines and Oddpost are the state of the art in terms of browser leverage.

I recently wrote about this type of architecture in my recounting of the history of the DHTML spreadsheet and presentation components that are the genetic forebears of OddPost.

The idea is to fetch an HTML skeleton, decide what content you need, fetch that (as XML), and cache it wherever you get a chance. Render incrementally.

The pattern is simple:
Database XML (Optional) JavaScript Object Bindings UI Bindings (HTML) + UI management code

...Perhaps K-station was too bleeding edge trying to go for XML over HTTP, DHTML and extreme leverage of the browser client 5 years ago but that experience was a great testbed for me and I learned lots of lessons about building rich REST-ful applications, the importance of URIs etc.

Again the major missing feature for this rich web application platform is offline usage and synchronization without introducing new security holes in the browser. But then that's why Bosworth is at Google as the rumour goes, right? I suspect he's got other things in mind though...

I think that increased leverage of the browser and the DOM is a good thing. It's also a clear trend and for many applications, the browser is good enough. Good enough for Google, good enough for Yahoo, good enough for me.

And yes, the Beta release of Google Suggest does make me wonder why it wasn't until the end of 2004 that we saw such a web-based application speedily interacting with a web server to complete the form with what you're likely to be typing. We see this kind of interactivity regularly with our Oracle Financials or Peoplesoft HR or Siebel Customer Support system, where the forms are filled in interactively. Why did we wait till the end of 2004 to see such an interactive weblication?

Looking at the reverse engineering of Google Suggest and the Slashdot post where they describe the technique, we discover that

The source for the page is quite simple; most of the work happens in a condensed JavaScript library [google.com]. Not easy reading (note the word "condensed" above, meaning function and variable names are 1 or 2 chars, and all extra whitespace was removed...), but it's actually pretty straightforward.

It disables your browser's autocomplete on that textfield (for obvious reasons). Then it basically just defines a hidden div for that auto-complete dropdown (variations on this depending on browser... frickin' incompatibilities).

Each time you type a character, it populates that div body with the results of a quick, tiny query back to Google. It's NOT running the search for you; it's hitting (I assume) a simple, probably totally in-memory list of the most popular searches and number of results. That's how it can be so quick a response -- the lookup on their end is super-minimal, and the data to be transferred is probably less than 1k each time.

Cool. Nice concept, nice execution. And one of those nice "only obvious in hindsight" additions.

I have the feeling that we've turned a corner, and that more "only obvious in hindsight" web-based application tricks will be developed in the years to come -- thereby solidifying The Web As A Platform and continuing the spread of The Web Way as more users become True Believers who won't give up their web-based applications no matter how hard the "fat, rich client" camps try. As Joyce Park has said to me, "simplicity is its own revelation." It feels as if the world has had tremendous convergence on the thoughts in this post in 2004, and as a result the future looks very bright for The Web Way.