3 posts tagged “web”
BY:
http://ifindkarma.typepad.com/relax/2004/12/weblications.html
Yesterday I started reading Paul Graham's Hackers and Painters, and it is wonderful. It literally is changing my perspective about how I think about the world we live in, and where we want to go from here.(Thank you for the suggestion, Aaron and Evan.)
So far the chapter that has really resonated with me most is The Other Road Ahead, in which Paul writes,
With web-based software, most users won't have to think about anything except the applications they use. All the messy, changing stuff will be sitting on a server somewhere, maintained by the kind of people who are good at that kind of thing... Desktop software forces users to become system administrators. Web-based software forces programmers to. There is less stress in total, but more for the programmers...
One thing that might deter you from writing web-based applications is the lameness of web pages as a UI. That is a problem, I admit. There were a few things we would have really liked to add to HTML and HTTP. What matters, though, is that web pages are just good enough...
Because web-based software assumes nothing about the client, it will work anywhere the Web works. That's a big advantage already, and the advantage will grow as new web devices proliferate. Users will like you because your software just works, and your life will be easier because you won't have to tweak it for every new client...
You don't have to ask anyone's permission to develop web-based applications. You don't have to do licensing deals, or get shelf space in retail stores, or grovel to have your application bundled with the OS. You can deliver software right to the browser, and no one can get between you and potential users without preventing them from browsing the Web.
Interestingly, I heard this line of reasoning using different words in the summer of 1998 when I was working at Microsoft. The source was an internal document called Weblications written by Adam Bosworth. To my delight, I have found a public document from May 1998 that contains a lot of Adam's discussions, titled Microsoft's vision for XML:
What have we learned from the Web?
First and foremost, we have learned that it isn't enough for something to be possible. It must be easy, open and flexible. The web predates HTML of course, but until the advent of HTTP and HTML, it didn't really explode. Why? The answer, succinctly, is empowerment. Once HTML and HTTP arrived, more people could play more easily. The solutions were not necessarily optimal from the point of view of performance or even robustness. They were optimal from the point of view of ease of getting started. In short, they were drop-dead simple. Many people point out the deficiencies of HTML, especially because of the sloppiness of its grammar...
The lesson is that simplicity and flexibility beat optimization and power in a world where connectivity is key. There is a second lesson which is key. Applications need to be constructed out of coarse-grained components that can be dynamically loaded rather than single large monolithic blocks. In the HTML world, these components are pages. In the applications world in general, however, this lesson applies. The reason for this is simple. The application starts more quickly, only consumes the resources it really needs, and most importantly can be dynamically loaded off of the net. Why is this so important? It is important because of deployment. Applications that can be dynamically loaded in from a central place don't require some massive, complex and difficult installation process onto clients' machines. Note that Java per se doesn't give one this. It is easy, as anyone who has built a large and complex Java application can testify, to build one, which requires literally hundreds of classes to run. That is monolithic. HTML had the serendipitous effect of forcing application designs to partition the application. To repeat, the lesson is that applications should be loaded in coarse-grained chunks.
It comes as no surprise then that Adam Bosworth recommends Hackers and Painters as well, for it echoes in his own philosophy:
For much (although certainly not all) of the work IT does, IT is like children building sand castles on the beach and watching the tide roll in. That tide is highly customizable web based solutions, Salesforce.com today, perhaps Talaris tomorrow. Ask the average Salesforce.com customer (meaning a salesrep) if he is happier with the solution he has now or the one he had back when IT was building a customer CRM for him. I think the answer will surprise you. Web Services have helped immensely here because it has made possible the integration of these solutions with internal logic for those things IT should still be working on. This is the promise and the future in my opinion.
To me, the passages from Paul Graham and Adam Bosworth articulate well The Web Way. Interestingly, Adam's journey has taken him from a startup called Analytica to Borland, to Microsoft, through a startup called Crossgain, to BEA, and now to Google; all of them were platform companies -- and it's notable that he hopped from a startup to a languages company, to an operating system platform company, through another startup to an application server platform company, and (now) to a web platform company.
I believe Adam's journey represents the evolution of the software industry over the last two decades: from desktop applications running on single-machines that helped individuals with productivity through word processing and spreadsheets and email, to enterprise applications in corporate data centers that helped workgroups and companies with productivity through automating business processes... and now to collaborative applications available to anyone from anywhere on the Internet, leveraging an increasingly-connected and ever-faster world. The web is the platform that subsumes the others.
The software platform of choice evolved from desktop operating systems, to enterprise operating systems, to what Tim O'Reilly calls the emergent Internet operating system. Which reminds me of Jason Kottke's description of the Google operating system:
Google's money won't be made with search... that's small peanuts compared to selling access to the world's biggest, best, and most cleverly-utilized map of the web...
Google isn't worried about Yahoo! or Microsoft's search efforts... although the media's focus on that is probably to their advantage. Their real target is Windows. Who needs Windows when anyone can have free unlimited access to the world's fastest computer running the smartest operating system? Mobile devices don't need big, bloated OSes... they'll be perfect platforms for accessing the GooOS. Using Gnome and Linux as a starting point, Google should design an OS for desktop computers that's modified to use the GooOS and sell it right alongside Windows ($200) at CompUSA for $10/apiece (available free online of course). Google Office (Goffice?) will be built in, with all your data stored locally, backed up remotely, and available to whomever it needs to be (SubEthaEdit-style collaboration on Word/Excel/PowerPoint-esque documents is only the beginning). Email, shopping, games, music, news, personal publishing, etc.; all the stuff that people use their computers for, it's all there.
By the way, Rick Skrenta pointed out (via the New York Times) in April 2004 that Google had at that time 100,000 servers. Eight months later, Charles Ferguson points out (via his MIT Tech Review article, What's Next For Google) in December 2004 that Google has 250,000 servers. If they double roughly every nine months, Google will have a million servers in summer 2006. Ferguson writes,
Today, the wholesale search market has significant barriers to entry. Economies of scale have asserted themselves, secondary competitors have folded, and the creation of new search engines by startups is becoming prohibitively expensive. Consider: to crawl, index, and search more than eight billion pages -- still only a fraction of the Web -- Google now operates a global infrastructure of more than 250,000 Linux-based servers of its own design, according to one Google executive I spoke with, and it is becoming a major consumer of electrical power, computer hardware, and telecommunications bandwidth.
I'm still stuck on the notion that in less than two years Google will have a million-node computer operating as a single, optimized operating system for web-based applications. Google gets it. Most professional developers look at web-based applications and all they see is "the lameness of web pages as a UI", as Paul Graham called it. They grumpily say in Eeyore fashion that web apps are the legacy apps of the future. They don't see that the power of Weblications is that "simplicity and flexibility beat optimization and power in a world where connectivity is key", as Adam Bosworth put it.
It is no coincidence that in 2004 Google hired Adam Bosworth. It is no coincidence that in 2004 Google hired among-the-best-and-brightest-in-the-world web developers such as Chris Wetherell and Aaron Boodman -- and they are searching for more. Because 2004 is the year that Google led the charge in making applications that showcase The Web Way -- using just the simple and flexible mix of dynamic HTML, JavaScript, style sheets, and a DOM-capable browser -- respectable and cool and (most importantly:) useful enough to be considered a viable alternative to The Three M's Of UI Lock-in (Macromedia, Microsoft, and Mozilla), all of which aim to fatten up the client and lock some users in and lock other would-be users out because they don't have Flash or XAML or XUL or any other doesn't-work-on-some-platforms complex soup like Java Server Faces and Struts taglibs. When it comes to simple, ubiquitous, usable user interfaces, Dynamic HTML is one ring to bing them all.
It took a long time -- upwards of a decade since Netscape released the first widely-used, widely-deployed browser -- for The Web Platform to become stable enough to build complete, beautiful applications upon. In 2004, Google's applications like Gmail and experiments like Google Suggest demonstrate that we can have robust, interactive, useful web-based applications. Joyce pointed this out over the weekend in her essay, Google is good for webdev, in which she wrote,
Google is quietly hiring -- they probably have between 5 and 10 of the top front-end devs, which doesn't sound like much until you realize there might be only a couple dozen out there with significant experience. That decreases the supply of the remaining ones, which as we all know means you're gonna have to show them some serious love to enjoy their scarce favors.
Most importantly, Google seems to be putting some chips down on the DHTML side of the table instead of the Flash or XAML or XUL or Laszlo sides. Given how important a few technology leaders are as role models for all developers -- I dunno how many arguments I've had where the magic words, "But Yahoo/Amazon/Google does it this way!" work their incantatory magic -- this is very much a Good Thing. So I've got to thank Google for making it viable to spend money building newer, faster, more responsive, standards-compliant, user-centric, cross-browser interfaces. Whatever the opposite of "collateral damage" is -- collateral benevolence? -- Google is doing it for webdev now.
What's so significant about the development of Gmail? Jon Udell looked under the hood, and was enthused by what he saw:
I’m ready to join the chorus singing the praises of GMail’s user-interface technology. Its combination of HTML, JavaScript, and the DOM makes the browser do some remarkable tricks.
One of my favorite acid tests is address completion. When you begin typing an e-mail address, your mail program should immediately show you the matching addresses and then dynamically constrain the list as you continue to type. Outlook does poorly on this test; you have to type CTRL-K to invoke the address book in a separate window. OS X’s Mail does address completion in situ, just as I expect. So does Gmail. And here’s the shocker: Gmail does it faster.
Gmail’s spell checker is another amazing hack. When you invoke it from the message composer, misspelled words turn red. Click one and a list of choices drops down, ending with an Edit choice. Click Edit and the suspect word converts, inline, to an input box. When you’re done correcting one or more words they merge back into the text.
As early adopters discovered long before I did, there’s an architecture behind this JavaScript/ DHTML wizardry. The best description I’ve found is from Johnvey Hwang, who deconstructed Gmail’s JavaScript code and created a .Net-based Gmail API. As Hwang described in his July 5 write-up, Gmail loads a JavaScript "UI engine" into your browser at the beginning of each session. Oddpost, he noted, was the first Web mail application to perfect this technique. That was a prophetic statement: Just four days later, on July 9, Yahoo acquired Oddpost.
Because Gmail’s behavior is embedded in the UI engine, all subsequent interaction between the browser and the Gmail service is just an exchange of data. What Hwang calls the DataPack format is not XML, though; it’s JavaScript. When you make a request to the Gmail service, whether to refresh your inbox or to modify the list of labels you can attach to messages, the response is a minimal set of JavaScript function calls and associated data objects that the engine uses to update the display.
This is very geeky stuff, I admit, but here are two important points to take away. First, as I’ve often said, intelligent use of browser-based technology can accomplish more than most people realize. You can’t do everything — not by a long shot — but for many of the things that information workers routinely do, even ordinary Web UI is good enough. And now Gmail is proving that we don’t have to settle for ordinary.
Second, Gmail’s architecture is not limited to Web UI. Because it is protocol-driven, developers can create new tools that speak to the DataPack format. Many have done so already.
In Gmail's case, the weblication was better than the non-web-based applications. Gmail raised the entire discussion of what is possible with web-based applications. It didn't go it alone -- it built on the knowledge of web-based applications learned by those who had previously written state-of-the-art weblications. What's significant is that it was able to break through expectations, and proved that "we don't have to settle for ordinary" with web-based applications. Koranteng Ofosu-Amaah clarifies this point in his essay On GMail and DHTML architecture again, in which he writes,
GMail's architecture is actually very generic for a DHTML app. Everyone with a clue should be trying to leverage the browser and that, in essence, is all they are doing.
It is riskier to do more on the rich browser client because it has been a more brittle platform over this past decade. Companies that do middleware and server-side tooling take a while to move from their core competency. Architects that thrive in that environment are essentially conservative and for good reason... Four years later, I now hear mutterings about drag-and-drop and richer clients in our corridors...
This only underscores the point that Jakob Nielsen's predictions about browser adoption cycles have turned out to be pretty accurate. Even though web application developers have been quietly spreading unobtrusive javascript usage in the interim, it is only now that there's a critical mass of clients that can leverage them; when Amazon and Yahoo move, something must be happening.
The developer tools and resources have gotten (slightly) better and there's more experience with the DOM. Increased adoption of broadband also helps reduce latency for the average client so you don't have to fight the inevitable arguments about performance and can couch your advocacy in terms of user interaction. In any case if and when you do have the performance discussion you can always argue that caching as close to the client as possible is a good thing and what better cache than the browser itself. It just so happens that applications like GMail, Bloglines and Oddpost are the state of the art in terms of browser leverage.
I recently wrote about this type of architecture in my recounting of the history of the DHTML spreadsheet and presentation components that are the genetic forebears of OddPost.
The idea is to fetch an HTML skeleton, decide what content you need, fetch that (as XML), and cache it wherever you get a chance. Render incrementally.
The pattern is simple:
Database XML (Optional) JavaScript Object Bindings UI Bindings (HTML) + UI management code
...Perhaps K-station was too bleeding edge trying to go for XML over HTTP, DHTML and extreme leverage of the browser client 5 years ago but that experience was a great testbed for me and I learned lots of lessons about building rich REST-ful applications, the importance of URIs etc.
Again the major missing feature for this rich web application platform is offline usage and synchronization without introducing new security holes in the browser. But then that's why Bosworth is at Google as the rumour goes, right? I suspect he's got other things in mind though...
I think that increased leverage of the browser and the DOM is a good thing. It's also a clear trend and for many applications, the browser is good enough. Good enough for Google, good enough for Yahoo, good enough for me.
And yes, the Beta release of Google Suggest does make me wonder why it wasn't until the end of 2004 that we saw such a web-based application speedily interacting with a web server to complete the form with what you're likely to be typing. We see this kind of interactivity regularly with our Oracle Financials or Peoplesoft HR or Siebel Customer Support system, where the forms are filled in interactively. Why did we wait till the end of 2004 to see such an interactive weblication?
Looking at the reverse engineering of Google Suggest and the Slashdot post where they describe the technique, we discover that
The source for the page is quite simple; most of the work happens in a condensed JavaScript library [google.com]. Not easy reading (note the word "condensed" above, meaning function and variable names are 1 or 2 chars, and all extra whitespace was removed...), but it's actually pretty straightforward.
It disables your browser's autocomplete on that textfield (for obvious reasons). Then it basically just defines a hidden div for that auto-complete dropdown (variations on this depending on browser... frickin' incompatibilities).
Each time you type a character, it populates that div body with the results of a quick, tiny query back to Google. It's NOT running the search for you; it's hitting (I assume) a simple, probably totally in-memory list of the most popular searches and number of results. That's how it can be so quick a response -- the lookup on their end is super-minimal, and the data to be transferred is probably less than 1k each time.
Cool. Nice concept, nice execution. And one of those nice "only obvious in hindsight" additions.
I have the feeling that we've turned a corner, and that more "only obvious in hindsight" web-based application tricks will be developed in the years to come -- thereby solidifying The Web As A Platform and continuing the spread of The Web Way as more users become True Believers who won't give up their web-based applications no matter how hard the "fat, rich client" camps try. As Joyce Park has said to me, "simplicity is its own revelation." It feels as if the world has had tremendous convergence on the thoughts in this post in 2004, and as a result the future looks very bright for The Web Way.http://www.wired.com/wired/archive/13.08/tech.html?pg=1&topic=tech&topic_set=
The Netscape IPO wasn't really about dot-commerce. At its heart was a new cultural force based on mass collaboration. Blogs, Wikipedia, open source, peer-to-peer - behold the power of the people.
Ten years ago, Netscape's explosive IPO ignited huge piles of money.
The brilliant flash revealed what had been invisible only a moment
before: the World Wide Web. As Eric Schmidt (then at Sun, now at
Google) noted, the day before the IPO, nothing about the Web; the day
after, everything.
Computing pioneer Vannevar Bush outlined the Web's core idea -
hyperlinked pages - in 1945, but the first person to try to build out
the concept was a freethinker named Ted Nelson who envisioned his own
scheme in 1965. However, he had little success connecting digital bits
on a useful scale, and his efforts were known only to an isolated group
of disciples. Few of the hackers writing code for the emerging Web in
the 1990s knew about Nelson or his hyperlinked dream machine.
At the suggestion of a computer-savvy friend, I got in touch with Nelson in 1984, a decade before Netscape. We met in a dark dockside bar in Sausalito, California. He was renting a houseboat nearby and had the air of someone with time on his hands. Folded notes erupted from his pockets, and long strips of paper slipped from overstuffed notebooks. Wearing a ballpoint pen on a string around his neck, he told me - way too earnestly for a bar at 4 o'clock in the afternoon - about his scheme for organizing all the knowledge of humanity. Salvation lay in cutting up 3 x 5 cards, of which he had plenty.
Although Nelson was polite, charming, and smooth, I was too slow for his fast talk. But I got an aha! from his marvelous notion of hypertext. He was certain that every document in the world should be a footnote to some other document, and computers could make the links between them visible and permanent. But that was just the beginning! Scribbling on index cards, he sketched out complicated notions of transferring authorship back to creators and tracking payments as readers hopped along networks of documents, what he called the docuverse. He spoke of "transclusion" and "intertwingularity" as he described the grand utopian benefits of his embedded structure. It was going to save the world from stupidity.
I believed him. Despite his quirks, it was clear to me that a hyperlinked world was inevitable - someday. But looking back now, after 10 years of living online, what surprises me about the genesis of the Web is how much was missing from Vannevar Bush's vision, Nelson's docuverse, and my own expectations. We all missed the big story. The revolution launched by Netscape's IPO was only marginally about hypertext and human knowledge. At its heart was a new kind of participation that has since developed into an emerging culture based on sharing. And the ways of participating unleashed by hyperlinks are creating a new type of thinking - part human and part machine - found nowhere else on the planet or in history.
Not only did we fail to imagine what the Web would become, we still don't see it today! We are blind to the miracle it has blossomed into. And as a result of ignoring what the Web really is, we are likely to miss what it will grow into over the next 10 years. Any hope of discerning the state of the Web in 2015 requires that we own up to how wrong we were 10 years ago.
1995
Before the Netscape browser illuminated the Web, the Internet did not
exist for most people. If it was acknowledged at all, it was
mischaracterized as either corporate email (as exciting as a necktie)
or a clubhouse for adolescent males (read: pimply nerds). It was hard
to use. On the Internet, even dogs had to type. Who wanted to waste
time on something so boring?
The memories of an early enthusiast like myself can be unreliable, so I recently spent a few weeks reading stacks of old magazines and newspapers. Any promising new invention will have its naysayers, and the bigger the promises, the louder the nays. It's not hard to find smart people saying stupid things about the Internet on the morning of its birth. In late 1994, Time magazine explained why the Internet would never go mainstream: "It was not designed for doing commerce, and it does not gracefully accommodate new arrivals." Newsweek put the doubts more bluntly in a February 1995 headline: "THE INTERNET? BAH!" The article was written by astrophysicist and Net maven Cliff Stoll, who captured the prevailing skepticism of virtual communities and online shopping with one word: "baloney."
This dismissive attitude pervaded a meeting I had with the top leaders of ABC in 1989. I was there to make a presentation to the corner office crowd about this "Internet stuff." To their credit, they realized something was happening. Still, nothing I could tell them would convince them that the Internet was not marginal, not just typing, and, most emphatically, not just teenage boys. Stephen Weiswasser, a senior VP, delivered the ultimate putdown: "The Internet will be the CB radio of the '90s," he told me, a charge he later repeated to the press. Weiswasser summed up ABC's argument for ignoring the new medium: "You aren't going to turn passive consumers into active trollers on the Internet."
I was shown the door. But I offered one tip before I left. "Look," I said. "I happen to know that the address www.abc.com has not been registered. Go down to your basement, find your most technical computer guy, and have him register www.abc.com immediately. Don't even think about it. It will be a good thing to do." They thanked me vacantly. I checked a week later. The domain was still unregistered.
While it is easy to smile at the dodos in TV land, they were not the only ones who had trouble imagining an alternative to couch potatoes. Wired did, too. When I examine issues of Wired from before the Netscape IPO (issues that I proudly edited), I am surprised to see them touting a future of high production-value content - 5,000 always-on channels and virtual reality, with a side order of email sprinkled with bits of the Library of Congress. In fact, Wired offered a vision nearly identical to that of Internet wannabes in the broadcast, publishing, software, and movie industries: basically, TV that worked. The question was who would program the box. Wired looked forward to a constellation of new media upstarts like Nintendo and Yahoo!, not old-media dinosaurs like ABC.
Problem was, content was expensive to produce, and 5,000 channels of it would be 5,000 times as costly. No company was rich enough, no industry large enough, to carry off such an enterprise. The great telecom companies, which were supposed to wire up the digital revolution, were paralyzed by the uncertainties of funding the Net. In June 1994, David Quinn of British Telecom admitted to a conference of software publishers, "I'm not sure how you'd make money out of it."
The immense sums of money supposedly required to fill the Net with content sent many technocritics into a tizzy. They were deeply concerned that cyberspace would become cyburbia - privately owned and operated. Writing in Electronic Engineering Times in 1995, Jeff Johnson worried: "Ideally, individuals and small businesses would use the information highway to communicate, but it is more likely that the information highway will be controlled by Fortune 500 companies in 10 years." The impact would be more than commercial. "Speech in cyberspace will not be free if we allow big business to control every square inch of the Net," wrote Andrew Shapiro in The Nation in July 1995.
The fear of commercialization was strongest among hardcore programmers: the coders, Unix weenies, TCP/IP fans, and selfless volunteer IT folk who kept the ad hoc network running. The major administrators thought of their work as noble, a gift to humanity. They saw the Internet as an open commons, not to be undone by greed or commercialization. It's hard to believe now, but until 1991, commercial enterprise on the Internet was strictly prohibited. Even then, the rules favored public institutions and forbade "extensive use for private or personal business."
In the mid-1980s, when I was involved in the WELL, an early nonprofit online system, we struggled to connect it to the emerging Internet but were thwarted, in part, by the "acceptable use" policy of the National Science Foundation (which ran the Internet backbone). In the eyes of the NSF, the Internet was funded for research, not commerce. At first this restriction wasn't a problem for online services, because most providers, the WELL included, were isolated from one another. Paying customers could send email within the system - but not outside it. In 1987, the WELL fudged a way to forward outside email through the Net without confronting the acceptable use policy, which our organization's own techies were reluctant to break. The NSF rule reflected a lingering sentiment that the Internet would be devalued, if not trashed, by opening it up to commercial interests. Spam was already a problem (one every week!).
This attitude prevailed even in the offices of Wired. In 1994, during the first design meetings for Wired's embryonic Web site, HotWired, programmers were upset that the innovation we were cooking up - what are now called clickthrough ad banners - subverted the great social potential of this new territory. The Web was hardly out of diapers, and already they were being asked to blight it with billboards and commercials. Only in May 1995, after the NSF finally opened the floodgates to ecommerce, did the geek elite begin to relax.
Three months later, Netscape's public offering took off, and in a blink a world of DIY possibilities was born. Suddenly it became clear that ordinary people could create material anyone with a connection could view. The burgeoning online audience no longer needed ABC for content. Netscape's stock peaked at $75 on its first day of trading, and the world gasped in awe. Was this insanity, or the start of something new?
2005
The scope of the Web today is hard to fathom. The total number of Web
pages, including those that are dynamically created upon request and
document files available through links, exceeds 600 billion. That's 100
pages per person alive.
How could we create so much, so fast, so well? In fewer than 4,000 days, we have encoded half a trillion versions of our collective story and put them in front of 1 billion people, or one-sixth of the world's population. That remarkable achievement was not in anyone's 10-year plan.
The accretion of tiny marvels can numb us to the arrival of the stupendous. Today, at any Net terminal, you can get: an amazing variety of music and video, an evolving encyclopedia, weather forecasts, help wanted ads, satellite images of anyplace on Earth, up-to-the-minute news from around the globe, tax forms, TV guides, road maps with driving directions, real-time stock quotes, telephone numbers, real estate listings with virtual walk-throughs, pictures of just about anything, sports scores, places to buy almost anything, records of political contributions, library catalogs, appliance manuals, live traffic reports, archives to major newspapers - all wrapped up in an interactive index that really works.
This view is spookily godlike. You can switch your gaze of a spot in the world from map to satellite to 3-D just by clicking. Recall the past? It's there. Or listen to the daily complaints and travails of almost anyone who blogs (and doesn't everyone?). I doubt angels have a better view of humanity.
Why aren't we more amazed by this fullness? Kings of old would have gone to war to win such abilities. Only small children would have dreamed such a magic window could be real. I have reviewed the expectations of waking adults and wise experts, and I can affirm that this comprehensive wealth of material, available on demand and free of charge, was not in anyone's scenario. Ten years ago, anyone silly enough to trumpet the above list as a vision of the near future would have been confronted by the evidence: There wasn't enough money in all the investment firms in the entire world to fund such a cornucopia. The success of the Web at this scale was impossible.
But if we have learned anything in the past decade, it is the plausibility of the impossible.
Take eBay. In some 4,000 days, eBay has gone from marginal Bay Area experiment in community markets to the most profitable spinoff of hypertext. At any one moment, 50 million auctions race through the site. An estimated half a million folks make their living selling through Internet auctions. Ten years ago I heard skeptics swear nobody would ever buy a car on the Web. Last year eBay Motors sold $11 billion worth of vehicles. EBay's 2001 auction of a $4.9 million private jet would have shocked anyone in 1995 - and still smells implausible today.
Nowhere in Ted Nelson's convoluted sketches of hypertext transclusion did the fantasy of a global flea market appear. Especially as the ultimate business model! He hoped to franchise his Xanadu hypertext systems in the physical world at the scale of a copy shop or café - you would go to a store to do your hypertexting. Xanadu would take a cut of the action.
Instead, we have an open global flea market that handles 1.4 billion auctions every year and operates from your bedroom. Users do most of the work; they photograph, catalog, post, and manage their own auctions. And they police themselves; while eBay and other auction sites do call in the authorities to arrest serial abusers, the chief method of ensuring fairness is a system of user-generated ratings. Three billion feedback comments can work wonders.
What we all failed to see was how much of this new world would be manufactured by users, not corporate interests. Amazon.com customers rushed with surprising speed and intelligence to write the reviews that made the site's long-tail selection usable. Owners of Adobe, Apple, and most major software products offer help and advice on the developer's forum Web pages, serving as high-quality customer support for new buyers. And in the greatest leverage of the common user, Google turns traffic and link patterns generated by 2 billion searches a month into the organizing intelligence for a new economy. This bottom-up takeover was not in anyone's 10-year vision.
No Web phenomenon is more confounding than blogging. Everything media experts knew about audiences - and they knew a lot - confirmed the focus group belief that audiences would never get off their butts and start making their own entertainment. Everyone knew writing and reading were dead; music was too much trouble to make when you could sit back and listen; video production was simply out of reach of amateurs. Blogs and other participant media would never happen, or if they happened they would not draw an audience, or if they drew an audience they would not matter. What a shock, then, to witness the near-instantaneous rise of 50 million blogs, with a new one appearing every two seconds. There - another new blog! One more person doing what AOL and ABC - and almost everyone else - expected only AOL and ABC to be doing. These user-created channels make no sense economically. Where are the time, energy, and resources coming from?
The audience.
I run a blog about cool tools. I write it for my own delight and for the benefit of friends. The Web extends my passion to a far wider group for no extra cost or effort. In this way, my site is part of a vast and growing gift economy, a visible underground of valuable creations - text, music, film, software, tools, and services - all given away for free. This gift economy fuels an abundance of choices. It spurs the grateful to reciprocate. It permits easy modification and reuse, and thus promotes consumers into producers.
The open source software movement is another example. Key ingredients of collaborative programming - swapping code, updating instantly, recruiting globally - didn't work on a large scale until the Web was woven. Then software became something you could join, either as a beta tester or as a coder on an open source project. The clever "view source" browser option let the average Web surfer in on the act. And anyone could rustle up a link - which, it turns out, is the most powerful invention of the decade.
Linking unleashes involvement and interactivity at levels once thought unfashionable or impossible. It transforms reading into navigating and enlarges small actions into powerful forces. For instance, hyperlinks made it much easier to create a seamless, scrolling street map of every town. They made it easier for people to refer to those maps. And hyperlinks made it possible for almost anyone to annotate, amend, and improve any map embedded in the Web. Cartography has gone from spectator art to participatory democracy.
The electricity of participation nudges ordinary folks to invest huge hunks of energy and time into making free encyclopedias, creating public tutorials for changing a flat tire, or cataloging the votes in the Senate. More and more of the Web runs in this mode. One study found that only 40 percent of the Web is commercial. The rest runs on duty or passion.
Coming out of the industrial age, when mass-produced goods outclassed anything you could make yourself, this sudden tilt toward consumer involvement is a complete Lazarus move: "We thought that died long ago." The deep enthusiasm for making things, for interacting more deeply than just choosing options, is the great force not reckoned 10 years ago. This impulse for participation has upended the economy and is steadily turning the sphere of social networking - smart mobs, hive minds, and collaborative action - into the main event.
When a company opens its databases to users, as Amazon, Google, and eBay have done with their Web services, it is encouraging participation at new levels. The corporation's data becomes part of the commons and an invitation to participate. People who take advantage of these capabilities are no longer customers; they're the company's developers, vendors, skunk works, and fan base.
A little over a decade ago, a phone survey by Macworld asked a few hundred people what they thought would be worth $10 per month on the information superhighway. The participants started with uplifting services: educational courses, reference books, electronic voting, and library information. The bottom of the list ended with sports statistics, role-playing games, gambling, and dating. Ten years later what folks actually use the Internet for is inverted. According to a 2004 Stanford study, people use the Internet for (in order): playing games, "just surfing," shopping the list ends with responsible activities like politics and banking. (Some even admitted to porn.) Remember, shopping wasn't supposed to happen. Where's Cliff Stoll, the guy who said the Internet was baloney and online catalogs humbug? He has a little online store where he sells handcrafted Klein bottles.
The public's fantasy, revealed in that 1994 survey, began reasonably with the conventional notions of a downloadable world. These assumptions were wired into the infrastructure. The bandwidth on cable and phone lines was asymmetrical: Download rates far exceeded upload rates. The dogma of the age held that ordinary people had no need to upload; they were consumers, not producers. Fast-forward to today, and the poster child of the new Internet regime is BitTorrent. The brilliance of BitTorrent is in its exploitation of near-symmetrical communication rates. Users upload stuff while they are downloading. It assumes participation, not mere consumption. Our communication infrastructure has taken only the first steps in this great shift from audience to participants, but that is where it will go in the next decade.
With the steady advance of new ways to share, the Web has embedded itself into every class, occupation, and region. Indeed, people's anxiety about the Internet being out of the mainstream seems quaint now. In part because of the ease of creation and dissemination, online culture is the culture. Likewise, the worry about the Internet being 100 percent male was entirely misplaced. Everyone missed the party celebrating the 2002 flip-point when women online first outnumbered men. Today, 52 percent of netizens are female. And, of course, the Internet is not and has never been a teenage realm. In 2005, the average user is a bone-creaking 41 years old.
What could be a better mark of irreversible acceptance than adoption by the Amish? I was visiting some Amish farmers recently. They fit the archetype perfectly: straw hats, scraggly beards, wives with bonnets, no electricity, no phones or TVs, horse and buggy outside. They have an undeserved reputation for resisting all technology, when actually they are just very late adopters. Still, I was amazed to hear them mention their Web sites.
"Amish Web sites?" I asked.
"For advertising our family business. We weld barbecue grills in our shop."
"Yes, but "
"Oh, we use the Internet terminal at the public library. And Yahoo!"
I knew then the battle was over.
2015
The Web continues to evolve from a world ruled by mass media and mass
audiences to one ruled by messy media and messy participation. How far
can this frenzy of creativity go? Encouraged by Web-enabled sales,
175,000 books were published and more than 30,000 music albums were
released in the US last year. At the same time, 14 million blogs
launched worldwide. All these numbers are escalating. A simple
extrapolation suggests that in the near future, everyone alive will (on
average) write a song, author a book, make a video, craft a weblog, and
code a program. This idea is less outrageous than the notion 150 years
ago that someday everyone would write a letter or take a photograph.
What happens when the data flow is asymmetrical - but in favor of creators? What happens when everyone is uploading far more than they download? If everyone is busy making, altering, mixing, and mashing, who will have time to sit back and veg out? Who will be a consumer?
No one. And that's just fine. A world where production outpaces consumption should not be sustainable; that's a lesson from Economics 101. But online, where many ideas that don't work in theory succeed in practice, the audience increasingly doesn't matter. What matters is the network of social creation, the community of collaborative interaction that futurist Alvin Toffler called prosumption. As with blogging and BitTorrent, prosumers produce and consume at once. The producers are the audience, the act of making is the act of watching, and every link is both a point of departure and a destination.
But if a roiling mess of participation is all we think the Web will become, we are likely to miss the big news, again. The experts are certainly missing it. The Pew Internet & American Life Project surveyed more than 1,200 professionals in 2004, asking them to predict the Net's next decade. One scenario earned agreement from two-thirds of the respondents: "As computing devices become embedded in everything from clothes to appliances to cars to phones, these networked devices will allow greater surveillance by governments and businesses." Another was affirmed by one-third: "By 2014, use of the Internet will increase the size of people's social networks far beyond what has traditionally been the case."
These are safe bets, but they fail to capture the Web's disruptive trajectory. The real transformation under way is more akin to what Sun's John Gage had in mind in 1988 when he famously said, "The network is the computer." He was talking about the company's vision of the thin-client desktop, but his phrase neatly sums up the destiny of the Web: As the OS for a megacomputer that encompasses the Internet, all its services, all peripheral chips and affiliated devices from scanners to satellites, and the billions of human minds entangled in this global network. This gargantuan Machine already exists in a primitive form. In the coming decade, it will evolve into an integral extension not only of our senses and bodies but our minds.
Today, the Machine acts like a very large computer with top-level functions that operate at approximately the clock speed of an early PC. It processes 1 million emails each second, which essentially means network email runs at 1 megahertz. Same with Web searches. Instant messaging runs at 100 kilohertz, SMS at 1 kilohertz. The Machine's total external RAM is about 200 terabytes. In any one second, 10 terabits can be coursing through its backbone, and each year it generates nearly 20 exabytes of data. Its distributed "chip" spans 1 billion active PCs, which is approximately the number of transistors in one PC.
This planet-sized computer is comparable in complexity to a human brain. Both the brain and the Web have hundreds of billions of neurons (or Web pages). Each biological neuron sprouts synaptic links to thousands of other neurons, while each Web page branches into dozens of hyperlinks. That adds up to a trillion "synapses" between the static pages on the Web. The human brain has about 100 times that number - but brains are not doubling in size every few years. The Machine is.
Since each of its "transistors" is itself a personal computer with a billion transistors running lower functions, the Machine is fractal. In total, it harnesses a quintillion transistors, expanding its complexity beyond that of a biological brain. It has already surpassed the 20-petahertz threshold for potential intelligence as calculated by Ray Kurzweil. For this reason some researchers pursuing artificial intelligence have switched their bets to the Net as the computer most likely to think first. Danny Hillis, a computer scientist who once claimed he wanted to make an AI "that would be proud of me," has invented massively parallel supercomputers in part to advance us in that direction. He now believes the first real AI will emerge not in a stand-alone supercomputer like IBM's proposed 23-teraflop Blue Brain, but in the vast digital tangle of the global Machine.
In 10 years, the system will contain hundreds of millions of miles of fiber-optic neurons linking the billions of ant-smart chips embedded into manufactured products, buried in environmental sensors, staring out from satellite cameras, guiding cars, and saturating our world with enough complexity to begin to learn. We will live inside this thing.
Today the nascent Machine routes packets around disturbances in its lines; by 2015 it will anticipate disturbances and avoid them. It will have a robust immune system, weeding spam from its trunk lines, eliminating viruses and denial-of-service attacks the moment they are launched, and dissuading malefactors from injuring it again. The patterns of the Machine's internal workings will be so complex they won't be repeatable; you won't always get the same answer to a given question. It will take intuition to maximize what the global network has to offer. The most obvious development birthed by this platform will be the absorption of routine. The Machine will take on anything we do more than twice. It will be the Anticipation Machine.
One great advantage the Machine holds in this regard: It's always on. It is very hard to learn if you keep getting turned off, which is the fate of most computers. AI researchers rejoice when an adaptive learning program runs for days without crashing. The fetal Machine has been running continuously for at least 10 years (30 if you want to be picky). I am aware of no other machine - of any type - that has run that long with zero downtime. While portions may spin down due to power outages or cascading infections, the entire thing is unlikely to go quiet in the coming decade. It will be the most reliable gadget we have.
And the most universal. By 2015, desktop operating systems will be largely irrelevant. The Web will be the only OS worth coding for. It won't matter what device you use, as long as it runs on the Web OS. You will reach the same distributed computer whether you log on via phone, PDA, laptop, or HDTV.
In the 1990s, the big players called that convergence. They peddled the image of multiple kinds of signals entering our lives through one box - a box they hoped to control. By 2015 this image will be turned inside out. In reality, each device is a differently shaped window that peers into the global computer. Nothing converges. The Machine is an unbounded thing that will take a billion windows to glimpse even part of. It is what you'll see on the other side of any screen.
And who will write the software that makes this contraption useful and productive? We will. In fact, we're already doing it, each of us, every day. When we post and then tag pictures on the community photo album Flickr, we are teaching the Machine to give names to images. The thickening links between caption and picture form a neural net that can learn. Think of the 100 billion times per day humans click on a Web page as a way of teaching the Machine what we think is important. Each time we forge a link between words, we teach it an idea. Wikipedia encourages its citizen authors to link each fact in an article to a reference citation. Over time, a Wikipedia article becomes totally underlined in blue as ideas are cross-referenced. That massive cross-referencing is how brains think and remember. It is how neural nets answer questions. It is how our global skin of neurons will adapt autonomously and acquire a higher level of knowledge.
The human brain has no department full of programming cells that configure the mind. Rather, brain cells program themselves simply by being used. Likewise, our questions program the Machine to answer questions. We think we are merely wasting time when we surf mindlessly or blog an item, but each time we click a link we strengthen a node somewhere in the Web OS, thereby programming the Machine by using it.
What will most surprise us is how dependent we will be on what the Machine knows - about us and about what we want to know. We already find it easier to Google something a second or third time rather than remember it ourselves. The more we teach this megacomputer, the more it will assume responsibility for our knowing. It will become our memory. Then it will become our identity. In 2015 many people, when divorced from the Machine, won't feel like themselves - as if they'd had a lobotomy.
Legend has it that Ted Nelson invented Xanadu as a remedy for his poor memory and attention deficit disorder. In this light, the Web as memory bank should be no surprise. Still, the birth of a machine that subsumes all other machines so that in effect there is only one Machine, which penetrates our lives to such a degree that it becomes essential to our identity - this will be full of surprises. Especially since it is only the beginning.
There is only one time in the history of each planet when its inhabitants first wire up its innumerable parts to make one large Machine. Later that Machine may run faster, but there is only one time when it is born.
You and I are alive at this moment.
We should marvel, but people alive at such times usually don't. Every few centuries, the steady march of change meets a discontinuity, and history hinges on that moment. We look back on those pivotal eras and wonder what it would have been like to be alive then. Confucius, Zoroaster, Buddha, and the latter Jewish patriarchs lived in the same historical era, an inflection point known as the axial age of religion. Few world religions were born after this time. Similarly, the great personalities converging upon the American Revolution and the geniuses who commingled during the invention of modern science in the 17th century mark additional axial phases in the short history of our civilization.
Three thousand years from now, when keen minds review the past, I believe that our ancient time, here at the cusp of the third millennium, will be seen as another such era. In the years roughly coincidental with the Netscape IPO, humans began animating inert objects with tiny slivers of intelligence, connecting them into a global field, and linking their own minds into a single thing. This will be recognized as the largest, most complex, and most surprising event on the planet. Weaving nerves out of glass and radio waves, our species began wiring up all regions, all processes, all facts and notions into a grand network. From this embryonic neural net was born a collaborative interface for our civilization, a sensing, cognitive device with power that exceeded any previous invention. The Machine provided a new way of thinking (perfect search, total recall) and a new mind for an old species. It was the Beginning.
In retrospect, the Netscape IPO was a puny rocket to herald such a moment. The product and the company quickly withered into irrelevance, and the excessive exuberance of its IPO was downright tame compared with the dotcoms that followed. First moments are often like that. After the hysteria has died down, after the millions of dollars have been gained and lost, after the strands of mind, once achingly isolated, have started to come together - the only thing we can say is: Our Machine is born. It's on.
Senior maverick Kevin Kelly (kk@kk.org) wrote about the universe as a computer in issue 10.12.
Original Article:Rondam Ramblings
Since I've started my new career as a venture capitalist I have become keenly aware of some of the classic mistakes that geeks make when trying to raise money for a new business. Instead of writing the same comments over and over again I thought I'd try to summarize some of the mistakes that people -- especially smart people -- make when they decide to try to turn their bright ideas into money. Here then is my top-ten list of geek business myths:
Myth #1: A brilliant idea will make you rich.Reality:
A brilliant idea is neither necessary nor sufficient for a successful business, although all else being equal it can't hurt. Microsoft is probably the canonical example of a successful business, and it has never had a single brilliant idea in its entire history. (To the contrary, Microsoft has achieved success largely by seeking out and destroying other people's brilliant ideas.) Google was based on a couple of brilliant ideas (Page rank, text-only ads, massive parallel implementation on cheap hardware) but none of those ideas were original with Larry or Sergey. This is not to say that Larry, Sergey and Bill are not bright guys -- all three of them are sharper than I can ever hope to be. But the idea that any of them woke up one day with an inspiration and coasted the rest of the way to riches is a myth.
Myth #2: If you build it they will come.
There is a grain of truth to this myth. There have been examples of businesses that just built a product, cast it upon the ether(net), and achieved success. (Google is the canonical example.) But for every Google there are ten examples of companies that had killer products that didn't sell for one reason or another. My favorite example of this is the first company I tried to start back in 1993. It was called FlowNet, and it was a new design for a high speed local area network. It ran at 500Mb/s in a time when 10 Mb/s ethernet was the norm. For more than five years, FlowNet had the best price/performance ratio of any available network. On top of that, FlowNet had built-in quality-of-service guarantees for streaming video. If FlowNet had taken over the world your streaming video would be working a lot better today than it does.But despite the fact that on a technical level FlowNet blew everything else out of the water it was an abysmal failure as a business. We never sold a single unit. The full story of why FlowNet failed would take me far afield, but if I had to sum it up in a nutshell the reason it didn't sell was very simple: it wasn't Ethernet. And if we'd done our homework and market research we could have known that this would be, if not a show-stopper at least a significant obstacle. And we would have known it before we spent tens of thousands of dollars of our own money on patent attorneys and prototypes.
Myth #3: Someone will steal your idea if you don't protect it.
Reality: No one gives a damn about your idea until you actually succeed and by then it's too late. Even on the off chance that you do manage to stumble across someone who is as excited about your idea as you are, if they have any brains they will join you rather than try to beat you. (And if they don't have any brains then it doesn't matter what they do.)Patent protection does serve one useful purpose: it can make investors feel warm and fuzzy, especially naive investors. But I strongly recommend that you do your own patent filings. It's not hard to do once you learn how (get the Nolo Press book "Patent it Yourself"). You'll do a better job than most patent attorneys and save yourself a lot of money.
Myth #4: What you think matters.
Reality: It matters not one whit that you and all your buddies think that your idea is the greatest thing since sliced pizza (unless, of course, your buddies are rich enough to be the customer base for your business). What matters is what your customers think. It is natural to assume that if you and your buddies think your idea is cool that millions of other people out there will think it's cool too, and sometimes it works out that way, but usually not. The reason is that if you are smart enough to have a brilliant idea then you (and most likely your buddies) are different from everyone else. I don't mean to sound condescending here, but the sad fact of the matter is that compared to you, most people are pretty dumb (look at how many people vote Republican ;-) and they care about dumb things. (I just heard about a new clothing store in Pasadena that has lines around the block. A clothing store!) If you cater only to people who care about the things that you care about then your customer base will be pretty small.
Myth #5: Financial models are bogus.
As with myth #2 there is a grain of truth here. As Carl Sagan was fond of saying, prophecy is a lost art. There is no way to know for sure how much money your business is going to make, or how much it will cost to get to market. The reason for doing financial models is to do a reality check and convince yourself that making a return on investment is even a plausible possibility. If you run the numbers and find out that in order to reach break-even you need a customer base that is ten times larger than the currently known market for your product then you should probably rethink things. As Dwight Eisenhower said: plans are useless, but planning is indispensible.This myth is the basis for one of the most classic mistakes that geeks make when pitching their ideas. They will say things like "Even if we only capture 1% of the market we'll make big bucks." Statements like that are a dead giveaway that you haven't done your homework to find out what your customers actually want. You may as well say: there's a good chance that only 1 customer in 100 will buy our product (and frankly, we're not even sure about that). Doesn't exactly inspire confidence.
Myth #6: What you know matters more than who you know.
Reality: You've been in denial about this your whole life. You were either brought up to believe that being smart mattered, or you just didn't believe your mother when she told you that getting along with the other kids was more important than getting straight A's.The truth is, who you know matters more than what you know. This is not to say that being smart and knowledgable is useless. Knowing "what" is often an effective means of getting introduced to the right "whos". But ultimately, the people you know and trust (and more importantly who trust you) matter more than the factual knowledge you may have at your immediate disposal. And there is a sound reason for this: business decisions are horrifically complicated. No one person can possibly amass all the knowledge and experience required to make a broad range of such decisions on their own, so effective business people delegate much of their decision-making to other people. And when they choose who to delegate to, their first pick is always people they know and trust.Ironically, C programmers understand this much better than Lisp programmers. One of the ironies of the programming world is that using Lisp is vastly more productive than using pretty much any other programming language, but successful businesses based on Lisp are quite rare. The reason for this, I think, is that Lisp allows you to be so productive that a single person can get things done without having to work together with anyone else, and so Lisp programmers never develop the social skills needed to work effectively as a member of a team. A C programmer, by contrast, can't do anything useful except as a member of a team. So although programming in C hobbles you in some ways, it forces you to form groups whose net effectiveness is greater than the sum of their parts, and who collectively can stomp on all the individual Lisp programmers out there, even though one-on-one a Lisper can run rings around a C programmer.
Myth #7: A Ph.D. means something.
Reality: The only thing a Ph.D. means is that you're not a moron, and you're willing to put up with the bullshit it takes to slog your way through a Ph.D. program somewhere. Empirically, having a Ph.D. is negatively correlated with business success. This is because the reward structure in academia is almost the exact opposite of what it is in business. In academia, what your peers think matters. In business, it's what your customers think that matters, and your customers are (almost certainly) not your peers.[UPDATE: this is not to say that getting a Ph.D. is useless. You can learn a lot of useful stuff by getting a Ph.D. But it's the knowledge and experience that you gain by going through the process that is potentially valuable (for business endeavors), not the degree itself.]
Myth #8: I need $5 million to start my business
Reality: Unless you're building hardware (in which case you should definitely rethink what you're doing) you most likely don't need any startup capital at all. Paul Graham has written extensively about this so I won't belabor it too much, except to say this: you don't need much startup capital, but what you do need is a willingness to work your buns off. You have to bring your brilliant idea to fruition yourself; no one else will do it for you, and no one will give you the money to hire someone to do it for you. The reason is very simple: if you don't believe in the commercial potential of your idea enough to give up your evenings and weekends to own a bigger chunk of it, why should anyone else believe in it enough to put their hard-earned money at risk?
Myth #9: The idea is the most important part of my business plan.
Reality: The idea is very nearly irrelevant. What matters is 1) who are your customers? 2) Why will they buy what you're selling? (Note that the reason for this could very well be something like, "Because I'm famous and I have a huge fan base and they will buy sacks of stale dog shit if it has my name on it." But in your case it will more likely be, "Because we have a great product that blows the competition out of the water.") 3) Who is on your team? and 4) What are the risks?
Myth #10: Having no competition is a good thing.
Reality: If you have no competition the most likely reason for that is that there's no money to be made. There are six billion people on this planet, and it's very unlikely that every last of them will have left a lucrative market niche completely unexploited.The good news is that it is very likely that your competition sucks. The vast majority of businesses are not run very well. They make shoddy products. They treat their customers and their employees like shit. It's not hard to find market opportunities where you can go in and kick the competition's ass. You don't want no competition, what you want is bad competition. And there's plenty of that out there.
Special bonus myth (free with your paid subscription): After the IPO I'll be happy.
If you don't enjoy the process of starting a business then you will probably not succeed. It's just too much work, and it will suck you dry if you're not having fun doing it. Even if you get filthy stinking rich you will just have more time to look back across the years you wasted being miserable and nursing your acid reflux. The charm of expensive cars and whatnot wears off quickly. There's only one kind of happiness that money can buy, and that is the opportunity to be on the other side of the table when some bright kid comes along with a brilliant idea for a business.All these myths can be neatly summarized in a pithy slogan: it's the customer, stupid. Success in business is not about having a brilliant idea. Bright ideas are a dime a dozen. Business is about taking a bright idea and assembling a team that can turn that idea into a product and bring that product to customers who want to buy it. It's that simple. And that complicated.Good luck.