Two Predictions for Open Source Software in 2009

Prediction #1: The adoption of open source will accelerate in 2009.

Open source is on a tear. It's being adopted more quickly and more widely than ever before. Evidence: In November, Gartner announced survey results that found that 85% of enterprises are already using open source software, and the remaining 15% plan to start using it soon. (And my guess is, a good portion of that 15% are already using it, but management doesn't know.)

On the Optaros blog, Bruno von Rotz offers a good summary of what's happened in 2008 in the world of open source, and a lot of it is encouraging news for open source vendors (sales growing faster than expected, more rounds of funding, etc.).

With companies in all industries hitting the pause button on spending, open source should prove especially attractive in 2009. If you're a project manager, and you're being asked to do more with less or even with practically nothing, you're likely to take a good, hard look at open source, at least for a pilot project, even if you've had qualms about the maturity or stability of open source products in your area of expertise.

Earlier this month, Gartner analysts advised their clients to prepare two IT budgets: one with a 2% increase in spending, the other with a 20% decrease in spending. If your IT budget really does get slashed by 20% or more, you'll have little choice but to consider software products that offer a basic version for free, and whose paid products have prices that will turn out to be highly negotiable.

Bottom line, then: 2009 will create many new opportunities for open source vendors to get their feet in the door.

Prediction #2: Many open source vendors will continue to struggle, and some well-known vendors will shut down.

Unfortunately, getting your foot in the door is no guarantee of success. Despite the impressive adoption of open source and the continued rounds of funding, many open source vendors are struggling to make money—in some cases, even after receiving many tens of millions of dollars of VC investments. The sad truth is that even some of the well known names in open source—companies with good products and impressively high numbers of downloads—are having trouble converting those downloads and installations into a viable business.

Part of the problem is that the free version of products are often good enough to satisfy customer requirements. If customers don't have to spend money to get features, and if the product is reliable enough (or not mission-critical), customers won't spend money on licensing and support.

Part of the problem is that while the download numbers are growing, the company has yet to find a repeatable sales model for converting downloads to sales; end user requirements are simply too varied to build a profitable business.

Part of the problem also is that free has to compete with cheap (or at least comparatively inexpensive). On the Open NMS blog, Tarus Balog recently pointed out that open source network management products have to compete with low-cost, easy-to-use products like Solar Winds. He's absolutely right. And Solar Winds has been on its own tear for many years, now. Another up-and-coming NMS platform is AdventNet's ManageEngine, which knits together network management and service desk functions into an easy-to-use, highly affordable whole. AdventNet now has tens of thousands of paying customers for its network management products—an enviable achievement from the point of view of many open source vendors. If price is what's driving you to open source, and you're not particularly interested in having access to a product's source code, you may end up choosing one of these highly affordable non-open source alternatives instead.

Another part of the problem with open source companies comes down to simple execution: building what customers really want rather that what the core development team feels comfortable with (especially if the company is VC-backed and has real targets to hit); solving customer problems promptly and effectively through support, documentation, and training; marketing well; etc. Most open source companies are developer-led organizations, and some (decidedly not all, but some) developer-led organizations fall into the trap of expecting their potential community to share the internal team's own predilections and priorities. (If you hear an engineering manager saying something like, "That should be reasonable, it shouldn't be that hard for the customer to figure out," stop the design discussion right there: customers should have to figure out almost nothing.)

A few vendors may find financial salvation by converting or substantially augmenting their open source business with a SaaS business and close deal that are SaaS subscriptions rather than on-premise software licenses. But for other vendors time will eventually run out. Investors will shift new funding other more viable ventures. Staff cuts will paralyze progress. Projects will be left to linger on SourceForge.

This is why I expect 2009 to be a mixed year for open source vendors. It will be a year of unprecedented opportunity for most, and a year of hard reckoning for some.

Zoho CloudSQL: An interview with Rodrigo Vaca

Earlier this week, Zoho announced CloudSQL, a new SQL interface to Zoho Reports, its popular Web application for online reporting and business intelligence. Zoho applications (in case you haven't heard of them) are credible alternatives to Google Software-as-a-Service (SaaS) applications such as Google Docs. Launched three years ago, the suite of Zoho applications has grown dramatically in number of applications, richness of features, and size of its user base. The company now boasts over 1 million users for its 19 applications. More applications are on the way.


Here's how Rodrigo Vaca, Zoho's Director of Marketing, described CloudSQL in a blog post earlier this week:

Zoho CloudSQL is a middleware technology that allows customers to interact with their business data stored in Zoho through the familiar SQL language. Customers are able to access Zoho cloud data using SQL on both other cloud applications as well as through traditional on-premises software.

At a high-level, Zoho CloudSQL serves as the bridge between the external application and the data stored inside Zoho. It receives the query in SQL, interprets it, delegates queries and aggregates results across the Zoho services.

There are in particular 3 things that stand out about Zoho CloudSQL:

• It's the first technology that allows customers to interact with their data on the cloud, from another cloud application or from an on-premises one through real SQL.


• It supports multiple SQL dialects. We support all the major (and even some not so major) ones: ANSI, Oracle, SQL Server, IBM DB2, MySQL, PostgreSQL and Informix.


• With our JDBC/ODBC drivers, developers can access data in the cloud just as easily as if it were stored in a local database.
  

A Quick Interview
I got in touch with Rodrigo Vaca to ask him a few follow-up questions.

JB: From your announcement, I'm gathering that CloudSQL is a SQL-based service for accessing data in Zoho applications. The interface will be of interest to engineers working on integration projects where they would like to simply work with SQL queries, rather than dealing with JSON or RESTful data access. Is this an accurate characterization?

RV: Yes, that's accurate. Zoho CloudSQL is about making the data in the Zoho cloud more accessible for our customers. SQL is something that most corporate developers know and are familiar with.


JB: The diagram on your December 2 blog post shows CloudSQL being able to access other Web services. Are there non-Zoho Web services you plan to support? Say, any Web services from StrikeIron, ProgrammableWeb, or even Google, etc.?

RV: Ah! You were paying attention! You noticed something that most other people missed. Yes, Zoho CloudSQL can be extended to non-Zoho services. At this point we're not focused or actively pursing that, since we need to first make sure that other Zoho services are accessible through CloudSQL first.

JB: Finally, I was intrigued to see that you're doing entity-mapping, which makes sense. It makes me think of the work Microsoft has been doing in its Project Astoria group (creating a framework now called ADO.NET), where they're using entity mapping to present a non-SQL-based interface to SQL-Server data. Do your RESTful APIs make use of this entity mapping? Does Zoho have plans to publish an Astoria-like interface to Zoho data?

RV: Our REST API should provide all the necessary details for developers, so we don't have plans for entity-mapping like Astoria. We would recommend CloudSQL, as the standard interface for developers, especially as we increase its coverage across Zoho applications.

To learn more about CloudSQL, visit this Zoho wiki page here.

Au Courant: Making the Most of Twitter for Business

Lots of high tech companies still don't use Twitter or use it well (e.g., they read tweets, but never post; they don't monitor keywords related to their business; they never initiate dialogs with strangers).

These tech-savvy but Twitter-naive companies could learn a thing or two from the world of letters. A growing number of small presses are taking advantage of the Twitter platform and 140-character tweets to share news, build communities, and offer promotions.

New Directions Press (@NewDirections) regularly posts about upcoming publications (and since they frequently involve Roberto Bolano, I update my bookstore wish list accordingly). Richard Nash of SoftSkull Press (@softskull) is another frequent Twitter user. He recently used Twitter to announce a new catalog:

Soft Skull Spring 09 catalog now available, yo. Download http://is.gd/8zrR & email for review copies. (Here's Winter 09 http://is.gd/8zE1 )

Today he retweeted a promotion from highly esteemed Graywolf Press (@GraywolfPress):

RT @GraywolfPress: REMINDER: get 25% off your entire order at www.graywolfpress.org. Use code "twitter25" in customer notes Good til Monday.

Businesses of all kinds, including software companies, could learn a thing or two about social media strategies from book people like Nash and the Twitter users at these other presses. And, of course, business people will also want to take advantage of that promotion from Graywolf—a longtime publisher of wonderful books. And then there's the new catalog from SoftSkull and ...

Moore's Law for Data Integration

The year is 1999, and your company wants to customize its new CRM system so sales people can access contract records from the Finance department's database. The CRM system itself took 9 months to deploy. Everyone's tired of the training classes. The six consultants who implemented the CRM system have lost a few people and gained a few people. Looks like they'll be bringing in someone else to manage this customization, which should take 4-5 months, tops.

Flashforward to 2008. It's September, you're using SugarCRM as your CRM system. You'd like to, again, integrate your CRM system with your Finance system. This time you'd like to do it using MindTouch Deki, a popular open source wiki and collaboration platform, and SnapLogic, an open source data integration framework. Good choice: the entire project, from start to finish, is completed in under two weeks.

No big, expensive consulting contracts. No "tent village" of Big Four consultants camped out by the computer room. An IT manager conceives the project, and within two weeks, it's done.

There's a paradigm shift that's occurred in enterprise IT, and it promises to make the next few years genuinely exciting, despite the downturn.

Over the past decade, application vendors have learned the wisdom of first, opening their APIs, and second, eliminating the complexity of their APIs by adopting a RESTful Web services model for integration.

The result has been a increase in agility for business users and IT department that is proving to be as dramatic, in terms of applications deployments and end user experience, as Moore's law has been for hardware development. Moore's law, you'll recall, is Gordon Moore's observation, first made in 1965, that "the number of transistors that can be placed inexpensively on an integrated circuit has increased exponentially, doubling approximately every two years" (Wikipedia). Thanks to Moore's law, your bookbag can hold a laptop more powerful that a mainframe from a few decades ago. And you watch videos on your MP3 player, which is roughly the size of a pack of gum.

Open source and RESTful APIs (which make use of basic GETs and PUTs, rather than relying on more complex messaging schemes) give enterprise IT organization powerful building blocks for rapidly building new, powerful application solutions—applications that literally would have required man years of programming less than a decade ago.

A year ago, there was a lot of buzz about enterprise mashups that blended two or more data sources. Mashups are still exciting, but what's equally exciting is a kind of mashup occurring at the application level.

Today's announcement by Salesforce.com that its Force.com development platform would work with Facebook APIs, enabling Salesforce.com's 100,000-strong developer community to more easily access the vast library of applications built on Salesforce, is more evidence of this trend.

Other evidence of this trend:

• The ongoing success of the Web site, ProgrammableWeb, which serves as a portal for discovering application APIs and mashups. ProgrammableWeb now has 1,000 APIs in its API directory. Clearly, a lot of companies are publishing APIs.
  
• Once published, APIs often become the dominant channel for accessing an application. As ProgrammableWeb's John Musser points out in a blog post about the 1,000-API milestone:
  
  
  
  • 60% of eBay's listing come from their APIs, rather than through their browser-based interface.
    
  • Twitter's APIs carry 10x the traffic of its Web site.
    
  
  
• REST is becoming the dominant programming model for APIs. 63% of the APIs listed in ProgrammableWeb's directory are RESTful.
  
• New Content Management Systems, such as Alfresco Enterprise 3.0, feature REST interfaces so they can easily access business data from other IT systems.
  
• New applications, such as an increasingly popular network management/IT operations platform offered by an Indian company, use REST APIs to facilitate communication among components. RESTful integration enables one product, for example, a network troubleshooting tool, to easily pass information to a related component, such as a trouble-ticketing application used by a help desk.
  

Enterprise Management Associates analyst Dennis Drogseth is fond of saying that enterprise IT organizations know they need to move to a "lego world." No single IT system has a monopoly on vital data. Best practices call for the automated flow of information seamlessly from one system to another: from a network diagnostic tool to a help desk application, for example, or from a finance system into a wiki, which in turn is embedded in SugarCRM. The key to that flow is having components designed to fit together with other components, even from other vendors.

It sounds like a lot of work. It's dramatically less work than it used to be. Thanks to open APIs, REST, and Web-centric architectures, data integration and application development can beneft from a Moore's law of their own.

Getting Strategic

Whether you're embarking on your normal planning for the next calendar year, or participating in emergency planning sessions in response to the economic slow-down, it's worth remembering the core attributes and benefits of a strategic plan.

Here's a summary from an earlier post.

About that Economic Slowdown

Another sign of the slowing economy: a sharp drop in retail sales in September. Here's the news from AP:

The Commerce Department reported Wednesday retail sales decreased 1.2 percent last month, nearly double the 0.7 percent drop that had been expected. It was the biggest decline since retail sales fell by 1.4 percent in August 2005.


The bigger-than-expected decline significantly increased the risks of a recession because consumer spending is two-thirds of total economic activity.

How big will this recession be? How long will it last, and what will the economy look like on the other side? Venture capitalist Paul Kedrosky offers his predictions, which include:

• "The unwinding of all this credit bubble will take longer than most people expect, and the damage will continue to be broader than most expect. Beyond banks and financial institutions, it will include many municipalities, some large-cap tech names reliant on major debt-financed network buildouts, a host of debt-financed non-financial companies, and some sovereign nations. Total cost: Bridgewater's $2.7-trillion looks close enough to me .
  
• We are already in a recession that will last well into the the fourth quarter of next year.
  
• Unemployment may touch 9% in the U.S. at trough.
  
• Housing will fall 10-15% further in U.S., and we are only beginning major declines in Canada, U.K., Australia, and elsewhere.
  
• U.S. consumers will become much more aggressive savers, both through debt reduction and direct saving. Similarly, future fiscal stimulus will largely be saved in service of this overdue need to fix domestic balance sheets.
  
• Commodities will stay under pressure for the next two years,and then reverse savagely as developed countries emerge from recession at very similar times. We have newly resynchronized the global economies, which will have immense consequences.
  
• Coming out the other side, we will see a barbell economy, with growth and investor interest at the mega-cap consolidator end, and at the entrepreneurial smaller end. The latter will be driven by major developments in clean technology, in particular, which was just given a two-year window to gestate before the major economies worldwide turn higher and begin driving energy prices straight up."
  

Read more of Paul Kedrosky's analysis here.

Two Suggestions for Social Media Strategies

A friend of mine who runs a PR agency called me yesterday, asking for my thoughts on social media strategies. Here's what I told him.

In addition to all the standard advice—start a blog, comment on other people's blogs, participate in forums, communicate and collaborate with your customers online, etc.—remember that an effective social media strategy involves both online and offline work. The online work is fairly obvious (the aforementioned blogs, etc.). The offline work may not be obvious, but it's pretty simple. Go meet with the people in your online communities. Blog about topics, then go to events where people talk about those topics. Meet new people there, and continue the relationships online. Your customers and partners are real, walking, talking, full-blooded people, not just users who who filled out profiles on your site. Meet them face-to-face, and have conversations you would never have online. If you think social media means hunkering down in a cubicle, blogging, linking, and fine-tuning SEO, think again. Social media is about being social with other people, online and off. Budget for travel. Get your social media marketing manager out there at trade shows, in restaurants, in blogger's lounges, and at any other location where your community hangs out.

Second, as I've said before about growing communities in the open source world, you've got to both "do" and "show that you do." In other words, you've got to document all your community work. If you show up at an unconference attended by 250 people in Cambridge, chances are that 50 people at most will remember meeting you (maybe 100 or 200, if you're already a big name or especially vocal or you're the guy wearing the weird hat with the flashing lights—but do people really know that guy?). None of your prospective customers in St. Louis or Chicago or London will know you went there. So you need to blog about. Post photos on Flickr. Link. Document your sociability and your community involvement. That way, your community involvement will have a much broader reach.

Ever notice that social media pundits are always posting pictures of themselves drinking good burgundy or Belgian beer with their fellow social media pundits? Now you know why. (Besides the fact that they're just having fun. Which is a good idea, too. And good burgundies should be celebrated, I suppose.)

Now, if anyone out there would like to grab a beer and continue this discussion, just let me know.

Data from a Straw

I originally wrote this piece after attending the Enterprise 2.0 Conference in Boston in June. I'm just back from the Web 2.0 Expo in New York, and I see that the ideas still apply. So here are some thoughts on data integration and social computing platforms. -JB

At the Enterprise 2.0 conference in Boston, there was a lot of talk about data. By applying Web 2.0 technology and practices—blogs, wikis, social networks, tagging, RSS, etc.—Enterprise 2.0 would transform enterprise IT infrastructure and foster the collaboration and knowledge-sharing promised by earlier technology practices such as knowledge management. In this new era, users at last will be able to find data easily and discover who else in the company has similar interests and pertinent knowledge. Through collaboration platforms such as Microsoft SharePoint and Jive Software Clearspace, data previously buried in email messages and PC desktops would be published on company blogs and wikis, where it could be found, read, and elaborated upon by coworkers and, if appropriate, by partners and customers.

The software companies creating these portals recognize that a lot of valuable data isn’t found in email or Words documents; instead, it's distributed across data centers and departments in databases and data warehouses. So the portal vendors talk about being able to access Oracle and SAP and other enterprise data sources, in order to pull this data into the collaboration platform.

But as I talked to vendors, I found their views of data access in many cases to be overly simplistic. Their premise seemed to be that all one needs to do is attach a connector to a data source and suck the data out, much as one might stick a straw into a paper cup and extract whatever concoction is sloshing about inside.

If you talk to data integration experts in data centers—or if you talk to security officers for Fortune 1000 companies—you quickly discover that the requirements for data access are much more varied and nuanced. It's rare that you’ll actually want to simply extract data and, say, stage it in an Excel worksheet on a server where it can be accessed by a homogeneous group of authorized users. More likely, you’ll want to apply access controls before the data even reaches a collaboration server, and you’ll need several different views of the data, based on business needs and permissions.

Instead of simply extracting data, it's more useful to think in terms of data access, data transformation, and data delivery. The tables below compare these approaches.

First, here’s the kind of straightforward data access that software vendors often talk about.

Table 1: Simple Data Access

Data Source	Data Access
Customer database	Post customer records as Excel spreadsheet for SharePoint

Next, here's a more realistic scenario, at least for organizations operating under security policies or industry regulations that mandate data security and data governance.

Table 2: Data Access with Support for Data Transformation and Data Delivery

Data Source	Data Access	Data Transformation	Data Delivery
Customer database	Query customer records, presenting only columns 1, 2, 5, and 7	Convert dollars to Euro	Post results to spreadsheet or Web page accessed by EMEA marketing group
	Query customer records, collecting columns 1-5 and 7-9	Add a unique ID to each record for use in this project	Post results to portal used by Private Client Group
	Query customer records, returning columns 1-4		Make this query executable for customer service agents working on the customer service portal

In enterprises operating with strict security and compliance controls, it's rare for data to be simply dumped from a database and made broadly accessible. Policy compliance requires tighter controls over data access (permission to extract the data from its source) and data delivery (the presentation of data to specific users).

Businesses—and software vendors—ought to recognize the critical importance of data transformation: changing, reformatting, or editing data to suit its particular purpose and audience. There's no point in delivering too much data, or financial results in dollars when they should be in yen, or raw data from three sources that end users have to combine for themselves through machinations with spreadsheets. In the real world of harried workers overloaded with information, data transformation is an essential capability for any effective solution for data management and knowledge sharing.

Just as Enterprise 2.0 frees workers from the clutter of irrelevant email messages, so flexible data access and transformation practices can ensure that the right users receive the right data at the right time. The goal should be to get everyone all the data they need—and nothing more.

Conclusion

At the Enterprise 2.0 Conference, it was obvious that software vendors of collaboration and community platforms have made clear progress developing attractive, usable front-ends. Now it's time to apply that same energy and thoughtfulness to developing the back end—data access, transformation, and delivery—in order to realize the full vision of business-ready data platforms for Enterprise 2.0.

Postscript: Since I wrote blog post back in June, SnapLogic, an open source data integration vendor and a client of mine, formed a partnership with MindTouch, an open source wiki company, to create a Customer Relationship Management (CRM) solution building on the kind of custom-tailored data access, transformation, and delivery I described above. In the SnapLogic-MindTouch solution (summarized with a diagram here), CRM applications such as Salesforce.com and SugarCRM are extended with collaborative dashboards based on MindTouch's wiki platform. The wiki is configured with SnapLogic data integration pipelines, enabling CRM users to securely access financial data and customer support records for prospects and customers. No tell-all spreadsheets insecurely posted on servers. Instead, a wealth of account-specific data is made available to authorized users.

I expect will see more partnerships like this one in the coming months.

In the Flesh

I had to laugh at one of the visitors to the show floor at Web 2.0 Expo last week.

The scene: The expo floor of the Javits Center on Thursday afternoon. The show is winding down. A couple of exhibitors have begun taking down signs and packing up laptops, though officially the show will remain open for another half hour.

On business for a client, I've been here and there on the show floor, talking to potential business partners. When I return to my client's booth, I find a middle-aged, pot-bellied man standing in front of our demo station, talking to a buddy of his. The pot-bellied man has set his laptop on our table and he's instructing his buddy, who's ill-shaven and looking a little worse for wear in his black suit, about which companies he should go call on. The pot-bellied man is unabashedly treating our booth as his private conference area.

I ask him if he'd like to see a demo. He says sure. I take control of our demo station, and I show him our demo: CRM software integrated with a Wiki. It's been wowing people all week.

He nods, but he's not really interested. I ask what he does. He offers to show me. He flips open his laptop, positions it in the middle of our demo table, and starts running online ads. You've seen them: video ads featuring a well-dressed model who wanders in from the side of the screen, blocks the Web page you're trying to view, and starts talking to you. He has a long list of demo links. I watch nattily dressed salespeople blocking the Web pages of several national retail brands.

I point out that my client sells software primarily to IT people, and that IT people have a low tolerance for anything that smacks of marketing, let alone anything as out-and-out slick and salesy as this. He slaps his business card on our table, makes perfunctory social remarks, and moves on.

Later, it occurred to me. I've always found ads like that intrusive, presumptuous, and annoying. And what kind of person would create and peddle ads like that? Someone who is himself intrusive, presumptuous, and annoying. Someone who would take over your booth at a tradeshow and use it as his personal office, blocking your computer and carrying on.

"By their fruits, ye shall know them." I had just seen a grating interstitial ad in the flesh.

At Web 2.0 Expo in New York, an Instant Solution for CRM Integration

The first Web 2.0 Expo in New York kicks off today. About 120 vendors will be showing off the latest in software and services related to community, Web-contributed content, and other aspects of Web 2.0.

Among the exhibitors will be SnapLogic, a client of mine, who just announced an OEM deal with MindTouch. The two companies have created a new software solution that extends CRM applications such as SugarCRM to include live data from other business systems, such as finance applications, databases, and more. The diagram below captures the gist of the solution.

Why is this useful? Now a salesperson reviewing customer data in a CRM application can also see relevant data from other business systems. Is this customer due for a maintenance renewal? What's their payment history? Do they typically pay on time? Have other sales, marketing, or support people left comments offering advice about working with this customer? Deki for CRM puts answers to questions like those right in the CRM application window, formatted in a readable, editable wiki workspace.

Five years ago, a solution like this would have entailed tens or hundreds of thousands of dollars and a six- to twelve-month roll-out of a major CRM application like Siebel. SnapLogic and MindTouch put the solution together in a matter of weeks. Customers can have it up and running in a day.

You can read more about Deki for CRM here.

Integrating SaaS Applications

Software-as-a-Service (SaaS) certainly qualifies as a hot trend in IT. According to Gartner, the SaaS market is growing at twice the rate of the software market overall. SaaS applications such as Salesforce.com are especially popular with SMBs, but even large enterprises with more than 25,000 employees are devoting 11% of their current software budgets to SaaS.

SaaS, then, is big and getting bigger. But getting bigger, too, is the challenge of integrating SaaS applications with the rest of an organization's IT infrastructure. After all, no organization can afford to let the latest SaaS application it subscribes to become another data silo. SaaS applications need to share data with other enterprise applications and IT assets.

Integrating SaaS applications can be difficult, in part because there's an technology mismatch between SaaS application design and the community of SaaS customers. Most SaaS applications feature SOAP Web services interfaces whose complexity and sophistication are far beyond the programming reach of SMBs, the biggest users of SaaS. Even in large organizations with SOAP programmers on staff, SOAP programming often remains a rare skill. SOAP programmers are usually assigned to big internal initiatives. They're usually not available to help a department integrate its data sources with a new, cost-saving SaaS application.

One of my clients, SnapLogic, has published a new white paper on SaaS integration. If you're working with SaaS applications now, or thinking of subscribing to Salesforce.com, SugarCRM, or any other popular SaaS application, the white paper is probably worth a read. You can register for it here.

Two Pictures of Hope on a Friday Night in Boston

Friday night, I walked up Boylston Street on my way to Fenway Park for the Red Sox-Orioles game. Just past the Hynes Center, I encountered a line of people stretching down the street and wrapping around the corner. Some hot new club or restaurant? No. People waiting to purchase new iPhone 3G's at the Apple Store.



At Fenway, the Orioles pulled ahead in the first inning and held on to win the game. But at the bottom of the ninth, there was one of those moments that makes me love baseball. Baltimore was up 7 to 3, but through hits and walks, the Red Sox found themselves at bat with two outs and the bases loaded. Kevin Youkilis came to the plate. The count went to 2 and 2. The Red Sox fans were on their feet. If he hit a grand slam, the game would be tied. Fate could change in an instant. And then . . . who knows?



Alas, he swung and missed. The Orioles won.

Oh, well. It was a perfect summer evening, and everyone left the stadium in a good mood.

Is Your Schedule Based on a Guess or an Estimate?

Clearcut Ideals and Messy Realities
Ever work with Microsoft Project? Ever spend hours and hours—or rather days and weeks—gathering project requirements and schedule projections from team members ("how many days do you think your part of the project will take? OK, we'll say three"), so you can generate draft after draft of Gantt charts and timelines, leading up to the official copy that you present to management, then print out—page after page of solid lines presented in a staggered order like a vast, irregular staircase—and tape up on the wall of your office?

If you've ever worked on a project plan like that, you may find yourself holding your breath now, because you know, in the pit of your stomach, that the process I've just described is only the beginning. It's only the beginning, because inevitably, important aspects of the project change. Some tasks finish late; others finish early; others disappear from the schedule entirely, while new ones, unimagined in the planning stages, miraculously appear. If you're lucky—and a lot of people are—the team will manage to complete the project—or some semblance of it—overall.

When the project is finished and you look back at all those charts you printed out and taped do your wall, how do you feel? Don't those solid lines and neat demarcations—progressing across the page with the neat precision of a well drilled marching band—now look hopelessly optimistic—like the budget projections of a politician or the crop forecasts in a Soviet five-year plan? I mean, how could anything as random as a bunch of human beings working on complex project ever proceed in such a neat manner, with such precision?

But what's your alternative? You can't afford to be vague when you're scheduling a project, can you? And you do have to produce some kind of schedule or plan. And whether you use Microsoft Project or some other planning program, most likely the output is going to be hard lines, those promises of firm dates, neat beginnings and endings.

Nothing really ever works out that way. Precise project scheduling is like penciling in a landing strip for a water balloon.

A New Approach to Planning
One of the most useful products I saw demonstrated at the Enterprise 2.0 Conference in Boston didn't really have much to do with Enterprise 2.0, as far as I could tell. It's a piece of collaboration software, but it's no more collaborative than Project or other planning tools that have been around for over a decade. It doesn't explicitly make use of network effects, though it does support discussion threads and Web-based scheduling. Most importantly, though, it offers a new and potentially very useful approach to planning.

The project is called Liquid Planner, and it's based on the premise, which seems blindingly obvious in retrospect, that accurate planning should be based on estimates and probabilities, not hard certainties.

Bruce Henry, whose title at Liquid Planner is Director of Rocket Science, explained the "Ah-ha!" moment that led to the founding of the company. He and some of his colleagues from Expedia were taking a class from Steve McConnell, the author of Software Estimation: Demystifying the Black Art and Rapid Development, among other books. McConnell pointing out that when you ask how long someone will take to do something, and they say, "4 to 6 days," and you say, "OK, we'll call it 5," you're making a guess, not an estimate. Estimates are based on ranges and probabilities. Guesses pick a number and use it as the basis of planning.

Most organizations base their planning on guesses. It's not surprising then, that most schedules slip, and that most Gantt charts end up looking hopelessly optimistic.

Two of Henry's colleagues from Expedia—Charles Seybold and Jason Carlson—founded Liquid Planner to address this problem. Henry joined them and wrote the probability engine that's at the heart of Liquid Planner's software. The goal: make project planning more accurate by enabling teams to base their schedules on realistic probabilities rather than unrealistic "certainties."

Here's a screenshot of the software, showing probabilities and date ranges for tasks.



Henry points out that seeing a list of probabilities can raise red flags early in the planning process. For example, if managers notice that a particular task has only a 30% chance of completing on time, they might ask why. They might discover dependencies they weren't aware of. They might be able to apply people and resources to address any dependencies or shortcomings, greatly increasing the task's chance of completing on time.

I haven't tried this software myself, but it seems like it's worth a look for any team beginning a new project.

The company launched its public Beta at the DEMO Conference in February, 2008. Since then, over 11,000 users and organizations Philips, Butterball Farms, and Reed Business Information have signed up for their online service. At the Enterprise 2.0 Conference in Boston in June, 2008, Liquid Planner announced its commercial version.

The service is free for teams with up to 3 members, for 501(c)(3) non-profits, and for educational users. Larger teams can take advantage of a free 15-day trial, then pay monthly or annual fees per user. You'll find pricing details here.

All That Data

None of my clients were exhibiting in the demo area of the Enterprise 2.0 Conference, so when the demo floor was open, I had the opportunity to stroll through the aisles and talk to various vendors instead of manning a booth and explaining a particular product or technology to passersby.

The Enterprise 2.0 movement—applying Web 2.0 technology to problems and processes within the enterprise—promises to transform the online experience of workers in companies large and small. Instead of being deluged with email and interrupted by IM, workers can access company news and information in RSS feeds when it's convenient. Instead of emailing Word documents to everyone on a team and trying to coordinate all the changes and comments, authors can jointly edit documents with tools like Google docs. The table below summarizes some of these changes:

Category	Web 1.0	Enterprise 2.0	Benefits from New Approach
Knowledge sharing	Email and irregular postings on portals	Wikis and blog posts	Publishes data in a more permanent format Makes information easier to discover Reaches stake-holders outside one's immediate group Enables non-technical users to post information without requiring custom clients or help from IT
Notification of changes and news	Email and phone calls	RSS	Occurs automatically when blogs or wikis are updated Reaches all interested parties, even those the author might not know about

(For more about this new way of working, and some thoughts on the pros and cons of email in particular, see this recent post by Harvard Business School's Andrew McAfee.)

Clearly, these platforms and portals are going to store a lot of data. How do we make it searchable? How we enable a product manager for a new leather cleaner product to find the blog post from three months ago that discussed product requirements for a similar product being developed by a partner in Switzerland.

One solution is to apply tags—meta-data keywords that summarize the content of a blog post, Web page, or some other piece of content. For example, the tags for that product requirement blog post might be "leather cleaner, research, product requirements, survey, partner, Switzerland."

Thomas Vander Wal is a consultant who has spent a great deal of time thinking about tagging and classifying data. He coined the term folksonomy to distinguish a bottom-up approach to classifying data, in which users apply the tags they think are relevant, from more traditional top-down approaches that rely on formal vocabularies and specialists in information taxonomy.

Getting users in the habit of tagging content and tagging it usefully can be a bit of a challenge, however. As Vander Wal pointed out in his presentation at the Enterprise 2.0 Conference, you can end up with problems like users not tagging content at all or using tags that are so general they prove useless in future searches. He gave the example of a company promising to reward workers who tagged documents, then discovering that workers were meeting this requirement by applying tags like "document." More tags are better than fewer, and overall, however they are applied, tags must serve the purpose of distinguishing one document for another.

The creators of software tools have a role to play here. They can create applications that prompt users to tag data. Some applications might even analyze data as it's being entered and propose tags for it.

Shortly after Vander Wal's talk, I found myself strolling through the demo area, wondering how the many social collaboration programs on display handled this important issue.

At the Microsoft booth, I heard from someone demoing SharePoint that customers simply don't use tagging all that much. The idea of tagging, in this person's opinion, was not turning out to be a success.

At the ThoughtFarmer booth, I met Darren Gibbons, the co-creator of the ThoughtFarmer intranet solution, and the president of OpenRoad Communications. I asked Darren what he thought about tagging. Should be automated? Left to individuals? How could it be made to work?

Here's a video with his answer, which is that tagging works best when it benefits both the tagger and the community overall.



On the next aisle, I got talking to Padmanabh Dabke, the founder and CTO of a company called SpigIt, about the challenge of creating meaningful tags on a large scale. SpigIt makes software that enables companies to collect from large communities of employees and customers, then rank the ideas to decide which ones should be pursued. In addition to offering guidance for investment, the software helps managers identify which employees and customers are consistently coming up with the best ideas.

Nabh pointed out that some older technology—namely, expert systems—could be applied to sort through the torrent of data in online communities and aid in speedy classification. (Pardon my shaky camera work in the first few moments of our conversation.)



Any conclusions? Yes. It's clear that companies are replacing or upgrading their old Web 1.0 intranets with these new, easier-to-use community platforms. Workers are getting used to blogging and using tools like Wikis and RSS feeds. Tagging will make all these tools more useful, and the best practices for tagging will probably combine user habits, helpful user interfaces, and powerful processing engines like that described by Nabh.

The Enterprise 2.0 Conference in Boston

I'm spending this week at the Enterprise 2.0 Conference in Boston. Monday started strong with an overview of Enterprise 2.0 concepts and tools by Dion Hinchcliffe. That evening, leading cloud vendors—Amazon, Google, and Salesforce—sat on a stage with potential customers in a lively, in-depth discussion arranged by TechWeb's David Berlind.

Tuesday's sessions were more uneven. One of the key topics of the day turned out to be tagging. Thomas Vander Wal, a social bookmarking consultant and the coiner of the term folksonomy, offered a look at the pros and cons of various approaches to managing tagging on a grand scale. Later in the afternoon, my discussions with software vendors in the demo area of the conference returned to the subject again.

More details—and a few video interviews—in upcoming posts.

Creativity in the Organization and How to Present Like Steve Jobs

Two articles worth reading:

First, an article recommended by Dion Hinchcliffe on Getting Down to the Business of Creativity. After conducting a three-year study of the daily creative work of hundreds of people, Professor Teresa Amabile at HBS offers these suggestions for fostering creativity in the workplace:

"Support employees' progress in their work every day. Set clear and meaningful goals for them; provide direct help, versus hindrance; offer adequate resources and time; respond to successes and failures by drawing on the experience as a learning opportunity, not just a moment to praise or reprimand; and establish a culture where people are treated with respect."

This article also offers an apt definition of entrepreneurship: "The pursuit of opportunity beyond the resources you currently control."

Second, article over on Bnet.com about How to Present Like Steve Jobs. Lots of people, including me, consider Jobs the best presenter around, so it's worthwhile to consider his approach to planning and delivering a presentation.

Two Upcoming Events in Boston

BarCampBoston 3, where coders and entrepreneurs will meet to share ideas and to hack code. The fun starts on Saturday, May 17 and continues through Sunday afternoon.



O'Reilly's Ignite Boston 3 on the evening of May 29 at Tommy Doyle's at Harvard Square.

Fast-paced and fun. Lots of ideas. Oh, and there's beer, too.

Learn more here.

The Numbers Racket

As bad as many of the economic numbers are these days, is truth far worse? Are the numbers proffered by the government and batted around by financial analysts simply wrong?

In an article called "Numbers Racket" in the May 2008 issue of Harper's, Kevin Phillips, author of Bad Money, presents a blow-by-blow account of how, beginning with the Kennedy administration in 1961, Democratic and Republican administrations have jiggered economic indicators and other numbers to minimize the bad news that would otherwise have to be reported. Rising housing costs are excluded from inflation. Rising fuel costs are excluded from inflation. People who would like a job but who have given up looking for one are excluded from unemployment numbers.

What would a more accurate assessment of our economy look like? Phillips writes:

The real numbers, to most economically minded Americans, would be a face full of cold water. Based on the criteria in place a quarter century ago, today's U.S. unemployment is somewhere between 9 percent and 12 percent; the inflation rate is as high as 7 or even 10 percent; economic growth since the recession of 2001 has been mediocre, despite a huge surge in the wealth and incomes of the superrich, and we are falling back into recession.

Certainly, at least to our household, inflation seems higher than the low numbers reported by the feds would suggest.

Bad data is deceptive, but it's pernicious in other ways, too. Artificially low numbers have led decision-makers to make bad decisions with far-reaching consequences. Phillips cites this example:

As Robert Hardaway, a professor at the University of Denver, pointed out last September, the subprime lending crises "can be directly traced back to the [1983] BLS decision to include the price of housing from the CPI. . . . With the illusion of low inflation inducing lenders to offer 6 percent loans, not only has speculation run rampant on the expectations of ever-rising home prices, but home buyers by the millions have been tricked into buying homes even though they only qualified for the teaser rates."

In economics, as in foreign policy, American are only too adept at spinning rose-colored fantasies, acting on them, and then blinking with the wonder of innocents at the mess that results.

You can find the article online here, but access is limited to subscribers. Of course, a subscription to Harper's has always been pretty affordable, inflation notwithstanding.

Organizing without Organizations (or Hush! Caution! Echoland! Here Comes Everybody)

Consider the following:

• In 1992, the Boston Globe runs news stories about a Catholic priest who abused children for decades before being pulled from rotation among parishes. Boston parishioners are upset, but their anger lacks any organizational punch. The most they can do is mutter into their missals and write a letter or two. The Church leaders treat the scandal as an internal affair, and the incident dies down. In 2002, the Globe runs similar news stories about yet another priest, John Geoghan, who abused children for several decades. This time the parishioners create an organization, Voice of the Faithful, whose ranks swell to 25,000 people within a few months. Voice of the Faithful's clout is such that after decades of successfully squelching criticism and revolt, the local Catholic bishop is forced to resign.
  
• In the 2004 presidential election, Howard Dean’s supporters organize online, produce surprisingly large crowds of supporters at events, and raise more funds than the rival campaigns. Despite this show of strength, Dean does poorly in the primaries and the nomination goes to John Kerry.
  
• In 1992, a young Finnish programmer posts a message on a message board, announcing that he’s going to free operating system as a hobby. Fifteen years later, the free operating system, Linux, is running on nearly 40% of the world’s servers.
  
• One morning in Cairo, an Egyptian blogger is arrested for libel. Using Twitter, a micro-blogging service (micro-blogging is blogging with very short blog posts), he informs his friends, who rally lawyers, and by 11 pm that night he is free.
  

What all these stories have in common is that social tools such as email enabled people to communicate and collaborate in ways that would have been unimaginable a decade or two ago. Email, wikis, blogs, and social network sites such as Facebook and MySpace make it easy for people to connect to one another and share information. The cost of communicating is negligible. The speed of communication is almost instant. The ramifications are many.

As Clay Shirky, an IT consultant and teacher at NYU's Interactive Telecommunications Program, writes in his new book, Here Comes Everybody: The Power of Organizing without Organizations, social tools like email have changed social behavior forever. Not all online social endeavors will succeed—the Dean campaign failed because it had a committed core but lacked a broad, committed base—but many will succeed, a few wildly so. As the Boston diocese discovered, when large numbers of people suddenly have the ability to organize easily, they become a powerful, just about unstoppable force for change. New broad-based movements can start at any time. And no traditional organization or government can contain them.

We're all aware of at least some of the impact of these tools: all the teenagers on Facebook, solicitations from PACs for emailing Congress, and so on. But few of us have surveyed the broad effects of these tools in a systematic way to gain an understanding of how people, social contracts, and software are all interacting. Shirky gives us this survey, and it's lucid and thought-provoking.

He points out:

"We now have communications tools that are flexible enough to match our social capabilities, and we are witnessing the rise of new ways of coordinating action that take advantage of that change. . . . These tools have radically altered the old limits on the size, sophistication, and scope of unsupervised effort."

Email, blogs, and wikis dramatically lower the cost of coordinating group efforts. They enable groups to organize quickly and easily and to take on daunting projects that traditional organizations, with their hierarchical structures and cost-consciousness, would never consider. When organization becomes easy, more people will organize. When more people can organize, new forces for change will sweep society, affecting government, business, culture, and personal lives.

In just under 300 pages, Shirky explores the characteristics, implications, and results of the new types of social interactions and organizations that have emerged in the past decade. Here’s a quick survey of some of his insights:

• The power law distribution (think of the curve you’ve seen in discussion of the long tail phenomenon) describes all sorts of social behavior, from contributors to Wikipedia to the popularity of blogs to programming work done on open source software projects.
  
• Participation in most social systems follows a power law distribution. A few contributors do most of the work. Most contributors do almost nothing. The work of the industrious few provides so much value to the less industrious majority, though, that they feel motivated to contribute, improving the breadth and quality of the entire project.
  
• If the majority of blog entries, MySpace pages, and other online content strikes you as trivial and inane; don’t worry. It doesn’t really matter, because you’re not the intended audience. These communications are public, but their intended for a select audience of friends and family. Shirk likens blog posts such as “What’s happening, dude?” to a conversation overheard in a shopping mall. Comparing the artistic quality of a blog post or MySpace page to a story in the New Yorker misses the point. And you will be overlooking the manifest transformative power of these new tools, were you simply to dismiss them because of their low editorial standards.
  
• As communication moves online, it becomes permanent through digital archives. Until now, most of what was preserved was official. Now everyday remarks are by default permanently "on the record."
  
• Social tools like the Web site Meetup.com enable small, splintered groups to meet and form. People who have explicitly left organizations (such as churches) can now easily organize themselves.
  
• Group projects follow this pattern: offer, tool, bargain. Someone initiates the group by making an offer, the group uses a tool to undertake group activity, and the behavior and expectations of the group are governed by an explicit bargain. Selecting the wrong offer, tool, or bargain can doom a project. That said, the particulars of group projects are so varied and complex, it would be folly to proscribe specific offers, tools, or bargains categorically.
  
• The best ideas in an organization usually come from people who bridge social groups within the organization. They’re able to assess a situation with a fresh perspective, and they feel less compulsion for conforming with peers and for preserving the status quo of a department.
  
• "No whining!" is a rule common to many social groups (and if that upsets you, please keep it to yourself).
  

Shirky does an excellent job analyzing the results—intended and otherwise—of new forms of social organization, such as meetups and groups like Voices of the Faithful. Whether you’re interested in applying some of these group dynamics to your business, or you simply want to read an engaging account of how these tools are changing the world you live in, you’ll find this book well worth your time.

POSTSCRIPT

As a Joycean, I feel compelled to mention that the phrase "Here Comes Everybody" is a kind of motif from Finnigans Wake, where the phrase serves as the basis for all sorts of other phrases with the initials HCE. For a list of HCE phrases, which you can imagine applying to social networks in all sorts of ways, click here. As you might guess, the title of this blog post, "Hush! Caution! Echoland!" is an HCE phrase from FW.

The Importance of Benevolence for Business

Paul Graham of Y Combinator has a thoughtful essay on how wildly successful start-ups (e.g., Google) often behave like non-profits. He discusses how being good can help a company become more focused, resilient, and successful.

You'll find a print version here and a video version here.

Keeping a Small Business Focused and on Track

In the CrowdVine discussion thread for the upcoming Web 2.0 Expo, Tony Stubblebine of Crowdvine asked people what tricks they use to keep their business on track.

Here's my answer, drawing on my experience with a number of organizations:

To keep a business on track, I use:

1) A strategic plan that lays out major goals (e.g., win at least 3 reference customers in key market X), each of which can be tracked by objectively measured milestones or supporting goals. Almost everyone thinks of a revenue target as an obvious strategic goal—and it can be—but it's just as important to think about other goals that address the company's position in the market and the company's various capacities (engineering capacity, sales capacity, etc.). In some cases, it might be better to settle for a lower revenue target while managing the company in such a way to address a latent deficiency. For example, it might be better for a cash-strapped company to manage its activities and sales expenses to end the year with more cash (but lower top-line revenue), instead of blowing out a revenue number with expensive sales campaigns and ending up with nothing in bank (again) and no ability to, say, hire critical engineers for the product overhaul planned for next year.

The main thing is to ask: Where do you want to be by the end of the year? What do you want to have in motion? Capture those ideas in specific goals and objectives.

All the company's major activities should be subsumed in the strategic plan. If you're working on something, you should be able to identify which one of the 5-10 major goals for the company it relates to. (And if you find that you've identified more than 10 goals, pare them down, especially if this is your first time trying to manage operations for an entire fiscal year according to a strategic plan.) Every organization, even small consultancies and non-profits, should develop and manage to such a plan. At the end of the year, you should be able to look back and judge how well you've done. If you've developed a good plan, you're going to find that you've not only hit your revenue goals, but that you've built a stronger organization that's better positioned for success.

2) A company mission statement or a summary of core values (see Collins and Porras' "Built to Last"). For example, I consider being responsive to customers to be one of the highest values for my work. One of the ways I keep my business on track is by asking myself several times each day, "Am I being responsive to my customers? Is there someone I should be calling?" If there's a value or behavior that's key to your company, note it. Write it down. Live by it.

I also:

3) Align technology to support #1 and #2. Take advantage of what's new (e.g., some of the great stuff that's going to be shown off at the Web 2.0 show), but don't get distracted. Use everything you can find—from software to devices to the right office chairs—that eliminates distractions, promotes productivity, and helps your employees meet your organization's goals while working in accordance with core values.

4) Hire people who are genuinely interested in growing a great business. (Suggestion for hiring managers: mention that you have a strategic plan, and see how the candidate reacts. Does he or she ask to see it, or does he or she simply nod and move on?)

Fire and Motion

If you had to reduce your business strategy to three words, what would they be?

For Joel Spolsky, the co-founder of Fog Creek Software and author of blog Joel on Software, the three words would be "Fire and Motion."

"Fire and Motion" is a military phrase that refers to firing your weapon (and causing your enemy to cower), while running forward (thereby gaining territory and through your promixity to your enemy, improving your aim).

Spolsky explains how everyone from small businesses to large companies like Starbucks can use "fire and motion" to drive their businesses forward.

Spolsky's article appears in the April edition of Inc. Magazine. You can find an online copy here.

As a metaphor, "fire and motion" does a good job of conveying the relentless activity that successful businesses engage in. But I find the idea of "firing" a little vague.

Spolsky cites the example of Starbucks throwing McDonalds off their game. Starbucks educated a lot of America about what coffee could be (at least compared to the watery stuff served in most restaurants) and offered customers a superior sipping and dining experience—purple armchairs and wood paneling beat hard plastic chairs every time.

But it's hard to derive this strategic approach simply from the idea of "firing." Firing is attacking the enemy. Undoubtedly Starbucks is attacking McDonalds by marketing good coffee and giving their customers a more comfortable environment for consuming fast food. But Starbucks could have interpreted "firing" with some other strategy for attacking McDonalds: cheaper burgers, more burgers, aggressive advertising, etc. And these other approaches would not necessarily work as well as the strategy Starbucks has chosen.

To fire with maximum effect, it's important to pay attention to Spolsky's comment about successful companies setting the agenda for their markets.

If you look at a competitive market, the successful company is always the one setting the agenda and forcing competitors to match it. For example, JetBlue's version of fire and motion came in the form of a superior customer experience.

This is the key: To fire effectively, you've got to be more than busy. You've got to set the agenda.

That's where a blue ocean strategy comes in. A blue ocean strategy creates new products and services that redefine the costs and benefits of a solution, enabling a company to move from an intensively competitive "red ocean" market to a new "blue ocean" market, free from competitors.

A blue ocean strategy sets the agenda for a market:

• JetBlue redefined the US airline market
  
• Starbucks redefined the fast food market
  
• Yellow Tail redefined the US wine market (it's now the top selling wine in the US)
  
• Salesforce.com redefined the sales force automation market
  

To gain sufficient fire-power, you've got act on a blue ocean strategy. Simply keeping busy with traditional tactics (promotions, store openings, etc.) doesn't cut it.

This is why I recommend that management teams develop value curves for their products and markets, then use those curves to develop a blue ocean strategy designed to deliver dramatic growth. (Value curves are described by Kim and Mauborgne in their book Blue Ocean Strategy. I help start-ups with this type of planning.)

By comparing value curves, I can understand how Starbucks is firing at McDonalds. I can see what Starbucks is adding and subtracting from the traditional fast food experience. I can get a sense of how Starbuck's choices relate to its costs and prices. And I can better understand how Starbuck's new fast-food agenda creates trouble for McDonalds, and why McDonalds is now responding the way it is (i.e., with better coffee and more stylish decors) to align itself with the market as Starbucks has redefined it.

So, be relentless. Use "fire and motion." But to figure out how you're going to fire and what ammo you're going to load, develop a blue ocean strategy.

Is Someone in Your Company Publishing All Your Confidential Files? How Do You Know?

I've written about IT security, in one form or another, for almost a decade now, so I've seen more than my fair share of stories about virus and worm attacks, employees stealing confidential information, malware being used to extort money from large companies, and other nefarious acts of theft and sabotage. But I have to say that a pair of articles—one by John Foley and another by Avi Baumstein—in a recent issue of InformationWeek managed to rattle even me.

The topic is data leakage caused by peer-to-peer (P2P) file-sharing applications. P2P applications enable users to share and transmit files over a vast network of computers all running the same P2P software. In some P2P networks, a node simply makes files available for other nodes to discover and download. For example, I might put a bunch of documents in a sharing folder. You might use the P2P application to search for these documents, discover them, and copy them from my system to yours.

Another model of P2P network is specially designed for handling large media files, such as software distribution packages and movies. These networks use a "swarming" protocol such as BitTorrent to disassemble very large files, transmit them as hordes of little files, and then reassemble them into a copy of the original on the other end. Because there are hundreds, thousands, or even millions of computers functioning as nodes in the network, this type of P2P network offers a convenient solution for efficiently distributing large files, such as 150 MB software packages, without putting excessive load on any one CPU or segment in the network.

Sounds clever and convenient, right? But P2P file-sharing applications can also be dangerous, because many of them allow anonymous remote users to browse and transfer a lot more content that the computer owner may realize. Here's a typical example: Joe comes home from a long day's work at a large accounting firm. He wants to download a song he heard on the radio. He uses a P2P network to find a bootleg copy of the song. He downloads the song. What he doesn't realize is that when he's installing the P2P application and clicking Next, Next, Next, to get through the installation, he's making the entire contents of his laptop accessible to the P2P network. Other users of the network can now browse his laptop and download whatever they find.

His company's firewall? Bypassed. His company's security policies? Moot. Joe is not intending to do harm (well, other than perhaps grabbing a pirated version of a song), but by using P2P software, he's effectively negating the millions of dollars of security controls his IT has developed and implemented to keep their business data confidential and in compliance with regulations such as SOX and Gramm-Leach-Bliley. He's publishing all the confidential materials he has on his laptop. Chances are, he's got quite a few.

When InformationWeek reporters investigated P2P networks to find out just how much confidential data was being accidentally leaked by P2P networks, they were shocked at what they found. Users were inadvertently publishing "spreadsheets, billing data, health records, RFPs, internal audits, product specs, and meeting notes . . . files with the home and cell phone numbers of senators, confidential meeting notes, and fund-raising plans [for a state political party] . . . spreadsheets listing patients' names along with their HIV and hepatitis status . . . [and] a slew of court documents regarding a sticky divorce."

Limewire, the most popular client of a P2P solution called Gnutella, is supposedly installed on over 18% of all computers.

Three suggestions, then:

1. Read the full InformationWeek articles (here
   and here) and encourage your managers and employees to do the same.
   
2. Forbid or tightly control the use of P2P programs such Limewire on your business computers.
   
3. Have an IT engineer use one of these programs immediately to discover if your business is already exposed.
   

Tracking the Recession

The economy is all over the headlines, but if you'd like to see a lot of telltale graphs collected in one place, check out the DismalScientist's Recession Watch.

Happy St. Patrick's Day

Skip the bad jokes about beer and leprechauns today. Here's a better taste of things Irish: two and a half minutes of sheer magic as Tommy Peoples plays a strathspey called the Laird of Drumblair.

Enjoy.

Time for a Check-up

We're just about halfway through March, which means that Q1 is just about over. If you spent Fall 2007 or the first weeks of January putting together a strategic plan or business plan for 2008, it's time for a check-up.

A well-designed strategic plan includes measurable milestones supporting each objective. It's time to call a meeting with stake-holders and see how much progress you're making against your objectives and milestones.

Given the turmoil the financial markets are going through, it's probably also a good time to sanity-check your plans for the year. Adjustments to goals and changes of course may be in order.

And if you've made general plans that lack specific milestones and measurable objectives, it's time to sit down and define those specifics, too, so that you're able to gauge the progress you're making in each area. Judging activity against measurable outcomes is the best way to distinguish work that's truly productive from work that merely keeps everyone busy.

Casting a Critical Eye on Expenses of All Kinds

Jason Calacanis, the CEO of a Web search start-up called Mahalo, has posted a list of recommendations of saving money when running a start-up.

What I like about the list is its thoroughness: Calacanis has really thought about where to invest (comfortable chairs so his employees will be able to work long hours comfortably) and where not to (he buys cheap tables, because all a table needs to be able to do is hold stuff). His recommendation to eschew phone lines in favor of IM, Skype, and cell phones reflects a reality I see in other Silicon Valley start-ups. The company phone list and traditional phone lines an anachronism, once everyone's on Skype. I even know one CEO who never answers his direct dial number, because the only people who call him on that line are vendors pitching him services. (How long before tradeshow vendors, printers, and PR firms realize that trolling Skype is probably going to be more productive than looking up phone numbers on Web sites? How long, after that, before executives begin guarding their privacy more closely on Skype and Twitter?)

Calacanis's recommendation to buy employees a second monitor reminded me of some research conducted by Jacob Nielsen, Sun's Web usability expert. Nielsen found:

Big monitors are the easiest way to increase white-collar productivity, and anyone who makes at least $50,000 per year ought to have at least 1600x1200 screen resolution. A flat-panel display with this resolution currently costs less than $500. So, as long as the bigger display increases productivity by at least 0.5%, you'll recover the investment in less than a year.

The examples of cost savings that Calacanis cites probably resonate most strongly with those in software companies. But managers in other industries could do well to think as critically about what employees really need in order to be productive.

Longstanding industry habits and daily routines can lead us to take too much about our work environments for granted. As a result, we might overlook simple changes we can make to increase worker productivity, minimize distractions, and perhaps even increase employee morale. (I'll bet the folks at Mahalo appreciate those monitors and iPhones.) As Calacanis's list shows, it's useful to critique everything from furniture to communication infrastructure, and put as much thought into what you're leaving out as what you're keeping in.

Beware of Vista

For the second time in about a month, my Vista machine has automatically downloaded upgrades from Microsoft that have broken Word.

I have to say, don't go near this operating system unless you really have to.

Update:

Here's a Network World Community blog posting, describing the same problem:

http://www.networkworld.com/community/node/23685

And here's a link to a Microsoft Support page telling users how to edit the registry and delete a registry key to fix the problem:

http://support.microsoft.com/default.aspx?scid=kb;EN-US;940791

Second Update:

More problems with Vista, discovered by Microsoft executives no less, are detailed in this New York Times story.

Disrupted by IT

When I lived in the Bay Area and would mention at some dinner the latest turmoil in the IT industry, my friends in other lines of work (lawyers, for example) would shake their heads in wonder. To them, the hectic pace and chaos of the IT industry seemed unfathomable. Down in Silicon Valley, Fortune's Wheel seemed to be spinning at a giddy pace, hurtling hordes of twenty- and thirty-somethings through comically rapid cycles of failure and success. By comparison, my friends' own jobs outside of IT seemed rational, stable, and predictable.

For a lot of people in industries other than IT, that rationality, stability, and predictability are now at risk. These days, industries that use IT, are succumbing to the turmoil that has long characterized the IT industry itself.

Andrew McAfee is an associate professor at Harvard Business School. Erik Brynjolfsson is a professor at MIT's Sloan School of Management. McAfee and Byrnjolfsson have been studying the way that IT technology changes markets and competition outside the IT industry. What they've found is that industries that invest heavily in IT are beginning to behave like the IT industry itself. In other words, take a non-IT industry, add IT infrastructure and ability to automate business processes, bridge operations, increase efficiencies, and so on—and you get the market turmoil and "creative destruction" for which the IT industry itself is famous.

McAfee and Brynjolfsson have published their findings in a Wall Street Journal article that you can find here.

In a nutshell, here's what they found:

"Over the past dozen years . . . information-technology consumption is associated the kinds of competitive dynamics we're accustomed to seeing the IT-producing industries. And because every industry will become even more IT-intensive over the next decade, we expect competition to become even more Schumpeterian."

Joseph Schumpeter was a leading 20th century economist who described the "creative destruction" inherent in capitalist economies. It's the process of incessant revolution in which an existing economic structure is destroyed by a new one that arises within it. Think of QuickBooks' effect on manual ledgers and hiring scores of accountants: creative destruction. Think of mainframe computing giving way to client-server computing giving way to World Wide Web. Incessant revolution.

What McAfee and Brynjolfsson are saying, then, is that technology is becoming a disruptive force in a growing number of markets. The greater an industry's investment in IT, the greater that market's instability. They cite examples of how CVS and Harrah's Entertainment were able to increase profits through the strategic use of technology.

In addition to McAfee and Brynjolfsson's advice, which I encourage you to read in their article, I would add these thoughts:

First, if you're in an industry where businesses are increasing their investment in IT, change your thinking. Don't begin this year with last year's assumptions about the pace of change in your industry and the opportunities available to you. Consider the application of IT—everything from Web portals to mobile computing—in whatever SWOT analysis or other analysis you're performing. How might competitors use technology to improve their offering? How might you use technology to beat them to the punch?

Second, learn from IT thought leaders. IT leaders and strategists have been living with creative destruction and incessant revolution for years. It can't hurt for you to learn from their ideas, their successes, and their failures. Pay attention to how businesses, even businesses outside your industry, are gaining advantage in the marketplace by applying technology in creative ways. How can you change processes and innovate in the area of products and services? Get inspiration from the people who ask these questions every day.

Third, consider bringing strategic IT expertise into your company. Learning to live with incessant revolution is probably a cultural change for you and your company. You'll need some people living and breathing incessant revolution for your own changes and strategies to take effect.

Goldman Sachs: U.S. economy will slip to #3 by 2050

Just about a year ago, The Times of India reported that Goldman Sachs has revised its 2003 analysis of the BRICs nations.

"Productivity growth will help India sustain over 8% growth until 2020 and become the second largest economy in the world, ahead of the US, by 2050, Goldman Sachs has said, scaling up estimates of the country's prospects in its October 2003 research paper widely known as the BRICs report."

The nations with the largest economies would then be China, India, and the U.S. in that order.

I'll have more to say about this in an upcoming post. Meanwhile, the blog A Wide Angle View of India offers a summary of the updated Goldman Sachs report.

Getting the culture right

I've been corresponding with a friend of mine who works at a software company that's struggling to roll out new products. My friend used to work at a major customer in the company's marketplace, so he has lots of industry knowledge and insights. Having been a customer, he understands the customer's point of view. He's frustrated, though, because within the four walls where he works, it's the developer's mindset, rather than the customer's, that prevails. The company was founded by developers and is run by developers. Their strategic point-of-view tends to center on what they've built and why it's better than everything else, rather than what's happening in the marketplace, what customers need, and what business opportunities can be seized.

Which reminds me of this dismal truth: the wrong company culture can thwart just about any strategy or plan.

So now, at the beginning of the year, as you're making your resolutions, cleaning out last year's clutter, and preparing to move resolutely forward, add this to your list of new year's resolutions:

Examine your organization's culture and the ways its mind hinder or support your plans and pursuits. If you're a manager of any kind, examine your own habits and behaviors—your own culture, as it were—and ask how you could change what you do to facilitate the changes and activities you'd like to undertake.

If you're a manager, your walk matters as much or more than your talk. In addition to planning change, make sure you're behavior supports it.