Can Joyent Create a PaaS Platform More Cost-effective than Amazon EC2?

At last week's Enterprise 2.0 Conference in Boston, I had the opportunity to talk to James Duncan, Director of Platform Strategies at Joyent. Joyent is a cloud computing vendor in the IaaS (infrastructure-as-a-service) market. They sell virtualized servers called Joyent Acclerators, which customers use for running Web applications.

The idea behind IaaS is that customers can grow or shrink their infrastructure dynamically. Expecting a surge of business, or need to crunch vast amounts of data for a rush project? No problem. IaaS lets you instantly access as many servers as you need: 50, 500, even 5000 or more. Project over? Again, no problem. Shut down the servers at once. You've managed to meet your computing needs without buying, configuring, and managing racks of servers yourself.

In January, Joyent bought Reasonably Smart, a cloud computing start-up that James and his business partner Bryan Bogensberger had founded the previous summer. First on their own and now at Joyent, James and Bryan have been developing the Smart Platform, a PaaS solution that lets programmers develop server applications in JavaScript and then deploy these applications to the cloud using Git, an open source version-control system. Now branded as a Joyent solution, the Smart Platform is ready to go into a private Beta test.

I asked James why he chose JavaScript as the programming language for the new platform. Why choose a language famous for improving interactivity in a browser, instead of a language more commonly associated with server-side applications, such as Python or Perl or Java?

He pointed out that JavaScript was designed from the outset to deliver a lot of functionality in a secure, self-contained environment (known in programming as a "sandbox"). Secure autonomy is a great feature for applications and services running in a shared environment such as a data center, where virtual servers from several customers might be running on the same machine. He also pointed out that more money is going into training and documentation work on JavaScript than into other popular languages, such as Ruby, PHP, and Python. And, of course, JavaScript is a proven language, already popular with lots of programmers, who have been using it for years to develop sophisticated client-side Web applications. Now these programmers can apply their knowledge of JavaScript to the development of fast, efficient server applications that run in the cloud.

The Smart Platform offers a couple of other advantages that cloud computing users may find attractive.

First, the platform is open source. Joyent hopes that customers will like Joyent's PaaS services so much, they'll subscribe to Joyent and pay them to run the platform. But customers and even non-customers are free to move the entire stack in-house. Running the platform in house could be helpful for developing prototypes, running private clouds (shared, elastic services run entirely behind the firewall), or combining private and public clouds, as needed. Also, by making the platform open source, Joyent expects to ease customers' concerns about being locked in to a particular PaaS vendor. (Vendor lock-in has been much discussed lately in cloud computing circles.)

Another important advantage of the Smart Platform is that it's truly priced like a utility: customers pay only when an application is servicing requests. They pay nothing when is present on a virtual server, but inactive (that is, not processing data or interacting with other applications). This utility pricing model has the potential to make Joyent far more cost-effective than even Amazon's highly affordable Elastic Cloud Compute (EC2) service.

James offered this example. Let's say you're hosting your Web site in the cloud for a month. On Amazon, you would pay for the Amazon Machine Image (the Amazon virtual server) for the entire month, even if the Web site gets only a few hits and actually active for a few seconds or minutes.

On the Joyent Smart Platform, in contrast, you would pay only for those few seconds or minutes of activity; you would pay nothing for all the time the server is hosted on the Smart Platform inactive. As a result, your bill should be a small fraction of what it would be from Amazon.

Pricing based on activity rather than deployment could make a big difference for customers whose hosted applications are active only sporadically. It could also be attractive for customers who have forgotten to decommission servers once a project is over and have then been surprised to receive a big bill from a cloud service provider.

If you're interested in participating Beta of the Joyent Smart Platform, visit www.reasonablysmart.com.

What Cloud Computing Offers Startups, Part 2

In a previous post, I discussed the operational and financial benefits of cloud computing for start-ups. Today I'd like to discuss another benefit that's just as important, and that has far-reaching implications for the direction of IT development in the coming years.

Cloud computing makes it easier than ever for software companies to deliver innovative, business-critical services to Small and Medium-sized Businesses (SMBs).

Until recently, most software start-ups avoided the SMB market. Selling products and services to SMBs seemed daunting for several reasons:

• Limited Budgets: SMBs are well known for having spartan IT budgets. Beyond buying basic networking gear, Microsoft Office, and perhaps an accounting system, a small business may make hardly any IT investment at all. Even larger, mid-sized businesses tend to be conservative spenders, leery of risk and demanding a clear ROI from a new product—even though it's often difficult or impossible to demonstrate an ROI with a brand new product.
  
• Distribution Overhead: Reaching SMBs has traditionally required a channel (e.g., a distributor who served SMBs in a given area or industry) or, worse, many channels and lots of advertising. Channels require a lot of attention in the form of training and support, and they take a bit out of a start-up's profit margin.
  
• Deal Flow: To achieve sufficient revenue, a start-up would need to close tens or hundreds of small deals to equal the same amount of revenue possible from one or two large enterprise sales. With limited staff, attention, and marketing funds, most start-ups (with the hearty encouragement of their investors) have preferred to pursue opportunities in the enterprise market.
  

But in the past couple of years, cloud computing has knocked down all these barriers. In fact, cloud computing solutions often start with the premise that the customer has limited time and money for managing complex, but important IT operations. The cloud computing vendor rushes in as the SMB hero, managing everything behind the scenes, while offering the customer an easy-to-use, comforting Web interface.

Old Obstacle	Cloud Computing Solution
Limited Budget	Cloud computing solutions, such as SaaS applications or hosted storage, can be delivered cost effectively. There's no need for on-premises hardware and time-consuming installation and configuration services. Customers buy just what they need, when they need it. Delivery on popular platforms such as Force.com and QuickBase greatly reduces customer-acquisition costs, which normally the vendor would have to pass along to the customer.
Distribution Overhead	Cloud computing services are marketed, sold, and delivered over the Web. Customers can discover point solutions built on cloud platforms offered by vendors they already know and trust (e.g., Amazon, Intuit, Microsoft, Salesforce.com). There's no need for a large sales team and offices scattered around the country, nor is there a need to sign on and train large numbers of resellers. Sales and marketing take place online.
Deal Flow	Through promotion in established platform communities, as well as through viral marketing, blogging, and targeted marketing efforts, vendors can find tens and then hundreds or thousands of new customers. At a time when enterprises are cutting their IT budgets, reaching SMBs who are looking for cost-saving, operational improvements and strategic advantages offered through new capabilities, seems like an attractive idea.

Cloud computing has changed the nature of the typical start-up. Instead of a capital-intensive organization building "enterprise-class" solutions for large companies, today's start-up is more likely a small, nimble team, taking full advantage of the economies offered by platforms like EC2 and open source, and delivering online services that are valuable to companies of all sizes—even another five-person company down the hall.


Photo of man and clouds by donabelandewen, Creative Commons License, some rights reserved.

What Cloud Computing Offers Startups

At the Intuit's mini-conference on "Startups and the Cloud," the discussions, which varied from investment to technology, repeatedly raised the question of what advantages, if any, cloud computing offered startups.

To answer that question, we need to know what cloud computing is. I'm going to borrow the definition developed by NIST, which Longworth Venture Partners analyst Vishy Venugopalan cited in his overview of cloud computing, which kicked off the day's events:

Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, storage, servers, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

Five or ten years ago, it was not uncommon for a newly-funded start-up to spend tens or hundreds of thousands of dollars on infrastructure. Servers, disk arrays, back-up power supplies, management and monitoring software, an air-conditioned room with a raised floor—the list of capital expenditures could be impressive, and this investment needed to be made before engineers could begin any significant work developing products.

Cloud computing changes all that: it minimizes infrastructure investments, so that companies only need to invest in IT services when they're needed.

Cloud computing eliminates the need for that air-conditioned room filled with expensive server racks. It eliminates the need for the local IT manager to watch over them. And it shortens the management team's list of headaches, by sparing them the details of back-up tapes and server upgrades. IT can be provisioned cheaply and immediately—today, this afternoon, now, and development can begin right away.

Todd Hixon from New Atlantic Ventures put it this way:

Cloud computing gives you a sandbox for delivering solutions while assessing demand. It enables you to avoid needless investment in infrastructure.

Jeffrey Beir from North Bridge Venture Partners agreed:

Cloud computing allows developers to focus on the IP (intellectual property) that's unique to them.

After all, it's that unique IP that's ultimately going to make or break the company. It's the unique IP—not a rack of Dell servers in a computer room—that's going to be the quintessence of the start-up's brand, differentiating the company from the hundreds of other start-ups and thousands of larger companies already crowding the market.

By minimizing infrastructure costs and infrastructure management costs, cloud computing enables young companies to spend its precious capital on what's most essential. For young companies in this time of tight budgets and hard decisions, cloud computing is a financial and operational boon.

The Six Traits of a Fundable Entrepreneur

Yesterday afternoon in Waltham, Massachusetts, Intuit hosted on a mini-conference about the opportunities that cloud-computing offers start-ups. "Startups and the Cloud" proved to be an engaging event, packing a small auditorium on the campus of Bentley College. Sessions ranged from Longworth Venture Partners' Vishy Venugopalan's overview of the cloud computing marketplace to an open Q&A with Intuit founder Scott Cook to panel discussions with venture capitalists and CEOs about cloud computing and the changing economic environment for young companies.

The venture capitalist panel featured a seasoned team of investors:

• Jeff Bussgang from Flybridge Venture Partners
  
• Jeffrey Beir from North Bridge Venture Partners
  
• Todd Hixon from New Atlantic Ventures
  
• Shawn Broderick from
  TechStars
  

This group offered much wit and wisdom on a variety of topics. For now, I'll simply offer their composite profile of a fundable entrepreneur.

In response to a question from the audience, the panel offered this list of traits that they look for in an entrepreneur:

• Passion
  
• Domain expertise (the entrepreneur knows his or her area thoroughly, and isn't just coming up with an idea in response to a news story, for example)
  
• Unfair advantage (something that gives this team of entrepreneurs a sustainable head-start in the market)
  
• Street smarts (knowing how to get things done in difficult situations)
  
• A whip-smart mind
  
• Salesmanship
  

And Shawn Broderick offered this observation, which deserves to be mounted on a plaque over every entrepreneur's desk:

"Execution is insanely important."

Flying Blind with Regard to Application Performance

By now, these trends are pretty obvious:

• Organizations are becoming increasingly distributed, spanning remote offices, strategic partners, outsourced workforces, and ad hoc teams in order to take advantage of the most talent for the lowest expense
  
• Organizations are cutting headcount and trimming operational budgets, making employee productivity more important than ever before
  
• Networked applications are involved in nearly every aspect of business operations, including product design, production, sales, marketing, accounting, logistics, and customer service
  

It would seem prudent, then, for organizations to monitor the performance of their applications, since networked applications are 1) the tools most workers are using, and 2) at risk of reducing worker productivity through performance problems and application outages.

But many organizations don't systematically monitor application performance at all. This jarring revelation appears in a study about cloud computing just published by InformationWeek.

The study noted that 40% of respondents didn't have a system in place to monitor internal applications, let alone cloud applications. An integrator interviewed in the study remarked that fewer than 30% of his customers had application monitoring systems in place; in other words, more than 70% didn't monitor applications.

Given the rising popularity of video and voice applications, which require high-performance, low-latency network connections, and the rising popularity of cloud computing—in use or about to be in use at 27% of the organizations surveyed—the lack of application monitoring seems like trouble in the offing.

The InformationWeek article offers a number of helpful suggestions, including the use of WAN optimization for accelerating applications serving remote offices. Application performance monitoring solutions from companies such as Blue Coat, Fluke Networks, NetScout, and WildPackets can also be helpful. A new standard called Apdex, which attempts to measure the quality of service an application delivers, is gaining a following and also worth a look.

Disclosure: Blue Coat is a client.

SaaS Credibility Survey

As you probably know, many businesses are wary of Software-as-a-Service (SaaS) applications because of concerns about security, reliability, and vendor lock-in.

In addition to building the absolutely best IT infrastructure possible, what can SaaS vendors do to assuage end user fears?

Take this short survey (2-3 minutes) and let your voice be heard.

Data Security for SaaS, PaaS, and Social Media

One of the most common objections to cloud computing is that cloud computing poses too great a risk for data security. Internal data that is being stored safely in an internal data center may be subject to interception in transit to or from a remote application. It might also be vulnerable when stored in the cloud itself.

Cloud vendors such as Amazon, Google, and Salesforce are going out of their way to demonstrate tight security controls to major clients. Nonetheless, a lot of CIOs, CSOs, and others have their doubts.

It's worth pointing out that, whether they realize it or not, most enterprises are leaking—nay, hemorrhaging—data to the public Internet. As I wrote nearly a year ago, summarizing some fine reporting in InformationWeek, P2P applications alone are responsible for massive data leaks even at large, public companies.

When InformationWeek reporters investigated P2P networks to find out just how much confidential data was being accidentally leaked by P2P networks, they were shocked at what they found. Users were inadvertently publishing "spreadsheets, billing data, health records, RFPs, internal audits, product specs, and meeting notes . . . files with the home and cell phone numbers of senators, confidential meeting notes, and fund-raising plans [for a state political party] . . . spreadsheets listing patients' names along with their HIV and hepatitis status . . . [and] a slew of court documents regarding a sticky divorce."

Fortunately, there's technology available to detect and thwart such leaks. Data leak protection (DLP) products, often available as network appliances, can scan data leaving the network and raise an alarm about confidential data leaking out. A lot of companies have jumped into this market; a few years ago, no less than 46 different start-ups were tackling this problem. A few companies have emerged as leaders. You can learn more about DLP at this informational site: www.dlpindepth.org.

Certainly it makes sense for any medium or large enterprise to have a DLP solution in place. Once it's in place, it should provide effective monitoring and control over data posted to the cloud.

DLP doesn't address the problem of security vulnerabilities in cloud storage, but it does address vulnerabilities in cloud communications, and it also enables enterprises to ensure they know what data is being posted to the cloud in the first place, regardless of whether the destination is Salesforce.com, AWS, Facebook, or some other app.

Who's Using PaaS? The Answer May Surprise You

When Platform-as-a-Service (PaaS) vendor Coghead hit the skids recently, other PaaS vendors such as Intuit QuickBase and TeamDesk were quick to offer Coghead customers free conversion tools and migration solutions, so they could keep their applications running smoothly in the cloud.

I was curious about the customers making this move. Had they been they using Coghead for in-house skunk works, pilot projects, or operational applications? In large companies, did management concerns about data security, SOX compliance, and other regulations relegate PaaS to department-level projects that were, shall we say, off the management radar screen? Is PaaS (when not an extension of a proven SaaS solution such as Salesforce.com or Sugar On Demand) simply a way of getting code up and running without having to requisition a server from a bureaucratic IT department? Or is PaaS something more?

To find out, I emailed TeamDesk, asking about their customers and the types of applications they were running. I promptly received a phone call from Val Karmazin, co-founder of ForeSoft Corporation, the company that offers TeamDesk. Promptness, it turns out, is a habit at ForeSoft. The company prides itself on prompt, reliable customer service.

ForeSoft offers four cloud computing solutions:

• BUGtracka project management and issue-tracking application
  
• dbFLEX, a platform for building business Web applications
  
• CRMdesk, a help desk application
  
• TeamDesk, a platform for building and easily configuring database applications, primarily for the back office
  

The company was founded in 2001. It's been profitable from the start and hasn't taken any outside investment. TeamDesk, launched three years ago, is now the fastest growing part of the business.

In our conversation, Val made a number of interesting points:

• The TeamDesk user community is a mix of people, and most of them aren't developers. What? I thought PaaS would appeal mostly to developers who were comfortable with Ruby, Python, Java, virtualization, and so on. That's not so in the case of TeamDesk. The platform is so easy to use and so easy to configure, thanks to a configuration dashboard, that many of the users are small business owners or IT engineers who know enough about databases to establish a relationship between two tables, but not much technical knowledge beyond that.
  
• Companies of all sizes are using the service. In addition to small businesses, Val named a major telecommunications carrier, a major shipping company, and a European office of a major Silicon Valley technology provider. Company size doesn't predict whether or not a customer will use PaaS; instead, the determining factor is company culture—how willing is the company to trust a PaaS vendor to do things right. Val says customers take advantage of TeamDesk's free trial, discover how easy to use and reliable the service is, and stick with it.
  
• Val points out the distinction between PaaS and SaaS is often illusory. When his customers use his platform to develop business applications that they then rely on day after day, haven't they created in effect a SaaS solution? Read TeamDesk's case study about the sales management application that Hochkoeppler Initiatives created for a customer, and you'll see what he means.
  
• TeamDesk users sometimes require a little more hands-on assistance from ForeSoft when they're getting started with the service, but so far the support workload remains manageable.
  

A couple of conclusions:

• PaaS is ready for prime time: customers are using it for business applications, not just for development projects.
  
• With the proper tools and application user interface, a PaaS vendor can reach beyond the development community to less technical users, broadening the pool of potential customers.
  
• Small, focused cloud computing vendors who execute well can run a profitable business.
  

Cloud Computing Predictions, Revisted

All right, I'm going to strike the tentative tone that had crept into one of my predictions for cloud computing in 2009. When I read about the success of Marketo, Central Desktop, and others SaaS vendors, I can more evidence of young SaaS vendors racking up impressive sales by solving important business problems. So I'm revising prediction #4 to read:

Small vendors who apply their domain expertise to bring the power and convenience of cloud computing to business areas underserved by IT, can gain market traction and growing a profitable business by delivering exceptional operational business value to customers.

I still believe that, as one commenter to this blog put it, "PaaS is a tough nut to crack." But SaaS offerings that are focused and immediately useful should have a very good 2009. Which, of course, is wonderful news.

Two SaaS Companies that Solve Business Problems for Customers

In a couple of recent blog posts (here and here), I raised the question of what type of cloud computing start-up would be likely to succeed in today's business environment, in which companies of all sizes are interested in cutting costs and minimizing risks. I suggested that business customers would feel comfortable with new programming paradigms (e.g., Salesforce.com's Apex) offered by large, stable companies, but shy away from similar offerings from smaller vendors. It's not that the smaller vendors won't get any customers; they just might have a hard time getting enough to stay in business.

Cloud Computing Opportunities for Start-ups and Other Small Companies

But, aside from infrastructure offerings, there are lots of great business opportunities for small cloud computing vendors. I believe that most of these opportunities share these traits:

business domain expertise + effective execution + cloud technology

Being small and new won't be problems (i.e., risk factors in the eyes of customers) for small vendors who demonstrate that:

• they thoroughly understand business process problems that are important to the customer, and they are dedicated to solving these problems
  
• they have a solution that directly addresses these problems in an immediately effective way
  

Getting Down to Business

Here are two software start-ups that offer examples of what I mean.

Compli is a SaaS company based in Portland, OR, that offers a software platform that enables car dealers to measure and manage their compliance with industry regulations. Through the Compli SaaS platform, car dealers can deliver compliance training to employees and employee test scores on compliance tests. As regulations evolve, and new state-specific regulations appear, dealers can distribute new compliance content to the appropriate employees and demonstrate "good faith" efforts at compliance.

Compliance is an important issue for car-dealers, an operational head-ache of sorts, and Compli's SaaS solution gives them an easy way to stay on top of the issue in a cost-effective way.

If you visit the Compli Web site, you'll have to hunt hard to find any references to SaaS and cloud computing. The words "SaaS" and "cloud" don't appear on the home page at all, and in the video featured on the home page, President and CFO Lon Leneve mentions SaaS only after discussing the scope and importance of compliance for car dealers. The company is focused on solving a business problem, and they're leveraging SaaS technology to do it. How's Compli doing? Last year was a record year.

I've written about Liquid Planner before, and I'll be writing more about them next week. LiquidPlanner offers a hosted project management solution that brings probabilistic analysis to project planning. In another words, while other programs like Microsoft Project force project planners to give a fixed estimate for how long a task will take, Liquid Planner lets planner input ranges and probabilities, so they identify risks up front. The result is planning software that's more detailed, more accurate, and more informative.

The solution includes other collaboration features, as well, but I'd like to point out that once again we have a SaaS company focused on solving an important business problem—project management—in a new and compelling way. Factoring probability into project planning makes so much sense, I think LiquidPlanner would be an attractive offering even in a traditional, in-house deployment; delivering LiquidPlanner as SaaS, so it can reach all members of a distributed project team while lowering hardware and software costs, only makes it more compelling.

And how's LiquidPlanner doing? Quite well. Business is growing. The company itself is a small, lean-and-mean team of ten people whose founders have extensive experience in data center management from Expedia. Steve McConnell, who has written authoritatively on rapid software development and software estimation, is an advisor to the company.

These two companies, Compli and LiquidPlanner, demonstrate the business opportunities available for SaaS start-ups. Both companies are focused on solving important business problems (regulatory compliance and project management) in new ways. The problems they're addressing will remain important to customers even in an economic downturn. The companies are leveraging SaaS to deliver their solutions broadly and cost-effectively. Both companies share share these characteristics:

business domain expertise + effective execution + cloud technology

Disclaimer: Neither Compli nor LiquidPlanner is a client.

Predictions for Cloud Computing in 2009

Short and sweet.

1. The cloud computing market will grow in 2009.
   
   
2. Most of that growth will be enjoyed by a small number of large vendors (e.g., Amazon, Salesforce.com) with solid reputations for technical prowess, reliable service, and financial stability. (The financial stability part is not to be underestimated; it will win over CFOs and CIOs.)
   
   
3. Despite all the buzz at conferences and the cheery encomiums exchanged in blogs, new entrants and small vendors—especially vendors who try to replicate or enhance in a minor way the offers of the major vendors—will have a tough time closing deals and generating cash.
   
   
4. Small vendors who pursue niche markets and apply their domain expertise to bring the power and convenience of cloud computing to business areas underserved by IT, stand a chance of gaining market traction and growing a small, profitable business by delivering operational business value to customers.
   

The success of this last group will depend as much on their domain expertise (e.g., about problems with insurance claims processing) and consulting skills as on their particular cloud infrastructure.

Comments welcome.

Coghead's Demise is a Reminder to Sober Up about Cloud Computing's Promise

I was sorry to learn that Coghead, a Web 2.0 Platform-as-a-Service (PaaS) provider, announced it was shutting down.

PaaS is an interesting model: a company offers a hosted service for developing, testing, running, and monitoring new applications. Customers can use the PaaS platform to launch new applications—or scale up existing ones—without deploying any local hardware or software at all. Sounds intriguing. Possibly very convenient. Possibly cost-effective.

It's proving to be too futuristic a vision. First, Bungee Labs, another PaaS provider, ran into trouble in 2008. Now Coghead is shutting its doors.

You wouldn't think this would be possible to read all the hyperventilating blog posts about cloud computing: how the time is right for cloud computing, everything will run in the cloud, what is the cloud—do we include PaaS? SaaS? If we don't define cloud computing properly, the cloud will perish! Many industry insiders are in a lather about these issues. It's as though the lottery has come up with the winning letters, which spell cloud computing, and now we have to scratch off the bonus letters just right to multiply our winnings or be sent home with a PC, jr. Nerves are a-jangle. Fingers are flying.

But if cloud computing is such an obvious remedy to the IT woes of business, why are these vendors in trouble?

Let's come down to earth for a moment. Let's consider this situation from the business customer's point of view. The business manager asks, Do I need PaaS? Is this really the most convenient, least risky way of building and deploying new applications?

I don't want to sound pedantic, but I think many cloud computing services will have trouble overcoming the obstacle that's snared many other technically impressive solutions in other IT markets of yore: just because you can build it and it's cool, doesn't mean that business customers will be comfortable with it. How many business customers in this increasingly risk-adverse environment are really going to adopt new programming paradigms and hosted services from small, evidently risky providers? Not many, I'm afraid.

Businesses want low cost and convenience. They want reliability and low risk, just as much. And (almost) nobody in business likes to learn anything new unless they have to.

I do think there are attractive opportunities for cloud computing in 2009. I'll details those in my new post. (Don't worry. It will be short.)

Note: Thanks to @chris_marino for tipping me off to the sad news about Coghead.

Not All Data Integration Connectors Are Alike

Connectors are a vital part of any data integration solution. No matter what data sources you're integrating—databases, applications, flat files, Web services, etc.—it's awfully handy to have a preconfigured connector or at least a template to minimize the amount of hand-coding required to move data out of or into a particular data source.

The importance of connectors is perfectly clear to customers. In news stories, such SaaS Integration: Real-World Problems, And How CIOs Are Solving Them, which appeared in InformationWeek in October, customers are blunt about their expectations regarding connectors: vendors need to have a lot of them, one for every piece of middleware being integrated, and vendors better know how to make them work.

[H.B. Fuller CIO Steven] John is asking SaaS vendors lots of questions related to middleware, such as whether they have developed plug-ins for a specific middleware package and whether they have direct experience implementing that middleware. "If they say no to either, it's a strike against them," he says.

Recognizing the importance of connectors to prospects, most integration vendors parade their list of connectors on their Web sites.

And certainly, if you walk the tradeshow floor at events like the O'Reilly Web 2.0 conference or the Enterprise 2.0 Conference in Boston, you'll find vendors rattling off the names of the connectors they have.

"SAP? Oh, yeah. We've got a connector for that."

I've written before about the misleading simplicity of this approach. Data connectors aren't like Converse sneakers. You can't simply amass a bunch of them (red, orange, purple, black), and assume you have what you need for every occasion.

Different data sources have different security and access requirements. Applications integrating with protected data need to ensure that the data remains protected. You certainly don't want to bypass all the security and access controls protecting, say, a SAP ERP system, simply so that social platform users can pull ERP data into their wiki pages. These requirements become even more pressing when you're dealing with data in the cloud, where it's outside the perimeter of an internally secured and controlled data center.

Another integration requirement, above mere "connectivity," is transformation. Data might need to be transformed ("groomed") or narrowed before being presented to a group. For example, if I'm pulling in sales numbers from the Tokyo office, I'd probably like to see the amounts in yen converted to dollars. It would be nice to have the integration solution do this, so that I know I'm using a tested conversion tool that the company officially endorses.

Beyond merely connecting, then, connectors may need to work as part of an integration solution that supports access controls, transformation, auditability, and orchestration (e.g., before providing data set X, ensure than operation Y is complete, so that X is valid and up-to-date).

The NetSuite Example

The other fallacy of the "Converse sneaker" approach to connectors is it assumes that all integration endpoints and APIs are more or less alike. There's a DB interface or a middleware API. You write to it. You're done.

Not always.

Some APIs are more interactive and complex.

NetSuite, the hosted provider of mid-market CRM, ERP, and accounting solutions, now has 6,600 active customers. So lots of people have reason to connect to NetSuite data.

Integrating with NetSuite, however, is more involved than integrating with other SaaS applications. Why? Because NetSuite provides a great deal of flexibility in customizing their basic record schema and in defining custom records. You can query meta data to discover some, but not all, aspects of the customization. So a connector cannot rely on an automated process for discovering how an account has been customized and access its data.

Ideally, a connector should mask as much of this complexity as possible from the IT user creating the integration, so a good NetSuite connector will take advantage of the flexibility of the NetSuite API, while hiding as much of its complexity as possible from the user building or using the integration.

SnapLogic, an open source data integration company, offers a NetSuite connector that automates as much of this discovery as possible, while supporting integration that works with custom NetSuite records. Here's an explanation from the SnapLogic documentation pages:

NetSuite allows customization of its schema by allowing users to define custom record types and by adding custom fields to existing records. This extension package can automatically discover all the custom record types in a NetSuite account at install time. However, NetSuite does not provide interfaces which allow the extension package to discover custom fields have been added to all existing record types. The extension package is able to do this discovery for some kinds of records (like entity records, item records and CRM records), but not all. For this reason, a manual approach has been provided for specifying the custom fields of records. The user can use the NetSuite UI to browse the records and find custom fields that are of interest. The user can then use the utility: netsuite/resources/customize_resources.py (provided by the extensions package) to manually add custom fields to the SnapLogic Resources that represent a given NetSuite record types.

The connector automatically discovers of the accounts data schema as possible, then supports one-time additions for custom data. Once this set-up work is done, the user has a collection of ready-to-use, snap-together building blocks for building integration pipelines.

Because the connection reads the NetSuites schema, generates components, and supports customizations, it's able to provide NetSuite customers with a flexible solution for integrating NetSuite with other applications and data sources.

A more perfunctory connector would look just as good in a check list of available connectors, but it wouldn't serve users nearly as well.

Zoho CloudSQL: An interview with Rodrigo Vaca

Earlier this week, Zoho announced CloudSQL, a new SQL interface to Zoho Reports, its popular Web application for online reporting and business intelligence. Zoho applications (in case you haven't heard of them) are credible alternatives to Google Software-as-a-Service (SaaS) applications such as Google Docs. Launched three years ago, the suite of Zoho applications has grown dramatically in number of applications, richness of features, and size of its user base. The company now boasts over 1 million users for its 19 applications. More applications are on the way.


Here's how Rodrigo Vaca, Zoho's Director of Marketing, described CloudSQL in a blog post earlier this week:

Zoho CloudSQL is a middleware technology that allows customers to interact with their business data stored in Zoho through the familiar SQL language. Customers are able to access Zoho cloud data using SQL on both other cloud applications as well as through traditional on-premises software.

At a high-level, Zoho CloudSQL serves as the bridge between the external application and the data stored inside Zoho. It receives the query in SQL, interprets it, delegates queries and aggregates results across the Zoho services.

There are in particular 3 things that stand out about Zoho CloudSQL:

• It's the first technology that allows customers to interact with their data on the cloud, from another cloud application or from an on-premises one through real SQL.


• It supports multiple SQL dialects. We support all the major (and even some not so major) ones: ANSI, Oracle, SQL Server, IBM DB2, MySQL, PostgreSQL and Informix.


• With our JDBC/ODBC drivers, developers can access data in the cloud just as easily as if it were stored in a local database.
  

A Quick Interview
I got in touch with Rodrigo Vaca to ask him a few follow-up questions.

JB: From your announcement, I'm gathering that CloudSQL is a SQL-based service for accessing data in Zoho applications. The interface will be of interest to engineers working on integration projects where they would like to simply work with SQL queries, rather than dealing with JSON or RESTful data access. Is this an accurate characterization?

RV: Yes, that's accurate. Zoho CloudSQL is about making the data in the Zoho cloud more accessible for our customers. SQL is something that most corporate developers know and are familiar with.


JB: The diagram on your December 2 blog post shows CloudSQL being able to access other Web services. Are there non-Zoho Web services you plan to support? Say, any Web services from StrikeIron, ProgrammableWeb, or even Google, etc.?

RV: Ah! You were paying attention! You noticed something that most other people missed. Yes, Zoho CloudSQL can be extended to non-Zoho services. At this point we're not focused or actively pursing that, since we need to first make sure that other Zoho services are accessible through CloudSQL first.

JB: Finally, I was intrigued to see that you're doing entity-mapping, which makes sense. It makes me think of the work Microsoft has been doing in its Project Astoria group (creating a framework now called ADO.NET), where they're using entity mapping to present a non-SQL-based interface to SQL-Server data. Do your RESTful APIs make use of this entity mapping? Does Zoho have plans to publish an Astoria-like interface to Zoho data?

RV: Our REST API should provide all the necessary details for developers, so we don't have plans for entity-mapping like Astoria. We would recommend CloudSQL, as the standard interface for developers, especially as we increase its coverage across Zoho applications.

To learn more about CloudSQL, visit this Zoho wiki page here.