Then it hit me.  Those words repeated in my head over and over again.  A. Portal. To. Commerce.  When I was running IMSafer, we spent quite a bit of time doing SEO and ad buys.  I always had a great deal of wonder for the many ins and outs of the various search engines, and the many people who claim to have cracked that nut.  It never ceases to amaze me how when I search for either SEO or the name of an SEO firm that wanted to win my business that the firm in question did not appear as the number on link.  That’s a story for another time.

As part of their method for determining ad word link order, Google has a metric which they call their quality score.  You see, it used to be that the only thing that mattered for your ad placement was the amount that you were willing to pay and the click through rate of your ad.  This actually rewarded people who wrote good ads in such way as to move their ad to the top of the page, and not require that they be the highest bidder.  As is the case with any technology business, things seldom stand still for very long, and the quality score metric has evolved.

The one bit that stands out, more than anything else, is that GOOG is in the business or arbitrating the commerce on the Internet.  With a very opaque system which exists, in their words, “to organize the world’s information and make it universally accessible and useful,” I find it very interesting that GOOG is the one who gets to determine the usefulness, and, more specifically, how accessible, any specific piece of information is.  As a business, if you are not on the front page of GOOG search results, you might as well not be in business.

There are several factors affecting quality score which are largely in the control of a business.  They can write great ad copy, and get great click through rates.  However, the addition of the landing page relevancy to the metric swings things dramatically in GOOG’s favor, with no recourse for the advertising business.  I am not the first to trod on this ground, as this post clearly outlines many of the issues calling for transparency from such a large force in the market.  Alan Rimm-Kaufman makes some great points, but the discussion left me feeling incomplete.

The connection I have not seen in any of these discussions, however, is that coupling this debate back to the topic of net neutrality.  GOOG has taken a very strong stance on the notion of net neutrality.  Again, in their own words:

Network neutrality is the principle that Internet users should be in control of what content they view and what applications they use on the Internet. The Internet has operated according to this neutrality principle since its earliest days. Indeed, it is this neutrality that has allowed many companies, including Google, to launch, grow, and innovate. Fundamentally, net neutrality is about equal access to the Internet.

Let me repeat that last point again.  Fundamentally, net neutrality is about equal access to the Internet.  To hear GOOG tell the story, it shouldn’t matter how much you are willing to spend on ads.  If their bot determines that you have a better landing page, then they can get you more traffic than someone who is willing to pay more for the placement.  Traditional media placement actually works the other way - you pay, you get the placement.  Demand determines price.  GOOG will no doubt claim that they are democratizing the web, because they are making millions of customers available to those who can make for a better customer experience.  GOOG is the arbiter of that experience, however, since they have little, if any, visibility into the conversion rates for traffic they send a business.  They have lexical analysis.  Businesses have hard numbers.  This is no doubt why they make Analytics available free of charge - a business’s traffic and conversion data feeds the GOOG beast, and gives them more data upon which to charge the business what they believe to be the appropriate price for a set of ads.

There’s the rub.  GOOG gets to charge you whatever they want for ad placement.  It doesn’t matter that the top link spot is going for $1 CPC.  If you are willing to pay $10, and have no prior historical data with GOOG about traffic or ads, they can decide, through the opaque process of their quality score and the fallback excuse that your landing page didn’t pass muster, to charge you $100 for top placement, or not let you advertise for that ad word at all.

GOOG doesn’t want the network providers to create a tiered Internet based on who they decide should be on whatever tiers they set up, but GOOG are in fact doing that very thing.  If GOOG can arbitrarily decide who gets a link off their search result page, they are the de facto arbiters of commerce.  It’s as if a mall operator showed up to a store the day after they opened, walked in, looked at their layout, and said “you now have to pay 10x the rent because I don’t think people will like coming in here.”  Further compounding this issue is the lack of clarity in the algorithms that GOOG uses.  There is no playbook by which a business can refer that enables them to know with certainty that if they do X, they will get Y result.

Which brings me to the curious case of “Let’s Make a Deal.”  Business owners don’t know if they should take what they have and understand (traditional advertising), or what’s in the box (GOOG ad words).  The box can be mightily seductive, but it can also bring ruin.  Play your cards right, and you get great rewards.  Play them wrong, and you go home with nothing.  GOOG now has close to 70% share in the search market, making them an undisputed monopoly, and, more abstractly, the new version of Monty Hall.  I’m not a huge fan of the government taking on the task of deciding who is and who is not a monopoly, having myself been deposed in the Microsoft anti-trust case.  I believe that customers and users will dictate where the market goes.  With that said, this is an angle on the story which I have not seen anywhere, and I hope more people start asking the hard questions around GOOG’s role as a portal to commerce, and their hypocritical role as the white knight for net neutrality.

UPDATE

Not two seconds after I posted this, I find this article, courtesy Hacker News.  The article lays out, pretty clearly, that GOOG is even more hypocritical than I had thought.  They are approaching the network providers with a plan to allow GOOG traffic to operate on a higher tier of business.  Incredible.

Additionsl coverage can be found at RWW, and GigaOM.

What I don’t quite get is what they really believe they are accomplishing.  Connecting up these services is good, in the context of plenty of users on either end asking for this sort of thing.  As I have written about before, the AppEngine framework really drives developers down the path of building specific types of applications.  Those applications, by and large, are very consumer focused in nature.  The connection between a SFDC prospect (or customer) and a user of a GAE application is one that I don’t make.  There simply isn’t a center of gravity there.

Let me put this another way.  California has a reasonably size deployment of refineries.  That would be SFDC.  So they decide to hook up with a petroleum company (GOOG) to build a pipeline somewhere.  They decide, for whatever reason, to build it to the state of Washington (AppEngine).  Hey, it makes sense.  Let’s connect up refining capacity to a market.  Unfortunately, according to the Department of Energy, WA has “few fossil fuel resources.”  I’m not trying to be pejorative or anything, but does GAE really have a robust ecosystem of applications, and, more to the point, a future large potential market of applications, to support such this effort?  Given the assets at the disposal of GOOG and the potential for cross-sell with SFDC, I am surprised that this was the path they went down.

It seems that they sometimes are looking for the one or two things that they can do that will guarantee, or at least dramatically improve, their probability of success.  While I don’t like to call things mistakes, but rather “opportunities for excellence,” I also try to look at things in as brutally an honest way as possible.  As such, in the true spirit of a postmortem, I try to focus on the negative.  Make a mistake once, and you are OK.  Make a mistake twice, and you are un-fundable.

Since I am now close to a full year out from having sold IMSafer, and have in fact been re-assimilated to Microsoft, I have had more than enough time to think about this topic.  There are many lessons, and I am happy to share those over a beer with whomever wants to listen.  However, there are two major mistakes that I made with IMSafer.

First, never, ever, take money from professional investors for whom this would not be a professional investment.  Most of my funding came from old friends and bosses who work in VC, private equity and hedge funds.  For them, they were investing in me, and not necessarily the company.  They aren’t angel investors by trade, though some of them make angel investments from time to time.  Unfortunately, even if they aren’t making the investment as a professional investment (i.e. part of their current investment vehicle), they will still expect to treat you like one of their portfolio companies.  If you consider the tax that this puts on a startup, especially a seed stage startup, you can quickly find yourself overwhelmed with what I affectionately call seagull management.  They will come in, make a lot of noise, crap all over everything, and leave.  It’s a very hard place to find yourself, especially when you know the most important thing that you need to do is execute, not managing a board and investors commensurate with a later stage investment.

This problem further manifested itself in the security that we ultimately closed on.  We spent a little more than 10% of the round paying lawyers for the complicated, ridiculous, onerous security that the investors put on us.  This is my own fault for allowing this to happen, but as I found out the hard way, things get rough when you need the money.  The closing documents were absurdly heavy, and we were required to go with a named Valley firm in order for the investors to feel comfortable the that deal gets done right.  When you hear “named Valley firm,” what you ought to hear is “expensive.”

Second, and this is a big one, make sure that your thinking and style, for the most part, line up with whomever is writing the biggest check.  If you take money from people, they are going to expect to have a voice in what you are doing.  This is even more the case the bigger the checks get.  When you have investors with strong opinions, things can get dicey.  When your single largest investor, who is most likely going to be on your board, has a very particular way of doing things, and very strong opinions, you are likely to find yourself at the losing end of decisions.  Why is this?  First of all, there is the success factor.  If that investor has had it (which is likely the case if they are giving you large sums of money), they tend to believe that they are right, as do others.  Further, the other investors, for their part, are going to want to align with each other.  One strong willed investor, with a lot of money on the table, can have very persuasive sway over the other investors.

I know in my heart that one of our critical errors as a company (a post for another time) can absolutely be chalked up to this dynamic, and my inability to stick to my gut and disagree with the investors cost us dearly.  Hindsight is always 20/20, but I knew better, and I allowed myself to be blinded by his success aura and let my self doubt dominate my thinking. 

Amazon

Let me start by giving a hat tip to the AMZN guys.  Their web services platform (AWS) has really been at the tip of the spear for cloud computing.  For the sake of this discussion, when I use the term “cloud computing,” I am talking about the developer platform, not things like GOOG apps.  The AMZN services are a loosely coupled set of services targeted at developers looking to avail themselves of infrastructure buildout.  Their infrastructure as a service (EC2) is a great way for developers to reduce their capital expenditures and take on a variable costing model for their servers.

The services, all up, encompass several key pieces for building applications that can take on a range of workloads.  The ability to now run Windows Server 2003 (which should get you terminal services) means that as a developer, you can deploy varying application types into AWS, beyond simply web apps.  From a developer standpoint, however, there is no unified development experience.  Which brings me to the discussion around motivations.

AMZN essentially spends 10 months out of the year building out servers.  They build out massive numbers of servers for the two months out of the year when they are handling the Christmas shopping season.  AMZN is, for all intents and purposes, a giant mall.  They spend hundreds of millions of dollars a year building more capacity for their mall.  Their store is the primary reason shoppers come in the front door, and most of the year, their mall is empty, relative to the size of the mall.  As a retailer with an ecommerce backend, their problems are no different than most enterprises.  They have machines that are sitting around doing little or nothing for large portions of the year.  Unlike most other enterprises, AMZN has the financial capacity to build out their server capacity.  At some point, AMZN made the decision to no longer be the primary traffic draw to their mall.  They realized that they could completely offset their own traffic (and then some) by taking the technology that they had built for themselves and making it available to other developers.

AMZN is, essentially, in the load management business.  They are a low margin retail operator that is running a hugely expensive infrastructure for which they are seeking maximum utilization.  They would like nothing more than to be noise in their own system.  AMZN is relentlessly metrics driven.  As such, they have a pretty good idea of how much money to expect off of traffic that walks through their front door.  They know how much to expect from traffic ending up at one of their marketplace partners.  With the addition of AWS, they have a new way to monetize their capacity, and with their predictable pricing model, they know exactly how much money they are going to make off of customers who deploy applications to their service.  Traffic on their network makes them money.  It may not make your app money, but it makes them money, so they are happy.  It more than likely saves you money, so you are probably happy too.

So, AMZN is motivated by maximization of infrastructure capacity, and optimization of what would otherwise be very low margin businesses.  They do this by delivering a loosely coupled set of services to developers looking for infrastructure as a service.  They aren’t necessarily targeting developers.  They aren’t a platform, per se.  They are targeting anyone who is looking to move workloads off of their own infrastructure and onto AMZN.

Google

Ahh, Google.  I love those guys.  They have done plenty of cool things, and their release of the AppEngine (GAE) was welcomed with very loud praise from many in the tech community.  According to some reports, they had 25,000 developers sign up for GAE in the first few hours of releasing it.  Taken as a whole, GAE is a very tightly coupled set of services designed to allow developers to build applications that will have no problem scaling up to, theoretically, infinite capacity.  The rigid requirements for a developer deploying into the GAE are such that you use one language (Python) and their database (big table).

Once again, when considering what has been built, one need look no further than the motivations of GOOG to really get where they are going.  When you consider the stricture imposed on the architecture of apps built for GAE (singple process, no long running queries, no local file access, no network access), a developer is all but required to build standard CRUD web applications.  There is no stack for enterprise integration.  The delivery vehicle is the web browser.  The data that is created goes right into big table.

Some might say that GOOG’s core business is search.  I actually have a different opinion.  GOOG’s core business is the monetization of page views.  Search is their instantiation of that business model, but many of their other applications have nothing to do with search, and everything to do with the monetization of page views.  Think about gmail - ads on the sidebar.

Applicaitons on GAE are mostly CRUD apps, storing structured data into big table.  As a developer, building an applicaiton on GAE, you are essentially feeding the GOOG beast.  While they have not yet released final pricing, allow me to put on my pointy tin foil hat and talk about what might come to pass.  GOOG knows exactly how much it costs to run their infrastructure, and as such could hand developers a bill for the resources which they consume.  However, GOOG doesn’t have AMZN’s problem.  Their traffic is mostly linear, and going up and to the right.  It’s probably logarithmic at this point, but who’s counting?  In any event, since they have little variability in their traffic patterns, they don’t have to get into the load management business.  By allowing developers to build applications on their infrastructure, they are incurring unnecessary costs.  Their motivations, however, are driven by their business model.  Each new app that is plugged into the infrastructure ads new data to their data set, and creates new opportunities for page monetization.

Think of the case where GOOG shows up to a developer and says, “hey, moboganda.com - how are you doing?  That’s a great app you put up on GAE.  We’ve been going over the data and we figured out that if we ran AdSense on your pages 20% of the time, we could actually recoup the cost of running the infrastructure for you.  In fact, you don’t even need to do anything, since we own the infrastructure, the feature is already in there.  You can turn it off with a JavaScript include.”  As the young CEO of a web startup, this could sound like a pretty cool deal.  Free infrastructure and they didn’t have to do anything to get that turned on.  Not bad.  Let’s take this game of what-if one step further.  The GOOG guys come back in two months with the following, “hey buddy, that app of yours is performing really well.  You made more money on those 20% ads than we thought.  What we would love to do is run ads 100% of the time.  Of course, you are free to sell that ad space yourself.  I’m sure you have a full time ad sales force.  We’re only a multi-billion dollar company focused soley on monetizing pageviews, but feel free to go it alone if you want.”  To the developer, this could be music to your ears - focus on building the application, and GOOG does the monetization for you.  If you really wanted to go crazy, one could imagine a world where GOOG says to their developers, “$5 bounty for all new GOOG accounts created through your app.”

Bottom line: GOOG is motivated by feeding the GOOG beast and monetizing pageviews.  That shines through in the design of their app delivery vehicle, the tight coupling of the services, and the expected design patters of the applications to be built on their infrastructure.

Microsoft

With all of the above points in mind, hopefully you can start drawing some pretty good conclusions about the potential future directions of the competitive cloud platforms in the market.  As for MSFT, there are plenty of things I could say, but let me simply state what I believe to be our motivations.  We are a platform company.  We very much believe that we are in the business of delivering the best platform and tools to developers to build great applications.  Our on-premise stack has proven to be extremely successful over the last several decades.  With the release of the Azure Services Platform, one of the core design tenets was that we would like to achieve parity between our on and off-premise stacks.  The entirety of the Azure Services Platform is designed to enable experienced MSFT developers to be combat effective on day one.

However, the platform, from my point of view, is but a part of the engine.  The heart of the engine is the application that you plan to build.  GOOG very much believes that the web is the platform, and thus the web browser is the only delivery vehicle for applications.  We have a divergent opinion.  It seems absolutely crazy to me that a person carries around more computing power in their pocket today (cell phone) than was available a scant 15 years ago in the high end desktop computers, and GOOG believes that those processor cycles should be shunted aside.  Our software plus services strategy really is about enabling developers to deliver the right experience to their customers, on the preferred end point device, at the right time from a single piece of infrastructure.  We are motivated by our developers building great applications, and Azure is the final leg of the software plus services strategy.

I’ve been off for a few weeks launching a little thing called Azure Services Platform.  Did you miss it?  There are plenty of videos over at the PDC 2008 site.  Here’s a great video of Steven Marx doing a walk-through of the Windows Azure code.

I will post my in depth thoughts on what we released at another time.  The coverage has been impressive, as you can see from these search results to the techmeme run.  I will revisit the topic later when I have something new to add.

The topic I did want to cover is the coming need for SLAs and trade agreements between trading partners who may not know that they exist in an application with one another.  Imagine a developer building a composite application through the use of multiple web services, each of them running via a different hosting provider.  The myriad of problems which can, and will, arise, have yet to be adequately addressed by the cloud providers.  One thing that any enterprise IT buyer knows how to say is “who’s throat do I choke?”  What they are referring to, of course, is the notion that should something go wrong with their applications, they need to know that there is someone whom they can call, scream at, and from whom they can expect a late night visit of the monkeys to the cages to fix whatever errant process is running amok.

When you have a composited application, who exactly are you going to be calling?  How can you even begin to diagnose the root cause of the issue.  Further, what if QoS (quality of service) is the culprit?  You calls are failing because the data is getting to you too slowly.  It eventually gets there, it just gets there too slow.  Is that factored into your agreement with the service providers?  Or were you only thinking about SLAs?  Either way, you still have the problem of who to blame, the ingress or egress traffic provider.  What happens when this is a duplex, synchronous transfer?  The real challenge for anyone looking to build composite apps will be ensuring that the service is uniform from each of their providers, which could be compounded by the fact that those providers may very well exist in different hosting facilities in different parts of the world.  Want yet more complication?  What happens when your cloud provider has multiple datacenters and has the ability to move your code around based on their need (read: not your need)?

As you can see, we’re just now starting to scratch the surface of what’s possible with cloud computing, but also just starting to understand what can go wrong.  Without proper planning and thinking, we are going to be digging ourselves some real holes in terms of end customer sat, partner sat, and developer sat.  Anyone have any thoughts?

(graphic courtesy my buddy responsible for Archinect Travels - a fantastic travel video blog wittily chronicling arcitecture across America)

I’m not saying anything…but I do find the speculative fiction from the news types (this taken from news.com)  The next couple of days will be fast and furious, and there is no shortage of noise coming from the PDC.  I can’t wait to take the muzzle off…

The intellect of individuals in a group decreases exponentially as the number of individuals in the group increases.

This holds true for a meeting of any kind, especially one where television cameras and a national audience are involved.  Yeeessh…

The funny thing (depending on your point of view) is that the darn things bounce pretty high when they have had as far a fall as the Dow has seen in the last month.  Recall, we’ve seen 26% decline in the last month alone.  Is now the time to stand up and cheer about the policies working, or a time to sit back and mull over the realities of the market positioning and where things are going?  I’m not sure which it is, but I would really hate for a bunch of people to start congratulating themselves with high-fives and back slaps around the notion that “the plan is working.”  We simply do have enough data at this point to call it.

In times past, ISVs would develop their software and ship it off to their customers when there was a sale.  Way, way back, diskettes were sent, which evolved to CDs, then to DVDs, and eventually net-based delivery.  For the really complicated stuff, a bona fide “systems engineer” would show up with the servers under his arm.  The beauty of this model, from the stand point of running a business, was that you knew exactly how much profit you were making on a customer.  Sales guys have some loose reigns, for the most part, so that they can get the deal done, and their lever is product price.  CFOs can then plug all the data into their spreadsheets and from that they derive the customer profitability.

You see, it doesn’t generally matter if you write bad code.  The first step in all customer service calls for any enterprise software packages is to poke the box.  No matter what the issue is, the first step is to reset the box.  How does that construct carry over in a cloud based world?  With shared hosting, the damage any single tenant can do is limited to the number of other deployments on that shared set of servers.  With a true cloud - a fabric of machines shared across all deployments — there is no proverbial machine to poke.  Mercifully, the cloud architects have thought this through, and are building in self-healing into the infrastructure.

What this example does not envisage is the profitability impact of bad code.  When code goes awry on a customer site, only that customer is impacted.  Profitability never enters into the equitation.  Service support calls, if they are not charged for, are at least baked into the standard service and p contract.  Further, a machine lockup cannot spread in this environment.  In a cloud, thanks to the magic of elasticity, without proper controls in place, errant code can actually spread like a virus, causing machine images to spin up, which in turn drains the profitability of that customer account.

Consumptive pricing for end customers hasn’t yet hit mainstream.  Most pricing (if you are charging at all) is based on price per seat per month or year.  If you are hosting your web app with a hoster, your profitability is locked in when you make the sale.  There is some variability associated with network throughput charges, but for the most part, you’re locked in.  In the cloud world, where all of your resources are billed out 0n a consumptive basis, profitability is now a function of variable cost inputs.  These inputs are impacted by how well your code performs.

Amazon loves to hold out Animoto as an example of the greatness of their platform.  They love to show the chart on the left here.  In a couple of days, usage of the Animoto service exploded.  There’s an accounting of the event in a blog post by the AWS team.  If you do the quick math, they were supporting approximately 74 users per machine instance, and their user/machine image density was on the decline with increased user accounts.  The story they like to tell from this chart is “wow, we were able to spin up 3000 machines over night.  It’s amazing!”  What I see is more along the lines of “holy crap, what is your code doing that you need that many instances for that many users?”  I don’t mean to impugn Animoto here, but I don’t want the point to be lost: the profitability of your project could disappear overnight on account of code behaving badly.

For the most part, the design and efficiency of your code is largely in your hands.  What about a potentially more onerous situation - customers behaving badly?  Revisiting the software license model, a customer can only use so many cycles on the servers to which your application is deployed.  That customer can abuse the code, but they cannot impact the customer experience of any other customer, and they can’t cause you to deliver more machines to them to support their load unless they pay for them.  As such, you’ve collected your money, and they have no impact on your profitability.  In a world where SaaS vendors haven’t yet figured out how to do consumptive based pricing for their offerings, the very real possibility exists that a small handful of customers can abuse your application and destroy not only the profitability of the account, but of your entire P&L.

The simple answer is “quotas,” though that is somewhat harder to enforce in practice when the consumptive unit is not easily measurable.  What kind of a quota can you put in place for a business intelligence application where a micro managing mid-level employee is running wild with “what-if” scenarios?  As more applications migrate to the cloud, there will be plenty of scenarios where the work units are not as simply to constrain as disk or processor usage.

Consumptive costing models, automatic scaling of applications and difficult to define atomic quota units have the potential to create serious financial challenges for cloud based application vendors.  The new challenges will manifest themselves by enforcing rigid software efficiency design goals on development teams, and forcing the operations team to entertain the notion of firing bad customers.  Designing good software is certainly not a new topic, but the possibility of bankrupting a company is not something about which the architects have ever really had to think.