The Cloud is Not a Railroad - An Argument Against the Vertical Separation of Cloud Providers

There's a move to regulate cloud providers by vertically separating the services they offer.

Like railroads of yore, who were not allowed to provide freight services on top of their base services, cloud providers would not be allowed to provide services on top of their base platform services.

Vertical separation would be new to the cloud industry. Is it a good idea? Would it actually solve any problems? My answers are no and no, but probably not for the reasons you think. Let's dive in.

Here are a few useful resources for exploring this argument:

• Maintaining monopolies with the cloud by Cory Doctorow. https://pluralistic.net/2022/09/28/other-peoples-computers.
• A group advocating for fair licensing. https://www.fairsoftwarelicensing.com/our-principles.
• Cloud Infrastructure Services -- An analysis of potentially anti-competitive practices by Professor Frédéric Jenny. https://www.fairsoftwarestudy.com/.

Now Cory is about 100x times smarter than I am, but this analogy is wrong:

That's why the trustbusters were so big on "structural separation": the principle that a business can own a platform or use the platform, but not both.

So this remedy is also wrong:

Rather than trying to construct rules that kept the referee honest even when they played on one of the teams, they ordered the companies to choose one role – either you were the ref, or you were a player, but you couldn't referee a match where you were also in the competition.

Because this idea, while trivially true, misses the point:

There is no cloud, there is only other people's computers.

This is like saying there is no human body, it's just a collection of cells. While true, it is the lesser truth that fails to recognize the importance of the body, organs, and the adaptive function of the brain. That's what a cloud provider is; it's the body electric for the cloud.

The report by Professor Jenny seems to be the main support for cleaving cloud providers in two.

I've read the report. Is there a smoking gun? Not that I found. But then, I'm not interested in enterprise licensing woes with Oracle or Office 365. What I'm concerned about, as a developer, is the collateral damage such ham-fisted regulation might cause.

The job of a cloud is to run workloads. Workloads are implemented using services. A cloud offers those services. The more high-quality services a cloud offers, the better products we can build.

Limit the cloud services a cloud provider can offer and you limit the quality of the software we can build. It would be like removing legos from a lego set. You can't do a proper build without the full set.

There is no arbitrary dividing line that can be made such that platform services are on one side of the line and non-platform services are on the other side. A cloud is a platform all the way down. Without cloud services, there is no cloud because a cloud is literally defined by the services it offers.

A cloud provider packages up vast pools of compute resources and resells them as cloud services. There is no structural separation as there is with a train service. What some consider infrastructure or platform is just another cloud service.

Would you really trust some committee or government agency to draw this line correctly? I would not because I don't think the people who think this way actually understand how the cloud works.

Does anyone really want to go back to the VM-centric days when we rolled everything ourselves? No.

What it means to be cloud-native has gone through several evolutions: VM to container to serverless. Each cloud-native evolution is about using the hardware more efficiently.

Do you think those evolutions would have happened with a structural separation in place? No.

Where's the incentive unless you derive a benefit from optimizing the whole stack? We would still be building higher and higher on top of VMs because that's where platform innovation would have stopped evolving.

This is because the cloud is not a railroad. They are different in kind. Building software is not moving freight. A railroad is fixed. The cloud is ever evolving.

This is my best argument for why the cloud should not be vertically separated. It would make the job of building quality software even harder and slower and more expensive.

Admittedly, it's a developer focussed argument, but there are more technical arguments against the move. Let's walk through them.

Is The Cloud a Natural Monopoly?

Wikipedia:

A natural monopoly is a monopoly in an industry in which high infrastructural costs and other barriers to entry relative to the size of the market give the largest supplier in an industry, often the first supplier in a market, an overwhelming advantage over potential competitors.

Let's look at the size of the market. For all the attention the cloud gets, it's still a child or maybe a pre-teen in terms of the overall IT industry.

The market value of the worldwide IT industry is $5 trillion. The global cloud computing industry was valued at $371 billion in 2020.

So, the cloud is not a monopoly. Why should it be regulated as one?

Examples of traditional natural monopolies are dams, railroads, power plants, water supply, and sewer services. The idea is it's practical to only have one of these for a given area. And even that list is not invulnerable. With solar power, the centralization of power generation is giving way to a more decentralized model.

Does the there can be only one apply to the cloud? No. Network effects are not the same as monopoly control.

Apple is not a monopoly in the smartphone market simply because people like their smartphones. Neither are clouds. People choose to run their workloads on the cloud from a large set of options. It's a choice. It's not because there are no other options available to them because of structural issues.

While high infrastructure costs do create a barrier to entry to creating a cloud provider, this misses an important point: the benefits of the cloud come from the cloud model, not any particular cloud implementation.

Here's an informal definition of a cloud provider taken from my book Explain the Cloud Like I'm 10:

A cloud provider installs a bunch of computers in a bunch of datacenters spread throughout the world, pools all those datacenter resources together, offers them over the internet as cloud services through an API, and rents them to the public in an on-demand, pay-for-what-you-use basis.

The cloud model means accessing services over a network through an API, safely running multiple workloads on a single server, cloud storage, cloud computing, auto-scaling, elasticity, resource pooling, on-demand pay-for-what-you-use billing, load balancing, fault tolerance, leveraging managed services, and building software using a cloud-native paradigm.

The cloud model has rightly attracted customers because it's a powerful way to create, distribute, and use software, but the cloud model is not relegated to only the cloud.

The cloud model and, thus, cloud-native software development can work in any context—on-prem, colo, hybrid, edge, bare-metal. It’s the model that matters. Not who implements it or how it’s implemented.

This is what I mean when I say people aren't understanding the cloud. They over-index on specific cloud implementation instances and not the value and flexibility of the cloud model itself. Does the free market work for the cloud?

Another way to address the natural monopoly questions is to answer: does the free market work for the cloud?

This is where we have a problem. Like telecom, the cloud provider market slipped into oligopoly. There is little incentive for the Big 3 to drive costs lower, but that's not the same as having a monopoly that should split and regulate an industry.

There are many alternatives for running workloads. Take a look at this still incomplete list of cloud providers from Wikipedia. The problem is there's no easy alternatives for running workloads, and that's because there is no standard cloud API.

We have a cloud model that offers a cloud-native method of software development, but it may take several more technological evolutions for the model to advance enough to have a minimal viable portability layer. With container-native PaaS (App Runner and Cloud Run) we are getting close.

We have several players like Cloudflare innovating to become alternative lower-cost cloud providers. And we have k8s striving to become a cloud in a box droppable into any datacenter. You can switch between clouds with effort. It has been done. You can even go old school and use non cloud-native architectures. Anyone can rent a rack in a colo and stand up a system. Did anyone promise you a rose garden? I don't think so.

The larger point is there is a large competitive market amongst a group of competing enterprises. There's just no easy solution to the perceived problem of fungible cloud alternatives.

Is there a way to create greater competition and expand economic choice for cloud customers? I don't know, but high switching costs isn't a proper test for regulating an industry. It's a technical issue that's fixable with effort and investment. Maybe start investing into open source alternatives rather than complaining they don't already exist? Invent the future you want. A Cloud Provider Absorbs Huge Fixed and Sunk Costs

Cloud providers incur huge fixed costs for creating and maintaining a network of datacenters spread throughout the word. Let's say you regulate cloud providers by requiring them to decouple platform services from their other services, whatever those services may be.

Then we must rely on third parties to fill in the gap with managed services of all varieties that must operate out of every datacenter while surviving datacenter level and region level failures.

That's easy to do, right? Of course not, but let's ignore that very few organizations in the world have the technological know-how to create such managed services, especially without low level control of the entire system. But we're ignoring that.

The fixed and sunk costs incurred by the cloud provider will dwarf any investment from third party service providers.

How do you create the right incentives for the cloud provider to keep investing huge amounts of capital in infrastructure while recovering its fixed costs? If competitive service level pricing levels do not cover the cost of the infrastructure then prices would have to be raised to a level that would deny lower-margin users.

We've seen telecom rest and vest on their installed infrastructure, rent seeking by a combination of regulatory capture and the avoidance of the infrastructure investments that would improve and maintain the system.

The disincentive to game service pricing levels in order to create short-term profits at the expense of long-term investment will be overwhelming. It will be impossible to properly set service prices at a level that will provide the ROI required by cloud providers, third party service providers, while minimizing costs for customers so they also can earn an adequate ROI. There's simply not enough room for everyone to get their piece of the pie. What we'll inevitably see are the constantly rising prices and declining service levels experienced in other industries that are regulated as utilities.

This is the single greatest weakness of the proposal for vertically separating cloud providers.

Is that the kind of cloud you want?

Benefits of Vertical Integration

Where is the optimal point of vertical separation in a cloud provider's stack of services? What are the consequences of making the wrong decision?

Any deliberation must recognize the technological interdependence between cloud infrastructure technology and the cloud services that can be offered.

There's a huge short-term and long-term efficiency of services that depends on the successful coordination of cloud services and infrastructure.

For example, AWS created Nitro. Nitro is a revolutionary combination of purpose-built hardware and software designed to provide performance and security. Would Nitro have been invented if AWS was restricted to being a platform provider? Unlikely. It would have had no way of propagating Nitro across an entire vertical stack of hardware and software services. So why bother innovating?

Vertically-integrated cloud providers can optimize the entire system by ensuring that each component is combined in sub-systems that work together and are optimized as whole.

One consequence of this tight integration is the overall high reliability offered by clouds. How reliable would the cloud be if a cloud provider did not control their entire stack of services? Given the experience with finger pointing in other highly interdependent systems, it would be less reliable and outages would no doubt last longer.

Are there any positive externalities to cloud providers that might justify keeping cloud providers? Very large integrated cloud providers might reduce the overall number of servers required to run the world's workloads, but that's not sufficient reason for me not to regulate them. I think my other arguments are reason enough.

Please leave our legos alone.