I was excited to see that Pat Helland has published another thought provoking paper: Immutability Changes Everything. If video is more your style, Pat gave a wonderful talk on the same subject at RICON2012 (video, slides).
It's fun to see how Pat's thinking is evolving over time as he's worked at Tandem Computers (TransactionMonitoring Facility), Amazon, Microsoft (Microsoft Transaction Server and SQL Service Broker), and now Salesforce.
You might have enjoyed some of Pat's other visionary papers: Life beyond Distributed Transactions: an Apostate’s Opinion, The end of an architectural era: (it's time for a complete rewrite), and Idempotence Is Not a Medical Condition.
This new paper is a high level overview of why immutability, the idea that destructive updates are not allowed, is a huge architectural win and because of cheaper disk, RAM, and compute, it's now financially feasible to keep all the things. The key insight is that without data updates, coordination in a distributed system becomes a much simpler problem to solve. RedHat is linking microservices and containers with immutability.
Immutability is an architectural concept that's been gaining steam on several fronts. Facebook is using a declarative immutable programming model in both the model and the view. We are seeing the idea of immutable infrastructure rise in DevOps. Aeron is a new messaging system that uses a persistent log to good advantage. The Lambda Architecture makes use of immutability. Datomic is a database data that treats data as a time-ordered series of immutable objects.
If that's of interest, then you'll like the paper.
There is an inexorable trend towards storing and sending immutable data. We need immutability to coordinate at a distance and we can afford immutability, as storage gets cheaper. This paper is simply an amuse-bouche on the repeated patterns of computing that leverage immutability. Climbing up and down the compute stack really does yield a sense of déjà vu all over again
It wasn’t that long ago that computation was expensive, disk storage was expensive, DRAM was expensive, but coordination with latches was cheap. Now, all these have changed using cheap computation (with many-core), cheap commodity disks, and cheap DRAM and SSD, while coordination with latches gets harder because latch latency loses lots of instruction opportunities. We can now afford to keep immutable copies of lots of data, and one payoff is reduced coordination challenges.
Designs are driving towards immutability. We need immutability to coordinate at ever increasing distances. We can afford immutability given room to store data for a long time. Versioning gives us a changing view of things while the underlying data is expressed with new contents bound to a unique identifier
- Pat Helland's Weblog while at Microsoft
- Normalization Is for Sissies by Pat Helland
- Memories, Guesses, and Apologies by Pat Helland
- Idempotence Is Not a Medical Condition by Pat Helland
- Video of Immutability Changes Everything talk at RICON2012. (slides)
- Trip Report -- HPTS (High Performance Transaction Systems) Workshop -- Part 1 of the Trip Report by Pat Helland
- 7 Design Patterns For Almost-Infinite Scalability
- Scaling Secret #2: Denormalizing Your Way To Speed And Profit
- Comments on the paper on Reddit and on Hacker News
- The Road to Akka Cluster and Beyond
- Get your own bumper sticker
- Why aren't all data immutable?
- If immutable objects are good, why do people keep creating mutable objects?
- Log-structured merge-tree
- irmin - a distributed database that follows the same design principles as Git
- "I am the Lord, I change not; therefore ye sons of Jacob are not consumed."—Malachi 3:6