How is Writing Lord of the Rings Like Writing Software?

Have you ever read a book and wondered how any human could have written something so brilliant? For me it was Lord of the Rings. I despaired that in a hundred lifetimes I could never write a book so rich, so deep, so beautiful. Since then I've learned a few things about about how LoTr was created that has made me reconsider. The kick-in-the-head is that it's the same lesson I learned long ago about writing software.

I've always been amazed how a program can start as a single source file and after years of continued effort turn into a working system that is so large no human can come close to understanding it. If you had tried from the start to build the system you ended up with you would have never ever got there. That's just not how it works. Software is path dependent.

I've experienced this growth from a single cell to a Cambrian explosion many times so I know it's a thing. What I hadn't considered is how it's also a thing for writing books too.

Creating good software is a process of evolution through the mechanism of constant iteration for the purpose of survival. This is also how good stories are made. What both have in common is creation through thought.

Thought needs an object to contemplate. Each intermediate state of a project is that object. By linking together a series of state inspired creative jumps something wonderful can be created that may contain only the faintest trace of its beginnings.

Here's how Lord of the Rings is a good example of this process...

LoTr started out as a sequel to the Hobbit. Tolkien's publisher wanted to cash in on the success of the Hobbit with a sequel. And The Silmarillion wasn't it. So Tolkien began with the intention of writing a sequel to the Hobbit. It was horrible.

The first book title was The Return of the Shadow, not Lord of the Rings. The prose was still written for children. Frodo was called Bingo. Strider was a hobbit called Trotter. Bilbo planned to get married. And the ring was still just a ring. The story had no clear motive or direction. "What more can hobbits do?" asked Tolkein. The ideas of the Hobbit were played out. The LoTr we know and love was far far away.

In draft after draft Tolkien probed and searched for a direction to take the story. It all turned when Tolkien wrote the scene with the Black Rider. At first the Black Rider was really a White Rider. It was Gandalf coming to talk to Bingo. But then some insight happened. A dizzying array of neurons conspired and the color of the horse changed from white to black and Gandalf transformed into a man wrapped in a great black cloak and hood. A new framework was creating itself.

How do we know? Fortunately, from Christopher Tolkien, we have the history of changes his father made to LoTr. Dr. Corey Olsen in a great series—The Return of the Shadow, Session 1 - In Search of a Sequel—walks us through what is essentially the git log for LoTr. Imagine a kind of Papers We Love treatment from a true Tolkien expert and gifted analyst. It's magical.

We see idea after idea worked through in the text. It was a continuous process of refactoring and new development. Some ideas were kept from beginning to end. Many were cut. Many morphed. Much dialogue was kept, but was given to different characters to say in different circumstances.

The whole feeling was very much like seeing software being developed, only the result wasn't a working app, but one of the most influential stories of all time.

The lesson for me was a deep and dramatic reconfirmation of an old idea: All successful large systems started as successful small systems.

This applies to us as writers and programmers. It's easy to get down on yourself during the creation process. Neither your story or program has to start out great; greatness is something that evolves.

In this new year, that's the lesson of Lord of the Rings for me.