The QuickBooks Platform
Monday, November 7, 2016 at 8:56AM
Todd Hoff in Example

This is a guest post by Siddharth Ram – Chief Architect, Small Business. [email protected].

The QuickBooks ecosystem is the largest small business SaaS product. The QuickBooks Platform supports bookkeeping, payroll and payment solutions for small businesses, their customers and accountants worldwide. Since QuickBooks is also a compliance & tax filing platform, consistency in reporting is extremely important.. Financial reporting requires flexibility in queries – a given report may have dozens of different dimensions that can be tweaked. Collaboration requires multiple edits by employees, Accountants and Business owners at the same time, leading to potential conflicts. All this leads to solving interesting scaling problems at Intuit.

Solving for scalability requires thinking on multiple time horizons and axes. Scaling is not just about scaling software – it is also about people scalability, process scalability and culture scalability. All these axes are actively worked on at Intuit. Our goal with employees is to create an atmosphere that allows them to do the best work of their lives.

Background

 

The QuickBooks Online product is a decade and a half old. It was built as a monolith, without clear separation of concerns. The monolith has served Intuit well – millions of customers use it with hundreds of millions of transactions in a given day. This was partly possible because swimlaning was built into the product. This allowed scaling by formulaic splits (customers are ‘homed’ in a particular shard). Today, we are well on our way to splitting the monolith into multiple services and moving to AWS as the primary hosting solution.

The Numbers

The Technology

The Culture

Interesting challenges in scalability

Scaling characteristics


Products like QuickBooks Online have scaling characteristics that are different from other SaaS products. A query for a report may have a dozen of more filters applied on it. These queries are generally not the same across companies. Reporting has tax consequences and hence must always be consistent with the books. Collaboration may take place with multiple edits being made by an Accountant and a Small Business owner at the same time. The software need to determine how to handle conflicts.  All this leads to interesting ways of scaling on the QuickBooks Platform at Intuit.

Variability

 

The QuickBooks platform serves millions of small businesses, their customers and Accountants globally. Customers are served for a diverse set of use cases: The variability the product needs to address include:

This is a subset of the diversity we handle. While there is a lot of diversity that is addressed, there are some fundamental concerns that the platform always solves for. Security, Privacy, Scalability and Quality are addressed as part of the fundamental building blocks of the software.

Scaling Philosophy


The QuickBooks platform generally follows the ‘scalability cube’ model, by scaling on three dimensions:

X Axis – Read Replicas

The X axis is used primarily for read replicas. As is the case for many SaaS products, there are a large number of reads compared to the number of writes. A common – and relatively expensive –read operation is reporting. Small Business and accountants need to understand how their business is doing. We deliver insights via a QuickBooks Money Bar, and more detailed insights through a large number of reports (e.g. Balance Sheet, Profit and Loss Reports). Read replicas allow us to both reduce the hotspots in access and deliver precomputed reports

Y Axis – Services

A second dimension to scaling is to break products in distinct services. The QuickBooks platform is modeled as 14 distinct domains, which are composed into products. The services API, is on its fourth revision now – the V3 QuickBooks API can be found at http://developer.intuit.com.

Z Axis – formulaic splits

The Z Axis refers to ‘sharding’ - de-normalized data that allows high scale. QuickBooks splits customers based on the attributes of the company. Each shard is then shared with other companies with similar attributes.

The C axis

In addition to these aspects, we often talking about a ‘C’ axis – the culture axis, which is a key way for us to scale. Key reinforcements for us on the C axis are (a) a culture of metrics – you cannot fix what you cannot observe (b) a culture of ownership via a devops model and (c) customer backed thinking in whatever we do.

Scaling the front end

At over 200KLOC, QuickBooks onlike has a very large Javascript footprint. Part of scaling the front end has meant understanding how to ensure that changes are isolated and components are reusable – to the point that 3rd parties can plug in directly to the front end, and anyone following rules can contribute components. Scaling also means having uniformity in styles that are easy to adhere to.

Journey to Public Cloud

A key strategy at Intuit is to move all software assets to AWS. In the public cloud, given shared infrastructure, additional considerations need to be made for security and privacy. We use the following general principles in moving software to AWS:

Key Lessons

 

Article originally appeared on (http://highscalability.com/).
See website for complete article licensing information.