How Notion Decreased Latency by 20% with Caching
Notion recently added SQLite for client-side caching in the Notion website. We'll talk about how they did this and what issues they had to resolve. Plus, how to influence with data as a software engineer, the history of Rust and more.
Arpan KG
August 03, 2024
Hey Everyone!
Today we’ll be talking about
How Notion uses SQLite for Caching - Notion recently added SQLite for client-side caching on the Notion website. This resulted in a 20% improvement in page navigation speeds.
Brief overview of SQLite and why it’s useful
How Notion uses a “SharedWorker” architecture to implement client-side caching
Issues the Notion team faced with SQLite corruption, slow disk reads/writes and slow initial page loads.
Tech Snippets
How to influence with data as a software engineer
How Rust went from a side project to the world’s most-loved programming language
Why CSV is still king
Product for Engineers is a fantastic newsletter by PostHog that helps developers learn how to find product-market fit and build apps that users love.
A/B testing and experimentation are crucial for building a feature roadmap, improving conversion rates and accelerating growth. However, many engineers don’t understand the ins-and-outs of how to run these tests effectively (they just leave it to the data scientists).
This edition of Product for Engineers delves into A/B tests and discusses
The 5 traits of good A/B tests
How to think about statistical significance and p-values
Avoiding false positives
And more.
To hone your product skills and read more articles like this, check out Product for Engineers below.
partnership
How Notion uses SQLite for Caching
Notion is a productivity application that serves as an “all-in-one” workspace. It’s super powerful and you can use it for taking notes, building spreadsheets, managing calendars, tracking timelines and much more.
You can use Notion through the browser (notion.so) or by downloading their desktop/mobile applications.
One challenge Notion engineers have to deal with is making sure that every user has a great experience regardless of whether they’re using Notion through the website, windows application, iOS app, macOS app etc.
In 2021, Notion started using a SQLite database for caching data in the windows and macOS apps. This change made initial page loads 50% faster and also sped up navigating between pages by 50%.
The engineering team wanted to use the same strategy to bring performance gains for users who use Notion through their browser.
They ended up using the WebAssembly implementation of sqlite3 and were able to achieve a 20% improvement in page navigation times for the Notion website.
Carlo Francisco is an engineer at Notion and he wrote a terrific blog post delving into how the Notion team implemented this change and some hurdles they faced.
We’ll first give a brief overview of SQLite and then talk about Notion’s process for caching on the client.
Intro to SQLite
SQLite is an open source, embedded relational database. It’s very powerful with support for ACID transactions, full-text search, geospatial queries and much more.
The key feature of SQLite is how it’s lightweight and “serverless”. It doesn’t require a separate process for the server. Instead, a single library contains the entire database system and it integrates directly into your application code. All your data is stored locally.
If you’re building a Python app then you can import the sqlite3 library and use it to create tables and make SQL queries.
SQLite is a great option when you’re writing an app where the user doesn’t have a stable internet connection. It lets you easily store the user’s data on their device and read/write the data with SQL.
SQLite has libraries for Java, C++, C, Python, JavaScript, Rust, Go and more.
It’s used extensively in areas like:
Embedded Systems - Devices like industrial controllers, automotive infotainment systems, home automation systems, etc. can often have limited internet connectivity while requiring high reliability and low latency. SQLite provides an easy way to store data on the device without taking up too much RAM/CPU.
Desktop & Mobile Apps - If you’re building apps for macOS, windows, iOS, Android, etc. then you can import SQLite into your application code and use it for writing data locally to your user’s disk. It’s useful if your users need to use the app without a network connection or if you want to cache data locally.
Web Development - With SQLite compiled to WebAssembly, you can also use SQLite with your web applications (you just import it in your JavaScript code). This is useful for offline functionality, caching data locally and more. In the rest of the article, we’ll talk about how Notion used WASM SQLite for caching data in the browser.
How Notion uses SQLite in the Browser
Notion uses the WebAssembly implementation of SQLite.
With WebAssembly, you can take code written in C, C++, Rust (and more) and compile it to a low-level assembly-like language (called WebAssembly or WASM) that runs in the browser. Modern browsers can all run WASM with near-native performance.
Under the hood, the WASM SQLite library uses the browser’s File System API to write the state of the SQLite database to the user’s hard drive. More specifically, the library uses the Origin Private File System endpoint in the File System API.
Here’s how the caching works at a high level
Each Tab has its own dedicated Web Worker - Each Notion tab that you open in your browser has its own dedicated web worker for writing to SQLite. A web worker is a feature browsers provide that let you run JavaScript in a background thread without affecting the browser’s UI thread. This lets you run computationally intensive tasks on your website locally without affecting the user experience.
Web Locks to prevent Multiple Writers - The “active tab” (whichever the user opened last) is the only tab that is permitted to use its web worker for writing to SQLite. Notion manages this by using the Web Locks API and a separate SharedWorker web worker that interacts with all the open Notion browser tabs.
SharedWorker Passes Queries to the Active Tab - SharedWorker will keep track of which tab is the “active tab”. Whenever SharedWorker gets a database write from a non-active Notion tab, SharedWorker will redirect the query to the active tab’s web worker.
Some issues the Notion team faced include:
Database Corruption Issues - Prior to using the SharedWorker-powered approach (with locks), Notion was facing issues around corruption with the SQLite database. Some users were seeing the wrong data on a page with comments attributed to the wrong co-worker or incorrect page previews.
The cause of the corruption had been down to concurrency issues. Before, Notion had each open Notion tab writing to the SQLite database simultaneously. The OPFS API (for reading/writing to disk) didn’t have good concurrency handling so that was causing corruption issues. This issue was resolved by switching to the SharedWorker-powered architecture we described above (only the active Notion tab can write to disk).
Slow Disk Reads - After adding the browser caching, the Notion team started to see performance regressions in the slowest devices (the 95% percentile times for page Navigation were getting worse). After investigating, they found it was because older Android phones were reading from disk extremely slowly. They couldn’t assume that loading data from the disk cache would always be faster than loading the same data from the Notion backend API.
They fixed this issue by having the browser “race” the two asynchronous requests (disk cache with SQLite vs. network request with Notion’s API). This meant that the website would automatically use network requests if reading from disk was too slow.
Not Loading SQLite Asynchronously - Initially, the Notion team wasn’t loading WASM SQLite asynchronously. This meant the download/processing was blocking the page load process and slowing down the initial page load. After loading WASM SQLite asynchronously, they saw faster page load times but it meant that the initial page data wouldn’t be loaded from SQLite (it would have to be a network request). However, this trade-off still resulted in a lower page load time for users.
Results
After adding SQLite for browser caching, Notion has seen a 20% improvement to navigation times without any other metrics regressing.
The fastest way to get promoted is to work on projects that have a big impact on your company. Big impact => better performance review => promotions and bigger bonuses.
But, how do you know what work is useful?
The key is in combining your abilities as a developer with product skills.
If you have a good sense of product, then you can understand what users want and which features will help the company get more engagement, revenue and profit.
Product for Engineers is a fantastic newsletter that’s dedicated to helping you learn these exact skills.
It’s totally free and they send out curated lessons for developers on areas like
How to run successful A/B tests
Using Feature Flags to ship faster
Startup marketing for engineers
and much more.
partnership
