Tech Dive on DynamoDB
We'll talk about the history of DynamoDB, it's architecture, downsides and more. Plus, how Dropbox selects data centers, a history of C compilers and more.
Hey Everyone!
Today we’ll be talking about
Tech Dive - DynamoDB
History behind DynamoDB and why Amazon created it
The Dynamo Project and SimpleDB
DynamoDB Data Model
Partitioning Scheme and Durability Guarantees
Downsides of DynamoDB
Case Studies at Uber, Instacart and Canva
Tech Snippets
How Dropbox selects Data Centers
Why Complexity sells better than Simplicity
A History of C Compilers
The Fallacy of having Developers Split Time on Multiple Projects
Time-based CSS Animations
Have you ever been curious about the inner workings of RAM and why it’s so much faster than disk? Are you familiar with the many optimizations that your computer does to minimize RAM usage like page sharing, demand loading, copy-on-write and more?
If you’d like to learn about this, Brilliant has a new detailed course on Computer Memory with 26 interactive lessons that you can easily do on your phone or laptop.
This is just one of thousands of lessons that Brilliant has that cover all topics across software engineering, machine learning, data science, quantitative finance and more.
They structure their lessons in bite-sized pieces, making it ideal for those short 10 minute breaks that you might otherwise waste on Instagram or Twitter.
With the link below, you can get a 30-day free trial to check it out. You’ll also get a 20% discount when you subscribe.
sponsored
DynamoDB Tech Dive
DynamoDB is a fully managed NoSQL database available on AWS. It’s proprietary, so the source code is private and it can only be run on AWS infrastructure. Although it’s closed source, AWS has published a bunch of talks and papers describing how it works.
In this article, we’ll be talking about the history behind DynamoDB, it’s data model, pros/cons, details on the architecture and more.
This is the first part of a tech dive we did on DynamoDB.
You can read the full article by subscribing here.
So, why use DynamoDB? Some of the core selling points are:
Scalability - DynamoDB is known for it’s ability to scale to extremely high usage while providing predictable performance around latency.
Amazon “dogfoods” their services heavily and DynamoDB has been able to handle loads of 80 million+ requests per second from Amazon use-cases. It’s also heavily used by companies like Netflix, Zoom, Lyft, Snapchat, Dropbox and more.
Database as a Service - DynamoDB has a serverless model where you don’t have to do any maintenance or auto-scaling. You can just use AWS’ API for creating tables, sending reads/writes, managing indexes, etc. and they handle everything internally. DynamoDB will accommodate your workload as it ramps up or down and you’ll be billed per request.
History
DynamoDB is the culmination of learnings from multiple previous projects at Amazon.
The two sources of inspiration were:
Dynamo
SimpleDB
Dynamo
Dynamo was the first NoSQL database system developed at Amazon. It was a key-value store built for internal use (Dynamo was built in the early 2000s so this was before AWS).
With Dynamo, Amazon was able to gain experience around building highly scalable storage systems with techniques like
consistent hashing for partitioning data across multiple nodes
object versioning for handling concurrent updates
quorums for maintaining consistency amongst replicas
and much more.
They discussed lessons learned in their 2007 paper on the Dynamo system. This paper was crucial to distributed databases and helped to popularize the idea of “eventual consistency”.
(Note - despite the similar names, DynamoDB has a completely different architecture than Dynamo)
One of the main downsides of Dynamo was that it was single-tenant. Each team at Amazon had to manage and scale their own Dynamo installation, leading to a lot of wasted time and inefficiency.
This helped led to SimpleDB.
SimpleDB
SimpleDB was announced in 2007 as a fully managed NoSQL database. The goal was to provide the same scalability and performance of Dynamo with a managed/easier-to-use experience.
Unlike Dynamo, SimpleDB was publicly available as one of AWS’ cloud offerings.
However, there were quite a few issues with SimpleDB:
No Consistent Reads - At launch, SimpleDB only provided eventually consistent reads. Consistent reads were added a few years after launch but Amazon discouraged their use due to higher latency and lower throughput.
Storage Limitations - SimpleDB had a strict storage limitation of 10 GB per domain. It was meant for small, less-demanding workloads.
Lack of Complex Querying and Transactions - SimpleDB didn’t have support for joins or aggregations. It also didn’t provide ACID transaction guarantees.
Despite being very limited, SimpleDB showed the developer demand for a fully managed, highly scalable, NoSQL database.
Amazon took their lessons from Dynamo and SimpleDB and created DynamoDB.
This is the first part of our Tech Dive on DynamoDB.
In the rest of the article, we’ll talk about
DynamoDB Data Model
Partitioning Scheme and Durability Guarantees
Downsides of DynamoDB
Case Studies at Uber, Instacart and Canva
You’ll also get access to a PDF of all our recent tech summaries, spaced-repetition flash cards on all the content covered in Quastor and more.
Have you ever wondered how your smartphone knows your precise location anywhere on Earth? It’s a fascinating process involving atomic clocks, trilateration, satellite geometry and more.
If you’re curious, Brilliant released a course called How Technology Works that delves into all of these topics in an engaging, easy to understand way.
This is just one of thousands of lessons that Brilliant has that cover all topics across software engineering, machine learning, data science, quantitative finance and more.
They structure their lessons in bite-sized pieces, making it ideal for those short 10 minute breaks that you might otherwise waste on Instagram or Twitter.
With the link below, you can get a 30-day free trial to check it out. You’ll also get a 20% discount when you subscribe.
sponsored
