Hey Everyone,

Today we’ll be talking about

• How Facebook implements sharding for their backend services.

  • Many independent teams at Facebook were building their own custom sharding solutions to scale up their services.

  • A team at Facebook then built Sharding Manager, a platform that all backend services can use to automatically shard their applications.

• Clean Code’s recommendations on commenting your code.

  • More comments does not mean cleaner code! In fact, it usually implies the opposite.

• Why the Internet went down for 2 hours on July 22nd.

  • If you guessed that it had something to do with DNS, then you can pat yourself on the back.

Join Quastor Daily If you’re Not Subscribed!

Tech Snippets

How Facebook implements Sharding (full article)

When you’re operating at Facebook’s scale, horizontal scalability is no longer optional.

Therefore, backend teams that work with data must use sharding to scale the services they’re building.

Things to Consider…

Here are some factors that engineers at Facebook had to consider when implementing sharding…

• Consistent hashing is typically used to spread the data across servers, but it gives issues when you have constraint-based allocation (ex. the data of European Union users should be stored in European data centers).Therefore, you need a scheme that can follow constraints when partitioning data into shards.

• Failover ability is also necessary for hardware/software failures. Your system should divert traffic away from failed servers and may need to rebuild impacted replicas on healthy servers. You’ll also have to ensure that each shard has enough healthy replicas.

• Some shards may receive far more requests than other shards. If you have multiple “popular” shards on the same server, then this could be an issue. Additionally, the popularity of shards is constantly changing. Therefore, you’ll also need load balancing to dynamically adjust which shards are stored on which servers (and add read-replicas if necessary).

Different teams at Facebook were already building their own custom solutions for sharding, so a team at Facebook designed Shard Manager as a generic shard management platform.

Shard Manager

Shard Manager manages tens of millions of shards hosted on hundreds of thousands of servers. It is used for hundreds of applications in production.

It is the only generic sharding platform in the industry that achieves wide adoption at Facebook-level scale.

Some of the features that were included when building Shard Manager are…

• Fault Tolerance - Shard Manager enables redundancy via replication by allowing users to configure replication factors on a per-shard basis. It also supports spreading shard replicas across different regions/locations.

• Load Balancing - Shard Manager takes in per-server and per-shard data to determine the optimal way to store shards. As mentioned previously, you wouldn’t want to put all the most popular shards on the same server.,

• Shard Scaling - Shard Manager can dynamically adjust the replication factor (and add more read-replicas for ex.) when the load on a shard deviates from a user-configured acceptable range.

Here’s the architecture of Shard Manager…

For a full explanation of the implementation, check out Facebook’s article!

Akamai DNS failure crippled the Internet for two hours

Websites and services that had outages include Amazon, UPS, FedEx, Airbnb, PlayStation Network Stream, Steam, LastPass and more.

The root cause of the outage was a failure in Akamai’s Edge DNS service.

Domain Name System (DNS) servers translate domain names (like google.com) into Google’s IP address.

Akamai’s Edge DNS service has thousands of DNS servers around the world so users can quickly get IP Addresses and Akamai’s service also helps you avoid DDoS attacks (A DDoS attack on a company’s DNS servers can be quite effective in taking their site down).

But, that obviously doesn’t work if Akamai’s service goes down!

By the way, Tom Leighton, the CEO and Co-Founder of Akamai (a $20 billion dollar company) is also a part-time math professor at MIT.

He also taught a useful class on Mathematics for Computer Science (concepts from Discrete Mathematics). You can view the lectures here.

Also, the term Consistent Hashing was actually coined by Tom Leighton and Daniel Lewin (the other co-founder of Akamai).

Join Quastor Daily If you’re Not Subscribed!

Clean Code’s advice on Comments

Comments should be thought of as a necessary-evil. You’ll have to occasionally add a comment to your code, but you should try your hardest to avoid them.

A comment is an indicator that you couldn’t properly express yourself in the code.

Therefore, if you think you need to write a comment, first try to see if you can express yourself in the code itself.

You might be wondering, why are comments bad? In fact, many new developers seem to think that more comments are always good.

Here’s why comments should be avoided...

Programmers can’t maintain comments

Your code is constantly changing and evolving. It’s getting refactored and new functions, classes, etc. are being added in.

When you refactor a piece of code, how do you know if you just invalidated a comment written somewhere else in your codebase?

Unless you keep track of every comment that pertains to the piece of code you just refactored (impossible to do if you have tons of comments), then you have no way of finding out.

Your compiler will tell you about errors in your code, but there is no way of knowing what comments in your code are incorrect/misleading.

Also, as you refactor your code, comments tend to get separated from the code they describe and eventually become blurbs. And as time passes on, the code they describe may change and the comment will become inaccurate.

The older a comment is, and the further away it is from the code it describes, the more likely it is to be just plain wrong.

Figuring out which comments in your codebase are wrong/misleading is an extremely difficult task.

Too many comments will make readers tune the comments out

Another issue with excessive comments is that eventually, the reader will start to tune them out.

If your code properly expresses itself but you still place redundant comments that just repeat what the code does, then the reader will learn to ignore the comments.

The comments aren’t conveying any new information, so what’s the point of reading them?

This becomes an issue because the reader will tune out important comments as well.

There’s no way for a reader to distinguish between which comments are important to read and which comments are redundant.

Comments cover up messy code

Many times, a programmer will write bad code and then try to cover up for it with a comment.

A common scenario is a comment next to a variable declaration, where the comment explains what the variable does. This just means that the variable was poorly named. You should be able to tell what the variable is doing based off it’s name (don’t fear long variable names!) and you shouldn’t need a comment to figure that out.

Another scenario is a comment next to a function, explaining what the function does. Again, this means you should refactor the function. It’s probably too long (functions should be 5 to 15 lines long) and it might be doing multiple things (functions should only do one thing)

But again, not all comments are bad.

Here are some scenarios where comments can be helpful…

Good Places to add a Comment

• Legal Comments - copyright and authorship statements may be necessary and reasonable things to put at start of each source file.

• Explanations of Intent - Expressing intent in your code can be quite difficult. Therefore, brief comments that explain design choices could be useful.

• Warning of Consequences - A comment that warns a developer about a specific action or change can be useful.

  • An example is if you have a test condition that takes a really long time to run.

  • A comment warning the reader that the test condition takes a long time to evaluate could be useful (if there’s no way of telling from reading the code).

• Amplification - A comment could be useful for amplifying the importance of something.

  • This ties back in to why you shouldn’t use redundant comments (comments that just restate what the code does).

  • If you have an excessive amount of redundant comments, then programmers will tune out all of your comments, including comments that amplify important information.

• Docstring for Public APIs - JavaDocs or Docstrings can be very useful for Public APIs. But again, just make sure they don’t become outdated or misleading.

Interview Question

A robot is located at the top-left corner of a m x n grid.

The robot can only move either down or right at any point in time.

The robot is trying to reach the bottom-right corner of the grid.

How many possible unique paths are there?

Here’s the question in LeetCode.

Join Quastor Daily If you’re Not Subscribed!