Twitter: Announcing Snowflake

With NoSQL being in its relative early days organizations are having to solve problems that have not existed in the traditional relational data store. Take for example, Twitter and its need to generate unique ids for each tweet. So what are the requirements for such a task?

We needed something that could generate tens of thousands of ids per second in a highly available manner. This naturally led us to choose an uncoordinated approach.

These ids need to be roughly sortable, meaning that if tweets A and B are posted around the same time, they should have ids in close proximity to one another since this is how we and most Twitter clients sort tweets.

Additionally, these numbers have to fit into 64 bits. We’ve been through the painful process of growing the number of bits used to store tweet ids before. It’s unsurprisingly hard to do when you have over 100,000 different codebases involved.

So what is the solution? Well it's project Snowflake of course!

Read more: Announcing Snowflake