Scaling MySQL

After the storm has passed regarding GigaOM’s controversial post Facebook trapped in MySQL ‘fate worse than death‘, Derrick Harris made his message clear once and for all:

Whatever database technology someone might choose to use for a new web application, anyone who hopes to achieve even a fraction of Facebook’s traffic should not go down the same path as Facebook did.

But bear in mind the situation of Facebook compared to other big companies:

1. Facebook is using open source MySQL

2. Facebook has “absolutely skilled” engineers (as Jim Starkey has noted)  and they don’t exist everywhere

3. Facebook has the added benefit of being able to pay them.

Facebook is a read-write software company, unlike the typical read-only big companies that have no time or capability rewriting software when it breaks down. Software engineering is Facebook’s forte because it’s in the business of software infrastructure. That ‘s the beauty of open source. You can tweak the code based on your needs.

But not everyone belongs to that category.

Architecture is the problem

According to database industry analyst Curt Monash, Stonebraker makes a valid point in citing Facebook’s complex MySQL situation, because Facebook isn’t using MySQL for its relational capabilities. MySQL might be a fine database choice for a low-end application that requires full relational capabilities, but sharded MySQL plus memcached is not. You lose a lot of those as soon as you begin sharding, he explained, and the application actually communicates directly with memcached for data that resides in that layer. It’s that architecture that’s the problem.

Monash believes there are two timelines for when a technology runs its course, depending on the situation: 1) when you shouldn’t use it to start a new project, and 2) when you should upgrade. For new projects that might have to scale massively, he said, you wouldn’t choose MySQL plus memcached.

As for the sharding, Starkey said, “The only thing sharding has going for it is the absence of alternatives.” He noted that although it’s difficult to find anything he and Stonebraker agree on, they do both agree that traditional SQL databases aren’t easy to scale. Because scaling them is so complex, Starkey — who, like Stonebraker, has a horse in the NewSQL race with NimbusDB — thinks all legacy databases will be irrelevant in a few years. All except low-end MySQL, that is.

So what are the options?

1) For new apps, you may use Clustrix,TokuDB, ScaleDB and Schooner MySQL with Active Cluster.

2) For retrofitting apps with transparent sharding,there is dbshards and ScaleBase.

Related link:

DBMS2 on Facebook/Stonebraker flap


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s