Why I Converted to Search Servers

chrisp — Mon, 10 Dec 2007 06:08:00 -0800

Many who have worked with me in the past know that I always resisted including search servers (ferret/lucene/solr) into my projects. I argued that they added unnecessary complexity and didn’t provide much that some fancy database searching couldn’t do.

I was wrong.

While I suppose that I still consider the above arguments to be valid, there is one more important argument on the side of using search servers - scalability. I share many rubyists’ disdain for the “S” word. It gets thrown around a lot and typically results in overbloated software that still doesn’t perform well even when “scaled.” There really isn’t a better word in this case though.

HTTP is incredible. As the recent RESTful advocates have pointed out, the protocol is infinitely scalable, simple, and is (in a large way) responsible for the explosive growth of the web. Web servers getting bogged down? No problem, just add another (as long as you’re not trying to do any - or at least not much - state tracking). You can keep adding servers to your heart’s content. A good admin can have this process down to a five minute setup.

Eventually all of those database calls add up though and your database starts straining under load. What do you do now? You can start optimizing your queries, do some fancy caching, add a slave database and keep a read-only connection open to it or (the mother of all evils) create a database cluster.

All of these options suck.

All of that beloved simplicity and scalability of HTTP is gone and replaced by infrastructure that’s extremely expensive and hard to maintain. Sometimes you just can’t get around the database cluster - but it should be a weapon of last resort. Query optimization isn’t so bad if done correctly, but it will only get you so far. Query optimization can also take nice clean code and turn it into a mangled nightmare. Caching is also good in moderation, but too much of it is also a code mangler and can make your application’s usability suffer (waiting 10 minutes for a user’s change to appear is often not usable). Database slaves are fine as failover/backup options but reading from one db and writing to the other also produces mangled code.

The solution? Search servers. Search servers can grow infinitely. You can run one on each web server or you can also have a centralized search server if you prefer. Whenever you pull results from the search server your database has avoided the blow. You’ll still probably need to hit the database occasionally, but your load will be dramatically lower. The load is on the web server end and can be shared across the web cluster without any fancy configuration. Once your search server is in place, use it for as much querying as you possibly can. The growth of your database load will be a nice slowly climbing linear line that will probably keep pace with processor speeds. If done right, you can avoid the extreme overhead involved with a database cluster.

clog: Tag Database

Why I Converted to Search Servers