Hypertable has Reached a Major Milestone!
02.14.2013 | RangeServer Failover
With the release of Hypertable version 0.9.7.0 comes support for automatic RangeServer failover. Hypertable will now detect when a RangeServer has failed, logically remove it from the system, and automatically re-assign the ranges that it was managing to other RangeServers. This represents a major milestone for Hypertable and alows for very large scale deployments. We have been activly working on this feature, full-time, for 1 1/2 years. To give you an idea of the magnitude of the change, here are the commit statistics:
- 441 changed files
- 17,522 line additions
- 6,384 line deletions
The reason that this feature has been a long time in the making is because we placed a very high standard of quality for this feature so that under no circumstance, a RangeServer failure would lead to consistency problems or data loss. We're confident that we've achieved 100% correctness under every conceivable circumstance. The two primary goals for the feature, robustness and applicaiton transparancy, are described below.
We designed the RangeServer failover feature to be extremely robust. RangeServers can fail in any state (mid-split, transferring, etc.) and will be recovered properly. The system can also withstand the loss of any RangeServer, even the ones holding the ROOT or other METADATA ranges. To achieve this level of robustness, we added 63 regression tests that verify the correct handling of RangeServer failures in every conceivable failure scenario. We will follow up later with a blog post describing these tests.
Another important aspect of our RangeServer failover implementation is application transparency. Aside from a transient delay in database access, RangeServer failures are completely transparent to applications that are actively reading or writing to Hypertable. This transparency applies to native C++ applications as well as Thrift-based applications (Java, PHP, Python, Perl, Ruby, etc.). We have introduced regression tests to verify this behavior and ensure that it continues to hold true for future releases.
To learn more about RangeServer failover, visit the Machine Failure page of the Hypertable documentation. I would like to thank the entire Hypertable development team for the tremendous effort that has gone into this feature. Together they have created a solid foundation which will allow us to rapidly build powerful new query capabilities going forward.
Posted By: Doug Judd, CEO, Hypertable Inc.