hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Wanted your thoughts on Rolling Upgrade Testing in HBase
Date Fri, 29 Mar 2013 22:54:33 GMT
Sounds great. +1.

On Fri, Mar 29, 2013 at 3:49 PM, Aleksandr Shulman <aleks@cloudera.com>wrote:

> Hi all,
> I'd like to propose a potential solution to testing HBase Rolling Upgrade
> between minor versions of HBase. It would be great to get feedback.
> *Benefits:*
> This is a tricky feature and it would be really strengthen our trust with
> the product users if we can show that it is stable and correct (e.g. no
> loss of data). The best way to do that is with a test framework that allows
> for repeatability, transparency, and reporting.
> *Considerations:*
> It would be great to see us all agree on what should be supported during a
> rolling upgrade and work towards it using this test framework.
> To that end, I have a few goals in mind:
> -Integration test should be able to trigger a rolling upgrade of an
> existing cluster, regardless of how it is set up
> -It should be extensible enough so that it can be extended to support
> different ways of setting up HBase (tarballs, packages, parcels, or any
> other formats that are relevant)
> -It should be able to report results of various testing operations
> -It should be easy to add new tests to the suite, particularly by those who
> have only passing familiarity with rolling upgrade
> -These tests should be first-class citizens WRT other tests
> -Should standardize what the minimum set of services is for a rolling
> upgrade (e.g. master, backup master, 2 regionservers, zk, etc.)
> -Should support both secure and insecure rolling upgrade
> *Potential Solution:*
> I propose using IntegrationTest to bind to an existing remote cluster. It
> would be able to perform the rolling upgrade steps while running tests over
> a cluster that is already in place.
> At a high level, we would first agree on and create a standard workflow for
> upgrading a cluster. Then, we would work on using IntegrationTest to run
> tests (e.g. insert data, do mapreduce over hbase jobs, etc.) while the
> cluster is in flux. The challenge would be adding code that will handle
> specific cluster setups (e.g. tarballs vs. packages vs. parcels), but we
> can abstract it away.
> Please let me know what you think about this idea and if you'd like to work
> together to accomplish this goal.
> --
> Best Regards,
> Aleks Shulman
> 847.814.5804
> Cloudera

// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message