Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <nfr6j6jyu2ynf0m3t6qodbd4.1364614338232@email.android.com>
References: <nfr6j6jyu2ynf0m3t6qodbd4.1364614338232@email.android.com>
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
Date: Sat, 30 Mar 2013 09:04:25 -0400
Message-ID: 
 <CAPQV63Xmn00E2YmBjW_CZ_YH=qX+XxhGgw0=hwf72iMVMf-K7g@mail.gmail.com>
Subject: Re: Wanted your thoughts on Rolling Upgrade Testing in HBase
To: dev@hbase.apache.org
Content-Type: text/plain; charset=UTF-8

Hi Aleks,

I'm totally +1 with that too!

With all the coming releases (0.95, 0.96 and 0.98) it might be a very
good idea to have a way to make those tests automatic!

Tracking issues and reporting them correctly over the rolling upgrade
might be a challenge, but the end results is a big add to the
releases.

There will be many things to test in the upgrade, and many scenarios.
How are you going to start the thinking on that? Is there a shared
google document where you have start to put all the ideas/design
around that?

JMS

2013/3/29 Lars Hofhansl <lhofhansl@yahoo.com>:
> Yes please. If possible it could also include a rolling downgrade.
>
> Aleksandr Shulman <aleks@cloudera.com> wrote:
>
>>Hi all,
>>
>>I'd like to propose a potential solution to testing HBase Rolling Upgrade
>>between minor versions of HBase. It would be great to get feedback.
>>
>>*Benefits:*
>>This is a tricky feature and it would be really strengthen our trust with
>>the product users if we can show that it is stable and correct (e.g. no
>>loss of data). The best way to do that is with a test framework that allows
>>for repeatability, transparency, and reporting.
>>
>>*Considerations:*
>>It would be great to see us all agree on what should be supported during a
>>rolling upgrade and work towards it using this test framework.
>>
>>To that end, I have a few goals in mind:
>>-Integration test should be able to trigger a rolling upgrade of an
>>existing cluster, regardless of how it is set up
>>-It should be extensible enough so that it can be extended to support
>>different ways of setting up HBase (tarballs, packages, parcels, or any
>>other formats that are relevant)
>>-It should be able to report results of various testing operations
>>-It should be easy to add new tests to the suite, particularly by those who
>>have only passing familiarity with rolling upgrade
>>-These tests should be first-class citizens WRT other tests
>>-Should standardize what the minimum set of services is for a rolling
>>upgrade (e.g. master, backup master, 2 regionservers, zk, etc.)
>>-Should support both secure and insecure rolling upgrade
>>
>>*Potential Solution:*
>>I propose using IntegrationTest to bind to an existing remote cluster. It
>>would be able to perform the rolling upgrade steps while running tests over
>>a cluster that is already in place.
>>
>>At a high level, we would first agree on and create a standard workflow for
>>upgrading a cluster. Then, we would work on using IntegrationTest to run
>>tests (e.g. insert data, do mapreduce over hbase jobs, etc.) while the
>>cluster is in flux. The challenge would be adding code that will handle
>>specific cluster setups (e.g. tarballs vs. packages vs. parcels), but we
>>can abstract it away.
>>
>>Please let me know what you think about this idea and if you'd like to work
>>together to accomplish this goal.
>>
>>--
>>Best Regards,
>>
>>Aleks Shulman
>>847.814.5804
>>Cloudera