accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: [DISCUSS] Establishing a contrib repo for upgrade testing
Date Fri, 06 Mar 2015 19:57:40 GMT
On Fri, Mar 6, 2015 at 1:03 PM, Josh Elser <josh.elser@gmail.com> wrote:

> First off, thanks for the good-will in taking the time to ask.
>
> My biggest concern in adopting it as a codebase would be ensuring that it
> isn't another codebase dropped into contrib/ and subsequently ignored. How
> do you plan to avoid this? Who do you see maintaining and running these
> tests?
>
>
I have not had a chance to take a look at the code yet.  If it works well,
I would  run it for RCs.


> Some more targeted implementation observations/questions -
>
> * Do you plan to update the scripts to work with Apache Accumulo instead
> of CDH specific artifacts? e.g. [1]
>
> * For the MapReduce job specifically, why did you write your own and not
> use an existing "vetted" job like Continuous Ingest? Is there something
> that the included M/R job does which is not already contained by our CI
> ingest and verify jobs?
>
> * It looks like the current script only works for 1.4 to 1.6? Do you plan
> to support 1.5->1.6, 1.5->1.7, 1.6->1.7? How do you envision this adoption
> occurring?
>
> * As far as exercising internal Accumulo implementation, I think you have
> the basics covered. What about some more tricky things over the metadata
> table (clone, import, export, merge, split table)? How might additional
> functionality be added in a way that can be automatically tested?
>
> * It seems like you have also targeted a physical set of nodes. Have you
> considered actually using some virtualization platform (e.g. vagrant) to
> fully automate upgrade-testing? If there is a way that a user can spin up a
> few VMs to do the testing, the barrier to entry is much lower (and likely
> more foolproof) than requiring the user to set up the environment.
>
> Thanks, Sean.
>
> - Josh
>
> [1] https://github.com/cloudera/accumulo-upgrade-test/blob/
> master/upgrade-test.sh#L35
>
> Sean Busbey wrote:
>
>> Hi Accumulo!
>>
>> Cloudera has recently made public some code used for doing correctness
>> testing for Accumulo installations across upgrades[1]. The project
>> contains
>> simple data load and verification tools as well as a rudimentary upgrade
>> test automation script.
>>
>> We would like to donate this code to the ASF and use it as a starting
>> place
>> for a contrib repository focused on testing Accumulo across versions
>> generally. If the PMC is interested, I'd be happy to take point on the
>> necessary paperwork for IP clearance[2] following an appropriate vote.
>>
>> Though simple, this upgrade testing has already found issues around 1.4 ->
>> 1.5 ->  1.6 and 1.4 ->1.6 upgrades in the past. There are some obvious
>> points of clean up that need to happen (vendor specific references in
>> docs,
>> artifacts, and packages), and I'd be happy to take point on that work as
>> well. I think it would make a good foundational layer for our future needs
>> and would give us a place where we could revitalize the work from
>> ACCUMULO-2145.
>>
>> Thoughts?
>>
>> [1]: https://github.com/cloudera/accumulo-upgrade-test/
>> [2]: http://incubator.apache.org/ip-clearance/
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message