accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser" <josh.el...@gmail.com>
Subject Re: Review Request 15650: ACCUMULO-1794 adds hdfs failover to continuous integration test.
Date Fri, 22 Nov 2013 19:47:43 GMT


> On Nov. 18, 2013, 6:33 p.m., Josh Elser wrote:
> > test/system/continuous/hdfs-agitator.pl, line 90
> > <https://reviews.apache.org/r/15650/diff/1/?file=388001#file388001line90>
> >
> >     It would be nice to default to running as the invoking user instead of forcing
sudo. If I have discrete users set up for each role, I may not always want to have sudoers
set up.
> >     
> >     Having the ability is definitely nice, though.
> 
> Sean Busbey wrote:
>     haadmin is only runnable as an HDFS super user, and AFAICT the continuous integration
test runs as either the accumulo user or root (for its kill stuff to work on the other components).
>     
>     If people run the agitator script as root, then the sudo is needed to allow the command
to run. If they run the agitator as something other than root, then we need either a sudo
to the accumulo user for the other agitator stuff or one here. Unless the accumulo user is
in the hdfs superuser group. But I don't want to encourage people to add the accumulo user
to the HDFS superuser group.
>     
>     
>     Maybe a docs update is need too?
>     
>     I think it'd be simpler to document "run the agitator as root because it's going
to need access to multiple users". Or is it worth the overhead of properly breaking the testing
out into users-per-role?
> 
> Sean Busbey wrote:
>     Changed to default to run as current user unless --sudo is given. If that fails,
makes an attempt to default to sudo on the path.

Awesome. Sounds good. Agreed that the docs likely need to be updated (or formally written)


> On Nov. 18, 2013, 6:33 p.m., Josh Elser wrote:
> > test/system/continuous/hdfs-agitator.pl, line 120
> > <https://reviews.apache.org/r/15650/diff/1/?file=388001#file388001line120>
> >
> >     Maybe rename this from hdfs-agitator to ha-hdfs-agitator (or similar). agitator.pl
also agitates datanodes so the name is a bit of a misnomer.
> 
> Sean Busbey wrote:
>     I'd rather pull the agitator.pl parts that mess with datanodes here. that way we'd
get better failure testing for when data nodes and tablet servers are not the same set of
machines and we could flex more advanced HDFS failure conditions (like handling rack loss).
>     
>     Sound good?

Yup. Makes sense.


- Josh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15650/#review29059
-----------------------------------------------------------


On Nov. 18, 2013, 5:13 p.m., Sean Busbey wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15650/
> -----------------------------------------------------------
> 
> (Updated Nov. 18, 2013, 5:13 p.m.)
> 
> 
> Review request for accumulo and Alex Moundalexis.
> 
> 
> Bugs: ACCUMULO-1794
>     https://issues.apache.org/jira/browse/ACCUMULO-1794
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> ACCUMULO-1794 adds hdfs failover to continuous integration test.
> 
> 
> Diffs
> -----
> 
>   test/system/continuous/continuous-env.sh.example 830ae86b5bf2398a840b853423755f6dd65f2dc0

>   test/system/continuous/hdfs-agitator.pl PRE-CREATION 
>   test/system/continuous/start-agitator.sh 52e5a4e82a4564fa624a71f73ad29fa20ba23246 
>   test/system/continuous/stop-agitator.sh b853a55b12f8402606af52e0748ca50daf95ed7f 
> 
> Diff: https://reviews.apache.org/r/15650/diff/
> 
> 
> Testing
> -------
> 
> Ran the hdfs agitator on a CDH4 cluster configured for HA. it successfully caused the
active namenode to failover as it went.
> 
> 
> Thanks,
> 
> Sean Busbey
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message