chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: [DISCUSSION] Chukwa 0.6 + Hadoop integration
Date Mon, 02 Dec 2013 05:52:11 GMT
HDFS API has changed a little, and it does not impact Chukwa integration.
 For mapreduce, there are some changes which requires demux job to be
written differently.  This is mainly more code to be written to schedule
map reduce job in YARN framework.  The instrumentation of collecting data
from YARN, will need designs to collect useful data in context of
monitoring YARN.  I agree that 1.x branch seems like a better target for
Chukwa 0.6.

regards,
Eric

On Sun, Dec 1, 2013 at 5:00 PM, Ariel Rabkin <asrabkin@gmail.com> wrote:

> Howdy.
>
> I think option 1 is fine -- let's release what we have for now. My
> impression is that there are still a lot of users on the 1xx branch.
>
> I've been a bit out of the loop on Hadoop development and don't have a
> sense how much pain it is to target different Hadoop versions. Is the
> issue entirely the demux jobs, or did the filesystem APIs also change
> in incompatible ways?
>
> --Ari
>
> On Thu, Nov 28, 2013 at 2:15 PM, Eric Yang <eyang@apache.org> wrote:
> > Hi all,
> >
> > I like to collect thoughts on plan for branching for Chukwa 0.6 release.
> >  We have four options:
> >
> > 1. Chukwa 0.6 target for Hadoop 1.x branch.
> > 2. Chukwa 0.6 target for Hadoop 2.x branch.
> > 3. Chukwa 0.6 target to work with either Hadoop 1.x or Hadoop 2.x with
> > maven profile switching.  Default to Hadoop 1.x branch.
> > 4. Chukwa 0.6 target to work with either Hadoop 1.x or Hadoop 2.x with
> > maven profile switching.  Default to Hadoop 2.x branch.
> >
> > Option 2 and 4 may put Chukwa on a longer development path to integrate
> > with YARN for map reduce job.
> >
> > My personal preference is option 1 to keep everything simple to help
> > release often.
> >
> > regards,
> > Eric
>
>
>
> --
> Ari Rabkin asrabkin@gmail.com
> Princeton Computer Science Department
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message