Mailing-List: contact chukwa-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-dev@incubator.apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 list@toolazydogs.com does not designate 209.85.160.45 as permitted sender)
Content-Type: text/plain; charset=GB2312
Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\))
Subject: Re: Incubator PMC/Board report for Jul 2013 ([ppmc])
From: "Alan D. Cabrera" <list@toolazydogs.com>
In-Reply-To: 
 <CADfbTpF9416GrAPMJwVORx=bvCGUKRLkQr0brmJgdEPzdroRXQ@mail.gmail.com>
Date: Mon, 8 Jul 2013 22:51:43 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <2B477B85-FB72-490E-AEE8-00546703AD3E@toolazydogs.com>
References: <20130701151146.F056A10FB3@minotaur.apache.org>
 <CADfbTpHZaypz4Yf=5QRhrX_y33P+JTHM+t6gdxBbtLwHUXELjg@mail.gmail.com>
 <CADfbTpEY_CzOYQiGYVabJJ50mEUDBxoqeOMYZGU1GyUZ3bx=nQ@mail.gmail.com>
 <096F75B3-7403-451C-8408-DAB9D8F337D7@toolazydogs.com>
 <CADfbTpEM7n-hSjc3syWeKFBB6aFtdDHn_72daKvddY5=sqNNNA@mail.gmail.com>
 <0D714EEC-673C-4DB8-8CA4-B6D6239586A7@gmail.com>
 <060C0631-B620-4780-A3EF-C2B945009BB9@toolazydogs.com>
 <CADfbTpF9416GrAPMJwVORx=bvCGUKRLkQr0brmJgdEPzdroRXQ@mail.gmail.com>
To: chukwa-dev@incubator.apache.org

I think it might be better to change it to what you had before.  :)

Chukwa code needs to be maintained to keep up with new versions of =
Hadoop.


Regards,
Alan

On Jul 8, 2013, at 9:00 PM, Eric Yang <eric818@gmail.com> wrote:

> When using Chukwa as a generic log collection framework.  There is no
> change necessary.  When using Chukwa to monitor Hadoop cluster, then =
the
> data source to collect the data is different between Hadoop 1.x and =
2.x.
> In our report, this is meant to apply our changes to monitor Hadoop =
2.x.
> I have done some experimentation with Hadoop 2.x with Chukwa trunk.  =
Some
> test cases failed because Jobtracker does not exist in 2.x, hence =
Hadoop
> data collection needs to be updated.  We probably should aim for =
making
> those changes compatible with Hadoop 2.x.
>=20
> regards,
> Eric
>=20
>=20
> On Mon, Jul 8, 2013 at 7:29 PM, Alan D. Cabrera =
<list@toolazydogs.com>wrote:
>=20
>> Awesome!  I'll update the report!
>>=20
>>=20
>> Regards,
>> Alan
>>=20
>> On Jul 8, 2013, at 6:33 PM, =CC=C6 =D3=B1 <ivytang0812@gmail.com> =
wrote:
>>=20
>>> Hi Eric=A3=AC
>>>=20
>>> My team is now migrating our m/r from hadoop 1.0 to 2. 0 .
>>> And the old m/r can run on hadoop 2.0 without changing any source =
code ,
>> just need to re-compile on hadoop 2.0.
>>> The yarn in not a necessary if you want to run mr from hadoop 1.0 on
>> hadoop 2.0.
>>> --
>>> Best regards,
>>>=20
>>> Ivy Tang
>>>=20
>>> On 2013-7-8, at =C9=CF=CE=E71:03, Eric Yang <eric818@gmail.com> =
wrote:
>>>=20
>>>> Hadoop has evolved in a non straight forward release branches.  =
Chukwa
>>>> currently works with Hadoop 1.x branch.  For moving forward to =
Hadoop
>> 2.x,
>>>> the mapreduce jobs need to be using the new Yarn framework to =
implement
>>>> some of the missing pieces that Hadoop broke the compatibility.  =
HBase
>> has
>>>> done something similar for Hadoop compatibility layer.  Chukwa may =
have
>> to
>>>> do the same thing to keep up to date.  I don't know how to quantify =
the
>> man
>>>> hours needed to keep up to date.  It may be one or two weeks to =
work on
>> it
>>>> full time.  It may spread across several months, when we work on =
this
>> part
>>>> time.
>>>>=20
>>>> The instrumentation points in MR v1 will needs to be migrated and
>>>> instrument Yarn.  It is going to be interesting work to monitor and
>> analyze
>>>> Yarn.  Therefore, the work may be interesting to people that would =
like
>> to
>>>> visualize yarn performance.  We probably should release 0.6 as =
bugfix
>>>> release before working on this.
>>>>=20
>>>> regards,
>>>> Eric
>>>>=20
>>>>=20
>>>> On Sun, Jul 7, 2013 at 9:27 AM, Alan D. Cabrera =
<list@toolazydogs.com
>>> wrote:
>>>>=20
>>>>>=20
>>>>> On Jul 6, 2013, at 7:52 PM, Eric Yang <eric818@gmail.com> wrote:
>>>>>=20
>>>>>> ChukwaChukwa is a log collection and analysis framework for =
Apache
>>>>> Hadoop clusters.Chukwa has been incubating since 2010-07-14.
>>>>>>=20
>>>>>> Three most important issues to address in the move towards =
graduation:
>>>>>>=20
>>>>>> 1. Keep code up-to-date with Hadoop trend
>>>>>> 2. Make another release
>>>>>> 3. Keep wiki page up to date and better documentation
>>>>>=20
>>>>> IMO, the number one item should be increasing an active community.
>>>>>=20
>>>>>> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need =
to be
>>>>> aware of?
>>>>>>=20
>>>>>> Chukwa code needs to be maintained to keep up with new versions =
of
>>>>> Hadoop.
>>>>>=20
>>>>> Can you characterize what is the scope of the work that needs to =
be
>> done?
>>>>> Especially in terms of estimated man hours needed to bring Chukwa =
up to
>>>>> date.
>>>>>=20
>>>>>> How has the community developed since the last report?
>>>>>>=20
>>>>>> Committers are communicate more actively than previous quarter.
>>>>>>=20
>>>>>> How has the project developed since the last report?
>>>>>>=20
>>>>>> More contributors have open JIRA to request features and =
experiment
>> with
>>>>> code base.  Website is now svnpubsub enabled.  Javadocs are
>> regenerated to
>>>>> remove frame injection vulnerability.
>>>>>>=20
>>>>>> Date of last release: January 2012
>>>>>>=20
>>>>>> Draft report attached for the group to review.  In Chukwa 0.6
>> release, we
>>>>>> have 2 outstanding issues, and hope the community will help out =
to
>> make
>>>>>> this release possible.  I also updated the team list to reflect =
the
>>>>> current
>>>>>> state.  Let me know if there is any questions or comments.  =
Thanks
>>>>>>=20
>>>>>> regards,
>>>>>> Eric
>>>>>=20
>>>>>=20
>>>=20
>>=20
>>=20