accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billie Rinaldi <billie.rina...@gmail.com>
Subject Re: Request for mentor for project on ACCUMULO-1197
Date Thu, 18 Jul 2013 01:51:21 GMT
It's unfortunate he's on vacation this week.  So, you could write the
proposal of what you think the integration will look like before talking to
him, but there are a couple of risks: maybe he won't want to work with
someone, or won't have time to do the part of the work you'd be integrating
into during the 3 months of your programme.

If you're willing to take the risk, we'll be able to find mentors for you
on the Accumulo side.

Billie
I got a reply from the HDFS list. Seems this person Andrew is actually
working on this Jira. Here's the email:

http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201307.mbox/%3CCA%2BqbEUM_ps2Nihqh4d2c2hA6neXJw5Xep3ceMxqFajmL10Adrw%40mail.gmail.com%3E


On Thu, Jul 18, 2013 at 12:37 AM, Sreejith Ramakrishnan <
sreejith.code@gmail.com> wrote:

> Thank you :) Alright. I'll put such an email and get back at the earliest.
>
>
> On Thu, Jul 18, 2013 at 12:21 AM, Billie Rinaldi <billie.rinaldi@gmail.com
> > wrote:
>
>> Both. You could also mention that it could be useful for HBase as well,
>> but
>> that you aren't planning to do the HBase integration as part of your
>> project.
>> On Jul 17, 2013 2:41 PM, "Sreejith Ramakrishnan" <sreejith.code@gmail.com
>> >
>> wrote:
>>
>> > So, when I'm putting an email in the HDFS list, shall I cite
>> ACCUMULO-1197
>> > or HDFS-4680?
>> >
>> >
>> > On Thu, Jul 18, 2013 at 12:03 AM, Billie Rinaldi
>> > <billie.rinaldi@gmail.com>wrote:
>> >
>> > > I suspect people may be reluctant due to the unknown nature of making
>> > > changes to HDFS.  Perhaps if you could find someone to agree to be
>> your
>> > > mentor on the HDFS side, it would be easier to find an Accumulo
mentor
>> > for
>> > > the part of the project that involves integrating the HDFS changes
>> with
>> > > Accumulo.
>> > >
>> > > Billie
>> > >
>> > >
>> > > On Wed, Jul 17, 2013 at 11:23 AM, Sreejith Ramakrishnan <
>> > > sreejith.code@gmail.com> wrote:
>> > >
>> > > > Oh. I saw the ticket. I'll try that direction as well. Also, what
do
>> > you
>> > > > think are my odds at getting a mentor since its too close to the
>> > > deadline?
>> > > >
>> > > > I'm still requesting mentorship since its a formality for the
>> > programme.
>> > > > Still keeping my hopes up.
>> > > >
>> > > >
>> > > > On Wed, Jul 17, 2013 at 11:45 PM, Billie Rinaldi
>> > > > <billie.rinaldi@gmail.com>wrote:
>> > > >
>> > > > > I realize there isn't a lot of time before the deadline, but
if
>> you
>> > are
>> > > > > interested in submitting a patch to HDFS I would encourage you
to
>> > > engage
>> > > > > with the HDFS community to see if they are interested in such
a
>> > feature
>> > > > to
>> > > > > increase the likelihood of your patch being accepted.  Their
dev
>> > > mailing
>> > > > > list is hdfs-dev@hadoop.apache.org.
>> > > > >
>> > > > > I found one reference to Dapper in the HDFS tickets, HDFS-4680.
>>  The
>> > > > ticket
>> > > > > isn't about tracing, but Todd Lipcon comments that he's
>> experimented
>> > > with
>> > > > > adding tracing to Hadoop RPC and asks if people would find that
>> > useful.
>> > > > > (There doesn't appear to be a reply.)
>> > > > >
>> > > > > Billie
>> > > > >
>> > > > >
>> > > > > On Wed, Jul 17, 2013 at 11:01 AM, Sreejith Ramakrishnan <
>> > > > > sreejith.code@gmail.com> wrote:
>> > > > >
>> > > > > > I think its a good idea to modify HDFS and Accumulo to support
>> > HTrace
>> > > > > since
>> > > > > > HBase has also started working in this direction. The progress
>> made
>> > > > could
>> > > > > > benefit both projects. And also, HTrace follows the Dapper
>> > > conventions.
>> > > > > >
>> > > > > > I'm not an expert. But from what I read in Google's paper
and
>> > what's
>> > > > > being
>> > > > > > provided in HTrace, we can modify HDFS instrumentation to
add a
>> > > 64-bit
>> > > > > > span-id to the RPC when tracing is enabled. And on the receiver
>> > side,
>> > > > if
>> > > > > it
>> > > > > > receives a span-id, it shall also be traced with both the
parent
>> > and
>> > > > > child
>> > > > > > nodes having the same trace-id.
>> > > > > >
>> > > > > > What do you think? Please do correct me if I'm wrong.
>> > > > > >
>> > > > > > P.S: Also, the last date for application for the mentoring
ends
>> on
>> > > 19th
>> > > > > > July. So, I request all the experts here to please let me
know
>> if
>> > > > you're
>> > > > > > ready to be a mentor. And I assure you, I won't bug you
too
>> much in
>> > > > your
>> > > > > > busy scheduler. It's a mandatory rule for the ICFOSS programme
-
>> > > > > >
>> http://community.apache.org/mentoringprogramme-icfoss-pilot.html
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Jul 17, 2013 at 6:01 PM, Keith Turner <keith@deenlo.com
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > > On Wed, Jul 17, 2013 at 7:32 AM, Sreejith Ramakrishnan
<
>> > > > > > > sreejith.code@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Wouldn't modifying HDFS in any way mean forking
the
project?
>> > > > Wouldn't
>> > > > > > > that
>> > > > > > > > have problems in the long run?
>> > > > > > > >
>> > > > > > > >
>> > > > > > > Forking HDFS is not the way to go.  Getting your ideas
and
>> > patches
>> > > > > > accepted
>> > > > > > > by HDFS is the way to go.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > >
>> > > > > > > > On Wed, Jul 17, 2013 at 12:04 AM, Sreejith Ramakrishnan
<
>> > > > > > > > sreejith.code@gmail.com> wrote:
>> > > > > > > >
>> > > > > > > > > Hey,
>> > > > > > > > >
>> > > > > > > > > I have an idea to tackle this Jira. I've
read the dapper
>> > paper
>> > > > and
>> > > > > > > > > currently examining the work done in thrift.
Shall I make
>> a
>> > > > > detailed
>> > > > > > > post
>> > > > > > > > > here?
>> > > > > > > > >
>> > > > > > > > > I'm part of the ICFOSS Joint Mentoring Programme.
So if
>> > someone
>> > > > can
>> > > > > > > > please
>> > > > > > > > > be my mentor, I'll be happy to work with
you :)
>> > > > > > > > >
>> > > > > > > > > [1]
>> > > > > http://community.apache.org/mentoringprogramme-icfoss-pilot.html
>> > > > > > > > >
>> > > > > > > > > Thank you,
>> > > > > > > > > Sreejith R
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Tue, Jul 16, 2013 at 8:46 AM, German Gutierrez
<
>> > > > > > > > gutierrezga00@gmail.com
>> > > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >> On Jul 12, 2013 8:53 PM, "Keith Turner"
<
>> keith@deenlo.com>
>> > > > wrote:
>> > > > > > > > >> >
>> > > > > > > > >> > On Thu, Jul 11, 2013 at 11:47 AM,
Ajay Bhat <
>> > > > > > a.ajay.bhat@gmail.com>
>> > > > > > > > >> wrote:
>> > > > > > > > >> >
>> > > > > > > > >> > > Thanks Keith. I am looking
into it now.
>> > > > > > > > >> > >
>> > > > > > > > >> > > Accumulo has done Dapper design
based tracing and
>> called
>> > > it
>> > > > > > > > >> Cloudtrace.
>> > > > > > > > >> I
>> > > > > > > > >> > > am not aware of the ins and
outs of this and would
>> like
>> > to
>> > > > > know
>> > > > > > > more
>> > > > > > > > >> > > indepth about it. Can anyone
help out here?
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >> > Cloudtrace is an implementation
of the ideas mentioned
>> in
>> > > the
>> > > > > > Dapper
>> > > > > > > > >> paper.
>> > > > > > > > >> >  Its only used by Accumulo at this
point.   The reason
>> I
>> > > > > mentioned
>> > > > > > > the
>> > > > > > > > >> > HBase work is so you could asses
the status of that
>> work.
>> > >  You
>> > > > > > > should
>> > > > > > > > >> > determine if any other work besides
the HBase and
>> Accumulo
>> > > > > efforts
>> > > > > > > > >> exists.
>> > > > > > > > >> >
>> > > > > > > > >> > I can think of a few ways to tackle
this problem.
>> > > > > > > > >> >
>> > > > > > > > >> >  1. Modify HDFS to use cloudtrace.
 Cloudtrace is
>> > currently
>> > > > > > > > implemented
>> > > > > > > > >> on
>> > > > > > > > >> > top of thrift, HDFS does not use
thrift.
>> > > > > > > > >> >  2. Modify HDFS and Accumulo to
use HBase  tracing.
>> > > > > > > > >> >  3. Modify HDFS to support hooks
for tracing.  Make
>> > > cloudtrace
>> > > > > use
>> > > > > > > > these
>> > > > > > > > >> > hooks.
>> > > > > > > > >> >  4. Create completely new tracing
system
>> > > > > > > > >> >
>> > > > > > > > >> > What are your thoughts on this project?
 What do you
>> hope
>> > to
>> > > > > > > > accomplish?
>> > > > > > > > >> > What is your timeframe?
>> > > > > > > > >> >
>> > > > > > > > >> >
>> > > > > > > > >> > > I have checked out the HBase
trunk also.
>> > > > > > > > >> > >
>> > > > > > > > >> > >  Some additional info about
ASF ICFOSS: The
>> ASF-ICFOSS
>> > > > > > Programme [
>> > > > > > > > >> > > http://icfoss.org/mentor.html]
was conducted by
>> Luciano
>> > > > > > Resende [
>> > > > > > > > >> > > http://people.apache.org/~lresende]
on June 21-23rd,
>> > 2013
>> > > > and
>> > > > > > as
>> > > > > > > > >> part of
>> > > > > > > > >> > > the Programme, students are
encouraged to take up an
>> > > Apache
>> > > > > JIRA
>> > > > > > > as
>> > > > > > > > a
>> > > > > > > > >> > > project.
>> > > > > > > > >> > >
>> > > > > > > > >> > >
>> > > > > > > > >> > > On Thu, Jul 11, 2013 at 7:27
PM, Keith Turner <
>> > > > > keith@deenlo.com
>> > > > > > >
>> > > > > > > > >> wrote:
>> > > > > > > > >> > >
>> > > > > > > > >> > > > Some HBase guys were also
looking into doing
>> something
>> > > w/
>> > > > > > > tracing.
>> > > > > > > > >>  I
>> > > > > > > > >> am
>> > > > > > > > >> > > > not sure what has been
done, but it would be
>> useful to
>> > > > look
>> > > > > > into
>> > > > > > > > >> that.  I
>> > > > > > > > >> > > > linked HBASE-6449 to the
Accumulo ticket.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > On Wed, Jul 10, 2013 at
8:41 PM, Ajay Bhat <
>> > > > > > > a.ajay.bhat@gmail.com
>> > > > > > > > >
>> > > > > > > > >> > > wrote:
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > > Hi all,
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > I had attended the
ASF-ICFOSS workshop held in
>> June.
>> > > > > > > > >> > > > >
>> > > > > > > >
>> > http://community.apache.org/mentoringprogramme-icfoss-pilot.html
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > I'd like to take
up this issue as a project.
>> > > > > > > > >> > > > >
>> https://issues.apache.org/jira/browse/ACCUMULO-1197
>> > > > > > > > >> > > > > Could someone in
the community act as a mentor
>> for
>> > > this
>> > > > > > > project?
>> > > > > > > > >> The
>> > > > > > > > >> > > > > process of selection
is very similar to GSoC.
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > I've checked out
the source code and used
>> Accumulo.
>> > I
>> > > > hope
>> > > > > > to
>> > > > > > > > have
>> > > > > > > > >> > > made a
>> > > > > > > > >> > > > > formal proposal by
Saturday, July 13th. I'd be
>> happy
>> > > to
>> > > > > get
>> > > > > > > some
>> > > > > > > > >> advice
>> > > > > > > > >> > > > on
>> > > > > > > > >> > > > > how I could get started
on it, and also any
minor
>> > > > issues I
>> > > > > > > could
>> > > > > > > > >> work
>> > > > > > > > >> > > on
>> > > > > > > > >> > > > in
>> > > > > > > > >> > > > > the interim.
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > Regards,
>> > > > > > > > >> > > > > Ajay
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message