accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sreejith Ramakrishnan <sreejith.c...@gmail.com>
Subject Re: Request for mentor for project on ACCUMULO-1197
Date Wed, 17 Jul 2013 18:01:27 GMT
I think its a good idea to modify HDFS and Accumulo to support HTrace since
HBase has also started working in this direction. The progress made could
benefit both projects. And also, HTrace follows the Dapper conventions.

I'm not an expert. But from what I read in Google's paper and what's being
provided in HTrace, we can modify HDFS instrumentation to add a 64-bit
span-id to the RPC when tracing is enabled. And on the receiver side, if it
receives a span-id, it shall also be traced with both the parent and child
nodes having the same trace-id.

What do you think? Please do correct me if I'm wrong.

P.S: Also, the last date for application for the mentoring ends on 19th
July. So, I request all the experts here to please let me know if you're
ready to be a mentor. And I assure you, I won't bug you too much in your
busy scheduler. It's a mandatory rule for the ICFOSS programme -
http://community.apache.org/mentoringprogramme-icfoss-pilot.html


On Wed, Jul 17, 2013 at 6:01 PM, Keith Turner <keith@deenlo.com> wrote:

> On Wed, Jul 17, 2013 at 7:32 AM, Sreejith Ramakrishnan <
> sreejith.code@gmail.com> wrote:
>
> > Wouldn't modifying HDFS in any way mean forking the project? Wouldn't
> that
> > have problems in the long run?
> >
> >
> Forking HDFS is not the way to go.  Getting your ideas and patches accepted
> by HDFS is the way to go.
>
>
>
> >
> > On Wed, Jul 17, 2013 at 12:04 AM, Sreejith Ramakrishnan <
> > sreejith.code@gmail.com> wrote:
> >
> > > Hey,
> > >
> > > I have an idea to tackle this Jira. I've read the dapper paper and
> > > currently examining the work done in thrift. Shall I make a detailed
> post
> > > here?
> > >
> > > I'm part of the ICFOSS Joint Mentoring Programme. So if someone can
> > please
> > > be my mentor, I'll be happy to work with you :)
> > >
> > > [1] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> > >
> > > Thank you,
> > > Sreejith R
> > >
> > >
> > > On Tue, Jul 16, 2013 at 8:46 AM, German Gutierrez <
> > gutierrezga00@gmail.com
> > > > wrote:
> > >
> > >> On Jul 12, 2013 8:53 PM, "Keith Turner" <keith@deenlo.com> wrote:
> > >> >
> > >> > On Thu, Jul 11, 2013 at 11:47 AM, Ajay Bhat <a.ajay.bhat@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Thanks Keith. I am looking into it now.
> > >> > >
> > >> > > Accumulo has done Dapper design based tracing and called it
> > >> Cloudtrace.
> > >> I
> > >> > > am not aware of the ins and outs of this and would like to know
> more
> > >> > > indepth about it. Can anyone help out here?
> > >> > >
> > >> >
> > >> > Cloudtrace is an implementation of the ideas mentioned in the Dapper
> > >> paper.
> > >> >  Its only used by Accumulo at this point.   The reason I mentioned
> the
> > >> > HBase work is so you could asses the status of that work.  You
> should
> > >> > determine if any other work besides the HBase and Accumulo efforts
> > >> exists.
> > >> >
> > >> > I can think of a few ways to tackle this problem.
> > >> >
> > >> >  1. Modify HDFS to use cloudtrace.  Cloudtrace is currently
> > implemented
> > >> on
> > >> > top of thrift, HDFS does not use thrift.
> > >> >  2. Modify HDFS and Accumulo to use HBase  tracing.
> > >> >  3. Modify HDFS to support hooks for tracing.  Make cloudtrace use
> > these
> > >> > hooks.
> > >> >  4. Create completely new tracing system
> > >> >
> > >> > What are your thoughts on this project?  What do you hope to
> > accomplish?
> > >> > What is your timeframe?
> > >> >
> > >> >
> > >> > > I have checked out the HBase trunk also.
> > >> > >
> > >> > >  Some additional info about ASF ICFOSS: The ASF-ICFOSS Programme
[
> > >> > > http://icfoss.org/mentor.html] was conducted by Luciano Resende
[
> > >> > > http://people.apache.org/~lresende] on June 21-23rd, 2013 and
as
> > >> part of
> > >> > > the Programme, students are encouraged to take up an Apache JIRA
> as
> > a
> > >> > > project.
> > >> > >
> > >> > >
> > >> > > On Thu, Jul 11, 2013 at 7:27 PM, Keith Turner <keith@deenlo.com>
> > >> wrote:
> > >> > >
> > >> > > > Some HBase guys were also looking into doing something w/
> tracing.
> > >>  I
> > >> am
> > >> > > > not sure what has been done, but it would be useful to look
into
> > >> that.  I
> > >> > > > linked HBASE-6449 to the Accumulo ticket.
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Jul 10, 2013 at 8:41 PM, Ajay Bhat <
> a.ajay.bhat@gmail.com
> > >
> > >> > > wrote:
> > >> > > >
> > >> > > > > Hi all,
> > >> > > > >
> > >> > > > > I had attended the ASF-ICFOSS workshop held in June.
> > >> > > > >
> > http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> > >> > > > >
> > >> > > > > I'd like to take up this issue as a project.
> > >> > > > > https://issues.apache.org/jira/browse/ACCUMULO-1197
> > >> > > > > Could someone in the community act as a mentor for
this
> project?
> > >> The
> > >> > > > > process of selection is very similar to GSoC.
> > >> > > > >
> > >> > > > > I've checked out the source code and used Accumulo.
I hope to
> > have
> > >> > > made a
> > >> > > > > formal proposal by Saturday, July 13th. I'd be happy
to get
> some
> > >> advice
> > >> > > > on
> > >> > > > > how I could get started on it, and also any minor issues
I
> could
> > >> work
> > >> > > on
> > >> > > > in
> > >> > > > > the interim.
> > >> > > > >
> > >> > > > > Regards,
> > >> > > > > Ajay
> > >> > > > >
> > >> > > >
> > >> > >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message