Return-Path: X-Original-To: apmail-accumulo-dev-archive@www.apache.org Delivered-To: apmail-accumulo-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 791FB10C00 for ; Thu, 18 Jul 2013 01:51:49 +0000 (UTC) Received: (qmail 35865 invoked by uid 500); 18 Jul 2013 01:51:49 -0000 Delivered-To: apmail-accumulo-dev-archive@accumulo.apache.org Received: (qmail 35785 invoked by uid 500); 18 Jul 2013 01:51:49 -0000 Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list dev@accumulo.apache.org Received: (qmail 35776 invoked by uid 99); 18 Jul 2013 01:51:49 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Jul 2013 01:51:49 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of billie.rinaldi@gmail.com designates 209.85.219.47 as permitted sender) Received: from [209.85.219.47] (HELO mail-oa0-f47.google.com) (209.85.219.47) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Jul 2013 01:51:42 +0000 Received: by mail-oa0-f47.google.com with SMTP id m1so3483354oag.6 for ; Wed, 17 Jul 2013 18:51:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=WN+Emc1E87Dz6/caryUl4ZibCbkDoRRRttL4wZMxjpw=; b=ScsayO50D3mnZ0A9QGrQgt5ZrkjABmaMN36bz6qG+A9gcyBKuOGLx48wPt+dyttFrZ FCzJ6GrLF6aVDeSVrzWWlbYUK9vUNxTUsIkDkkol/Q2uA4umuV6dQARwNW2NW1/pJqBY SQ+6rHIN2H98FQuOs5M0bo/WS4X5fJWwA5qn46eMg+GqlFOPLZ8ul1MvbcflQ8U5RasA 242MtgJ1rdHaudjvLP+iczqM/1iKjUI0377kJV6dQXqEJ02+4YXKkUohzIzUA08SwLtT XbvhtL7ZY5TSVTzxWtIQTWzoMfSkNj9C9zyQbArUmgHGl/u1XgkKo6k3j+VLRTNnUHcC fy0w== MIME-Version: 1.0 X-Received: by 10.60.80.165 with SMTP id s5mr10972244oex.55.1374112281520; Wed, 17 Jul 2013 18:51:21 -0700 (PDT) Received: by 10.76.107.228 with HTTP; Wed, 17 Jul 2013 18:51:21 -0700 (PDT) Received: by 10.76.107.228 with HTTP; Wed, 17 Jul 2013 18:51:21 -0700 (PDT) In-Reply-To: References: Date: Wed, 17 Jul 2013 18:51:21 -0700 Message-ID: Subject: Re: Request for mentor for project on ACCUMULO-1197 From: Billie Rinaldi To: dev@accumulo.apache.org Content-Type: multipart/alternative; boundary=089e012299ba6bf00e04e1bf728e X-Virus-Checked: Checked by ClamAV on apache.org --089e012299ba6bf00e04e1bf728e Content-Type: text/plain; charset=ISO-8859-1 It's unfortunate he's on vacation this week. So, you could write the proposal of what you think the integration will look like before talking to him, but there are a couple of risks: maybe he won't want to work with someone, or won't have time to do the part of the work you'd be integrating into during the 3 months of your programme. If you're willing to take the risk, we'll be able to find mentors for you on the Accumulo side. Billie I got a reply from the HDFS list. Seems this person Andrew is actually working on this Jira. Here's the email: http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-dev/201307.mbox/%3CCA%2BqbEUM_ps2Nihqh4d2c2hA6neXJw5Xep3ceMxqFajmL10Adrw%40mail.gmail.com%3E On Thu, Jul 18, 2013 at 12:37 AM, Sreejith Ramakrishnan < sreejith.code@gmail.com> wrote: > Thank you :) Alright. I'll put such an email and get back at the earliest. > > > On Thu, Jul 18, 2013 at 12:21 AM, Billie Rinaldi > wrote: > >> Both. You could also mention that it could be useful for HBase as well, >> but >> that you aren't planning to do the HBase integration as part of your >> project. >> On Jul 17, 2013 2:41 PM, "Sreejith Ramakrishnan" > > >> wrote: >> >> > So, when I'm putting an email in the HDFS list, shall I cite >> ACCUMULO-1197 >> > or HDFS-4680? >> > >> > >> > On Thu, Jul 18, 2013 at 12:03 AM, Billie Rinaldi >> > wrote: >> > >> > > I suspect people may be reluctant due to the unknown nature of making >> > > changes to HDFS. Perhaps if you could find someone to agree to be >> your >> > > mentor on the HDFS side, it would be easier to find an Accumulo mentor >> > for >> > > the part of the project that involves integrating the HDFS changes >> with >> > > Accumulo. >> > > >> > > Billie >> > > >> > > >> > > On Wed, Jul 17, 2013 at 11:23 AM, Sreejith Ramakrishnan < >> > > sreejith.code@gmail.com> wrote: >> > > >> > > > Oh. I saw the ticket. I'll try that direction as well. Also, what do >> > you >> > > > think are my odds at getting a mentor since its too close to the >> > > deadline? >> > > > >> > > > I'm still requesting mentorship since its a formality for the >> > programme. >> > > > Still keeping my hopes up. >> > > > >> > > > >> > > > On Wed, Jul 17, 2013 at 11:45 PM, Billie Rinaldi >> > > > wrote: >> > > > >> > > > > I realize there isn't a lot of time before the deadline, but if >> you >> > are >> > > > > interested in submitting a patch to HDFS I would encourage you to >> > > engage >> > > > > with the HDFS community to see if they are interested in such a >> > feature >> > > > to >> > > > > increase the likelihood of your patch being accepted. Their dev >> > > mailing >> > > > > list is hdfs-dev@hadoop.apache.org. >> > > > > >> > > > > I found one reference to Dapper in the HDFS tickets, HDFS-4680. >> The >> > > > ticket >> > > > > isn't about tracing, but Todd Lipcon comments that he's >> experimented >> > > with >> > > > > adding tracing to Hadoop RPC and asks if people would find that >> > useful. >> > > > > (There doesn't appear to be a reply.) >> > > > > >> > > > > Billie >> > > > > >> > > > > >> > > > > On Wed, Jul 17, 2013 at 11:01 AM, Sreejith Ramakrishnan < >> > > > > sreejith.code@gmail.com> wrote: >> > > > > >> > > > > > I think its a good idea to modify HDFS and Accumulo to support >> > HTrace >> > > > > since >> > > > > > HBase has also started working in this direction. The progress >> made >> > > > could >> > > > > > benefit both projects. And also, HTrace follows the Dapper >> > > conventions. >> > > > > > >> > > > > > I'm not an expert. But from what I read in Google's paper and >> > what's >> > > > > being >> > > > > > provided in HTrace, we can modify HDFS instrumentation to add a >> > > 64-bit >> > > > > > span-id to the RPC when tracing is enabled. And on the receiver >> > side, >> > > > if >> > > > > it >> > > > > > receives a span-id, it shall also be traced with both the parent >> > and >> > > > > child >> > > > > > nodes having the same trace-id. >> > > > > > >> > > > > > What do you think? Please do correct me if I'm wrong. >> > > > > > >> > > > > > P.S: Also, the last date for application for the mentoring ends >> on >> > > 19th >> > > > > > July. So, I request all the experts here to please let me know >> if >> > > > you're >> > > > > > ready to be a mentor. And I assure you, I won't bug you too >> much in >> > > > your >> > > > > > busy scheduler. It's a mandatory rule for the ICFOSS programme - >> > > > > > >> http://community.apache.org/mentoringprogramme-icfoss-pilot.html >> > > > > > >> > > > > > >> > > > > > On Wed, Jul 17, 2013 at 6:01 PM, Keith Turner > > >> > > > wrote: >> > > > > > >> > > > > > > On Wed, Jul 17, 2013 at 7:32 AM, Sreejith Ramakrishnan < >> > > > > > > sreejith.code@gmail.com> wrote: >> > > > > > > >> > > > > > > > Wouldn't modifying HDFS in any way mean forking the project? >> > > > Wouldn't >> > > > > > > that >> > > > > > > > have problems in the long run? >> > > > > > > > >> > > > > > > > >> > > > > > > Forking HDFS is not the way to go. Getting your ideas and >> > patches >> > > > > > accepted >> > > > > > > by HDFS is the way to go. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > > On Wed, Jul 17, 2013 at 12:04 AM, Sreejith Ramakrishnan < >> > > > > > > > sreejith.code@gmail.com> wrote: >> > > > > > > > >> > > > > > > > > Hey, >> > > > > > > > > >> > > > > > > > > I have an idea to tackle this Jira. I've read the dapper >> > paper >> > > > and >> > > > > > > > > currently examining the work done in thrift. Shall I make >> a >> > > > > detailed >> > > > > > > post >> > > > > > > > > here? >> > > > > > > > > >> > > > > > > > > I'm part of the ICFOSS Joint Mentoring Programme. So if >> > someone >> > > > can >> > > > > > > > please >> > > > > > > > > be my mentor, I'll be happy to work with you :) >> > > > > > > > > >> > > > > > > > > [1] >> > > > > http://community.apache.org/mentoringprogramme-icfoss-pilot.html >> > > > > > > > > >> > > > > > > > > Thank you, >> > > > > > > > > Sreejith R >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Tue, Jul 16, 2013 at 8:46 AM, German Gutierrez < >> > > > > > > > gutierrezga00@gmail.com >> > > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > >> On Jul 12, 2013 8:53 PM, "Keith Turner" < >> keith@deenlo.com> >> > > > wrote: >> > > > > > > > >> > >> > > > > > > > >> > On Thu, Jul 11, 2013 at 11:47 AM, Ajay Bhat < >> > > > > > a.ajay.bhat@gmail.com> >> > > > > > > > >> wrote: >> > > > > > > > >> > >> > > > > > > > >> > > Thanks Keith. I am looking into it now. >> > > > > > > > >> > > >> > > > > > > > >> > > Accumulo has done Dapper design based tracing and >> called >> > > it >> > > > > > > > >> Cloudtrace. >> > > > > > > > >> I >> > > > > > > > >> > > am not aware of the ins and outs of this and would >> like >> > to >> > > > > know >> > > > > > > more >> > > > > > > > >> > > indepth about it. Can anyone help out here? >> > > > > > > > >> > > >> > > > > > > > >> > >> > > > > > > > >> > Cloudtrace is an implementation of the ideas mentioned >> in >> > > the >> > > > > > Dapper >> > > > > > > > >> paper. >> > > > > > > > >> > Its only used by Accumulo at this point. The reason >> I >> > > > > mentioned >> > > > > > > the >> > > > > > > > >> > HBase work is so you could asses the status of that >> work. >> > > You >> > > > > > > should >> > > > > > > > >> > determine if any other work besides the HBase and >> Accumulo >> > > > > efforts >> > > > > > > > >> exists. >> > > > > > > > >> > >> > > > > > > > >> > I can think of a few ways to tackle this problem. >> > > > > > > > >> > >> > > > > > > > >> > 1. Modify HDFS to use cloudtrace. Cloudtrace is >> > currently >> > > > > > > > implemented >> > > > > > > > >> on >> > > > > > > > >> > top of thrift, HDFS does not use thrift. >> > > > > > > > >> > 2. Modify HDFS and Accumulo to use HBase tracing. >> > > > > > > > >> > 3. Modify HDFS to support hooks for tracing. Make >> > > cloudtrace >> > > > > use >> > > > > > > > these >> > > > > > > > >> > hooks. >> > > > > > > > >> > 4. Create completely new tracing system >> > > > > > > > >> > >> > > > > > > > >> > What are your thoughts on this project? What do you >> hope >> > to >> > > > > > > > accomplish? >> > > > > > > > >> > What is your timeframe? >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > > I have checked out the HBase trunk also. >> > > > > > > > >> > > >> > > > > > > > >> > > Some additional info about ASF ICFOSS: The >> ASF-ICFOSS >> > > > > > Programme [ >> > > > > > > > >> > > http://icfoss.org/mentor.html] was conducted by >> Luciano >> > > > > > Resende [ >> > > > > > > > >> > > http://people.apache.org/~lresende] on June 21-23rd, >> > 2013 >> > > > and >> > > > > > as >> > > > > > > > >> part of >> > > > > > > > >> > > the Programme, students are encouraged to take up an >> > > Apache >> > > > > JIRA >> > > > > > > as >> > > > > > > > a >> > > > > > > > >> > > project. >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > >> > > On Thu, Jul 11, 2013 at 7:27 PM, Keith Turner < >> > > > > keith@deenlo.com >> > > > > > > >> > > > > > > > >> wrote: >> > > > > > > > >> > > >> > > > > > > > >> > > > Some HBase guys were also looking into doing >> something >> > > w/ >> > > > > > > tracing. >> > > > > > > > >> I >> > > > > > > > >> am >> > > > > > > > >> > > > not sure what has been done, but it would be >> useful to >> > > > look >> > > > > > into >> > > > > > > > >> that. I >> > > > > > > > >> > > > linked HBASE-6449 to the Accumulo ticket. >> > > > > > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > > >> > > > On Wed, Jul 10, 2013 at 8:41 PM, Ajay Bhat < >> > > > > > > a.ajay.bhat@gmail.com >> > > > > > > > > >> > > > > > > > >> > > wrote: >> > > > > > > > >> > > > >> > > > > > > > >> > > > > Hi all, >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > I had attended the ASF-ICFOSS workshop held in >> June. >> > > > > > > > >> > > > > >> > > > > > > > >> > http://community.apache.org/mentoringprogramme-icfoss-pilot.html >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > I'd like to take up this issue as a project. >> > > > > > > > >> > > > > >> https://issues.apache.org/jira/browse/ACCUMULO-1197 >> > > > > > > > >> > > > > Could someone in the community act as a mentor >> for >> > > this >> > > > > > > project? >> > > > > > > > >> The >> > > > > > > > >> > > > > process of selection is very similar to GSoC. >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > I've checked out the source code and used >> Accumulo. >> > I >> > > > hope >> > > > > > to >> > > > > > > > have >> > > > > > > > >> > > made a >> > > > > > > > >> > > > > formal proposal by Saturday, July 13th. I'd be >> happy >> > > to >> > > > > get >> > > > > > > some >> > > > > > > > >> advice >> > > > > > > > >> > > > on >> > > > > > > > >> > > > > how I could get started on it, and also any minor >> > > > issues I >> > > > > > > could >> > > > > > > > >> work >> > > > > > > > >> > > on >> > > > > > > > >> > > > in >> > > > > > > > >> > > > > the interim. >> > > > > > > > >> > > > > >> > > > > > > > >> > > > > Regards, >> > > > > > > > >> > > > > Ajay >> > > > > > > > >> > > > > >> > > > > > > > >> > > > >> > > > > > > > >> > > >> > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > > --089e012299ba6bf00e04e1bf728e--