Return-Path: X-Original-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 359B7C70C for ; Sun, 16 Jun 2013 19:04:55 +0000 (UTC) Received: (qmail 64907 invoked by uid 500); 16 Jun 2013 19:04:54 -0000 Delivered-To: apmail-hadoop-hdfs-dev-archive@hadoop.apache.org Received: (qmail 64833 invoked by uid 500); 16 Jun 2013 19:04:54 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 64825 invoked by uid 99); 16 Jun 2013 19:04:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 16 Jun 2013 19:04:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of acm@hortonworks.com designates 209.85.220.52 as permitted sender) Received: from [209.85.220.52] (HELO mail-pa0-f52.google.com) (209.85.220.52) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 16 Jun 2013 19:04:46 +0000 Received: by mail-pa0-f52.google.com with SMTP id kq13so2167032pab.11 for ; Sun, 16 Jun 2013 12:04:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to:x-mailer:x-gm-message-state; bh=WiQxc4WqlHBwKZSwnH+Ol4Lm3xWHWSvcRae3TPnzHPo=; b=iVvFP6azmzGlPH8ZwZIlbJ8pWxMstt1uiA7FImu5k/qZ4TtkVlORvZw4n3ta4Xx6qt a2XcLFyeDqhlTjUdY9ey9Kfd5/l/a1kANOJ7ASkmw0srJ/y7MFNLUVaUh0UhStf8NTEr gT1fLOeYTVeB311U7Px+y6rAqupXgiFzvGgY2PBMxYTMWmnwRFmwQREdba9+7TgpyYD1 tTnP+rZGRCteZZ52wOhBddo+kBwyN7JV1oseoZAjYerB/tQkt7WJIOFdylNwwS97PEeo 3yyBIm9eUU7Y/b/U2gU1VJ7TYbW7aEe6wg9qqXUwPWYmP9uHU0OvLO6y2RDYgO21d/bx avag== X-Received: by 10.66.4.10 with SMTP id g10mr10271195pag.217.1371409465375; Sun, 16 Jun 2013 12:04:25 -0700 (PDT) Received: from [10.0.1.20] (c-98-234-189-94.hsd1.ca.comcast.net. [98.234.189.94]) by mx.google.com with ESMTPSA id re16sm11649970pac.16.2013.06.16.12.04.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 16 Jun 2013 12:04:24 -0700 (PDT) Subject: Re: Heads up: branch-2.1-beta Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/alternative; boundary=Apple-Mail-16-112755163 From: Arun C Murthy In-Reply-To: <155DD016-461A-409E-A614-E44E22105CD1@open-mpi.org> Date: Sun, 16 Jun 2013 12:04:22 -0700 Cc: yarn-dev@hadoop.apache.org, "mapreduce-dev@hadoop.apache.org" , "hdfs-dev@hadoop.apache.org" Message-Id: References: <809CF17D-1A54-4C64-8AA7-997289D3930D@hortonworks.com> <27CADA79-5559-4709-BA67-01580FC4BB64@hortonworks.com>

<82F00E13-2F48-42CA-8072-5A65C4F6D627@open-mpi.org>

<048B8DD0-16F4-4B05-88E1-212C49056848@hortonworks.com> <155DD016-461A-409E-A614-E44E22105CD1@open-mpi.org> To: common-dev@hadoop.apache.org X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQmr9p+Z9C2FZkR2eKhO370ct+9cW/GW2MF9gAqi6N39vxVZ0V+lrofO+V4TVp5/3qrcsTCf X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail-16-112755163 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii HDFS-4866 was committed a couple of days ago - hence isn't showing up in = my filters. Can you please file a new jira for the linker issues? thanks, Arun On Jun 16, 2013, at 10:16 AM, Ralph Castain wrote: > HDFS-4866 is marked as a blocker. A patch was submitted and applied, = but it only fixes compilation. As I marked on the ticket, the results = remain unusable due to linker issues. >=20 >=20 > On Jun 16, 2013, at 5:33 AM, Arun C Murthy = wrote: >=20 >> Which one are you talking about Ralph? This doesn't show up on any = blocker list. >>=20 >> http://s.apache.org/hadoop-blocker-bugs >>=20 >> Arun >>=20 >>=20 >> On Jun 15, 2013, at 4:44 PM, Ralph Castain wrote: >>=20 >>> Not trying to be a pain, but I am trying to get clarification. The = protocol buffer support is still broken. Do you intend to release 2.1 = with that unfixed? >>>=20 >>>=20 >>> On Jun 15, 2013, at 4:18 PM, Karthik Kambatla = wrote: >>>=20 >>>> Re-posting here to the wider audience: >>>>=20 >>>> HADOOP-9517 (Document Hadoop Compatibility): I believe I have = incorporated >>>> all suggestions so far. It would be great if people could take = another look >>>> at it. I ll iterate fast on any comments so we get this in by the = time rest >>>> of the code pieces are committed. >>>>=20 >>>> Thanks >>>> Karthik >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> On Sat, Jun 15, 2013 at 11:46 AM, Ralph Castain = wrote: >>>>=20 >>>>> Just curious of your procedures. Given that there is at least one = blocker >>>>> JIRA out there that has yet to be fully resolved, do you intend to = release >>>>> anyway? >>>>>=20 >>>>>=20 >>>>> On Jun 15, 2013, at 8:19 AM, Alejandro Abdelnur = wrote: >>>>>=20 >>>>>> If the intention is to get the release out in time for the Hadoop = Summit >>>>> we >>>>>> have a very tight schedule. >>>>>>=20 >>>>>> Because the release vote runs for 7 days, we should have an RC = latest >>>>>> Monday afternoon, and we should encourage folks to verify & vote = ASAP, so >>>>>> if we need to cut a new RC we can do it on Tuesday. Another thing = to >>>>>> consider is that if the changes on an RC are corrections that do = not >>>>> affect >>>>>> code, we could agree on not reseting the voting period clock if = we need >>>>> to >>>>>> cut a new RC (ie doc, build, notes changes). >>>>>>=20 >>>>>> Of the JIRAs in my laundry list for 2.1 the ones I would really = want in >>>>> are >>>>>> YARN-752, MAPREDUCE-5171 & YARN-787. >>>>>>=20 >>>>>> The first 2 are already +1ed, the last one needs to be reviewed. >>>>>>=20 >>>>>> I have not committed the first 2 ones yet because I don't want to = disrupt >>>>>> things for the folks doing QA. >>>>>>=20 >>>>>> Arun, as you are coordinating the work for this release, please = do commit >>>>>> them or give me the go ahead and I'll commit. >>>>>>=20 >>>>>> Also, it would be great if you can review YARN-787 (as per = discussions, >>>>>> the changes on the milli-slot calculations do not affect the = current >>>>>> calculations, that would be left for MAPREDUCE-5311 to do). >>>>>>=20 >>>>>> I'll be checking my email over the weekend and I can take care of = some >>>>>> stuff if needed (while the monkeys sleep). >>>>>>=20 >>>>>> Thx >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> On Fri, Jun 14, 2013 at 9:56 PM, Alejandro Abdelnur = >>>>> wrote: >>>>>>=20 >>>>>>> Following is a revisited assessment of JIRAs I would like to get = in the >>>>>>> 2.1 release: >>>>>>>=20 >>>>>>> =46rom the 1st group I think all 3 should make. >>>>>>>=20 >>>>>>> =46rom the 2nd group I think YARN-791 should make it for sure = and ideally >>>>>>> MAPREDUCE-5130. >>>>>>>=20 >>>>>>> =46rom the 3rd group, I don't think this JIRA will make it. >>>>>>>=20 >>>>>>> =46rom the 4th group, we don't need to worry about this or 2.1 >>>>>>>=20 >>>>>>> Thanks >>>>>>>=20 >>>>>>> Alejandro >>>>>>>=20 >>>>>>> ------------------------------------------------------ >>>>>>> JIRAs that are in shape to make it to 2.1 >>>>>>>=20 >>>>>>> * YARN-752: In AMRMClient, automatically add corresponding rack = requests >>>>>>> for requested nodes >>>>>>>=20 >>>>>>> impact: behavior change >>>>>>>=20 >>>>>>> status: patch avail, +1ed. >>>>>>>=20 >>>>>>> * MAPREDUCE-5171: Expose blacklisted nodes from the MR AM REST = API >>>>>>>=20 >>>>>>> impact: Addition to MRAM HTTP API >>>>>>>=20 >>>>>>> status: patch avail, +1ed, needs to be committed >>>>>>>=20 >>>>>>> * YARN-787: Remove resource min from Yarn client API >>>>>>>=20 >>>>>>> impact: Yarn client API change >>>>>>>=20 >>>>>>> status: patch avail, needs to be reviewed. (the calculation of >>>>> slot-millis >>>>>>> is not affected, the MIN is taken from conf for now) >>>>>>>=20 >>>>>>> ------------------------------------------------------ >>>>>>> JIRAs that require minor work to make it to 2.1 >>>>>>>=20 >>>>>>> * YARN-521: Augment AM - RM client module to be able to request >>>>> containers >>>>>>> only at specific locations >>>>>>>=20 >>>>>>> impact: AMRM client API change >>>>>>>=20 >>>>>>> status: patch not avail yet (requires YARN-752) >>>>>>>=20 >>>>>>> * YARN-791: Ensure that RM RPC APIs that return nodes are = consistent >>>>> with >>>>>>> /nodes REST API >>>>>>>=20 >>>>>>> impact: Yarn client API & proto change >>>>>>>=20 >>>>>>> status: patch avail, review in progress >>>>>>>=20 >>>>>>> * MAPREDUCE-5130: Add missing job config options to = mapred-default.xml >>>>>>>=20 >>>>>>> impact: behavior change >>>>>>>=20 >>>>>>> status: patch avail but some tests are failing >>>>>>>=20 >>>>>>> ------------------------------------------------------ >>>>>>> JIRAs that require significant work to make it to 2.1 and may = not make >>>>> it >>>>>>>=20 >>>>>>> * YARN-649: Make container logs available over HTTP in plain = text >>>>>>>=20 >>>>>>> impact: Addition to NM HTTP REST API. Needed for MAPREDUCE-4362 = (which >>>>>>> does not change API) >>>>>>>=20 >>>>>>> status: patch avail, review in progress >>>>>>>=20 >>>>>>> ------------------------------------------------------ >>>>>>> JIRAs that don't need to make it to 2.1 >>>>>>>=20 >>>>>>> * MAPREDUCE-5311: Remove slot millis computation logic and = deprecate >>>>>>> counter constants >>>>>>>=20 >>>>>>> impact: behavior change >>>>>>>=20 >>>>>>> status: per discussion we should first add memory-millis and >>>>> vcores-millis >>>>>>>=20 >>>>>>> ------------------------------------------------------ >>>>>>>=20 >>>>>>>=20 >>>>>>> On Fri, Jun 14, 2013 at 7:17 PM, Roman Shaposhnik = >>>>> wrote: >>>>>>>=20 >>>>>>>> On Thu, Jun 6, 2013 at 4:48 AM, Arun C Murthy = >>>>>>>> wrote: >>>>>>>>>=20 >>>>>>>>> On Jun 5, 2013, at 11:04 AM, Roman Shaposhnik wrote >>>>>>>>>>=20 >>>>>>>>>> On the Bigtop side of things, once we have stable Bigtop = 0.6.0 >>>>> platform >>>>>>>>>> based on Hadoop 2.0.x codeline we plan to start running the = same >>>>>>>> battery >>>>>>>>>> of integration tests on the branch-2.1-beta. >>>>>>>>>>=20 >>>>>>>>>> We plan to simply file JIRAs if anything gets detected and I = will >>>>> also >>>>>>>>>> publish the URL of the Jenkins job once it gets created. >>>>>>>>>=20 >>>>>>>>> Thanks Roman. Is there an ETA for this? Also, please file = jiras with >>>>>>>> Blocker priority to catch attention. >>>>>>>>=20 >>>>>>>> The build is up and running (and all green on all of the 9 = Linux >>>>>>>> platforms!): >>>>>>>> http://bigtop01.cloudera.org:8080/job/Hadoop-2.1.0/ >>>>>>>>=20 >>>>>>>> The immediate benefit here is that we get to see that the >>>>>>>> build is ok on all these Linuxes and all anybody can easily >>>>>>>> install packaged Hadoop 2.1.0 nightly builds. >>>>>>>>=20 >>>>>>>> Starting from next week, I'll start running regular tests >>>>>>>> on these bits and will keep you guys posted! >>>>>>>>=20 >>>>>>>> Thanks, >>>>>>>> Roman. >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> -- >>>>>>> Alejandro >>>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> -- >>>>>> Alejandro >>>>>=20 >>>>>=20 >>>=20 >>=20 >> -- >> Arun C. Murthy >> Hortonworks Inc. >> http://hortonworks.com/ >>=20 >>=20 >=20 -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ --Apple-Mail-16-112755163--