hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Heads up: branch-2.1-beta
Date Sun, 16 Jun 2013 12:41:17 GMT
Karthik,

 My apologies - I'll definitely spend time on this *today*.

Arun

On Jun 15, 2013, at 4:18 PM, Karthik Kambatla wrote:

> Re-posting here to the wider audience:
> 
> HADOOP-9517 (Document Hadoop Compatibility): I believe I have incorporated
> all suggestions so far. It would be great if people could take another look
> at it. I ll iterate fast on any comments so we get this in by the time rest
> of the code pieces are committed.
> 
> Thanks
> Karthik
> 
> 
> 
> 
> On Sat, Jun 15, 2013 at 11:46 AM, Ralph Castain <rhc@open-mpi.org> wrote:
> 
>> Just curious of your procedures. Given that there is at least one blocker
>> JIRA out there that has yet to be fully resolved, do you intend to release
>> anyway?
>> 
>> 
>> On Jun 15, 2013, at 8:19 AM, Alejandro Abdelnur <tucu@cloudera.com> wrote:
>> 
>>> If the intention is to get the release out in time for the Hadoop Summit
>> we
>>> have a very tight schedule.
>>> 
>>> Because the release vote runs for 7 days, we should have an RC latest
>>> Monday afternoon, and we should encourage folks to verify & vote ASAP, so
>>> if we need to cut a new RC we can do it on Tuesday. Another thing to
>>> consider is that if the changes on an RC are corrections that do not
>> affect
>>> code, we could agree on not reseting the voting period clock if we need
>> to
>>> cut a new RC (ie doc, build, notes changes).
>>> 
>>> Of the JIRAs in my laundry list for 2.1 the ones I would really want in
>> are
>>> YARN-752, MAPREDUCE-5171 & YARN-787.
>>> 
>>> The first 2 are already +1ed, the last one needs to be reviewed.
>>> 
>>> I have not committed the first 2 ones yet because I don't want to disrupt
>>> things for the folks doing QA.
>>> 
>>> Arun, as you are coordinating the work for this release, please do commit
>>> them or give me the go ahead and I'll commit.
>>> 
>>> Also, it  would be great if you can review YARN-787 (as per discussions,
>>> the changes on the milli-slot calculations do not affect the current
>>> calculations, that would be left for MAPREDUCE-5311 to do).
>>> 
>>> I'll be checking my email over the weekend and I can take care of some
>>> stuff if needed (while the monkeys sleep).
>>> 
>>> Thx
>>> 
>>> 
>>> 
>>> On Fri, Jun 14, 2013 at 9:56 PM, Alejandro Abdelnur <tucu@cloudera.com
>>> wrote:
>>> 
>>>> Following is a revisited assessment of JIRAs I would like to get in the
>>>> 2.1 release:
>>>> 
>>>> From the 1st group I think all 3 should make.
>>>> 
>>>> From the 2nd group I think YARN-791 should make it for sure and ideally
>>>> MAPREDUCE-5130.
>>>> 
>>>> From the 3rd group, I don't think this JIRA will make it.
>>>> 
>>>> From the 4th group, we don't need to worry about this or 2.1
>>>> 
>>>> Thanks
>>>> 
>>>> Alejandro
>>>> 
>>>> ------------------------------------------------------
>>>> JIRAs that are in shape to make it to 2.1
>>>> 
>>>> * YARN-752: In AMRMClient, automatically add corresponding rack requests
>>>> for requested nodes
>>>> 
>>>> impact: behavior change
>>>> 
>>>> status: patch avail, +1ed.
>>>> 
>>>> * MAPREDUCE-5171: Expose blacklisted nodes from the MR AM REST API
>>>> 
>>>> impact: Addition to MRAM HTTP API
>>>> 
>>>> status: patch avail, +1ed, needs to be committed
>>>> 
>>>> * YARN-787: Remove resource min from Yarn client API
>>>> 
>>>> impact: Yarn client API change
>>>> 
>>>> status: patch avail, needs to be reviewed. (the calculation of
>> slot-millis
>>>> is not affected, the MIN is taken from conf for now)
>>>> 
>>>> ------------------------------------------------------
>>>> JIRAs that require minor work to make it to 2.1
>>>> 
>>>> * YARN-521: Augment AM - RM client module to be able to request
>> containers
>>>> only at specific locations
>>>> 
>>>> impact: AMRM client API change
>>>> 
>>>> status: patch not avail yet (requires YARN-752)
>>>> 
>>>> * YARN-791: Ensure that RM RPC APIs that return nodes are consistent
>> with
>>>> /nodes REST API
>>>> 
>>>> impact: Yarn client API & proto change
>>>> 
>>>> status: patch avail, review in progress
>>>> 
>>>> * MAPREDUCE-5130: Add missing job config options to mapred-default.xml
>>>> 
>>>> impact: behavior change
>>>> 
>>>> status: patch avail but some tests are failing
>>>> 
>>>> ------------------------------------------------------
>>>> JIRAs that require significant work to make it to 2.1 and may not make
>> it
>>>> 
>>>> * YARN-649: Make container logs available over HTTP in plain text
>>>> 
>>>> impact: Addition to NM HTTP REST API. Needed for MAPREDUCE-4362 (which
>>>> does not change API)
>>>> 
>>>> status: patch avail, review in progress
>>>> 
>>>> ------------------------------------------------------
>>>> JIRAs that don't need to make it to 2.1
>>>> 
>>>> * MAPREDUCE-5311: Remove slot millis computation logic and deprecate
>>>> counter constants
>>>> 
>>>> impact: behavior change
>>>> 
>>>> status: per discussion we should first add memory-millis and
>> vcores-millis
>>>> 
>>>> ------------------------------------------------------
>>>> 
>>>> 
>>>> On Fri, Jun 14, 2013 at 7:17 PM, Roman Shaposhnik <rvs@apache.org>
>> wrote:
>>>> 
>>>>> On Thu, Jun 6, 2013 at 4:48 AM, Arun C Murthy <acm@hortonworks.com>
>>>>> wrote:
>>>>>> 
>>>>>> On Jun 5, 2013, at 11:04 AM, Roman Shaposhnik wrote
>>>>>>> 
>>>>>>> On the Bigtop side of things, once we have stable Bigtop 0.6.0
>> platform
>>>>>>> based on Hadoop 2.0.x codeline we plan to start running the same
>>>>> battery
>>>>>>> of integration tests on the branch-2.1-beta.
>>>>>>> 
>>>>>>> We plan to simply file JIRAs if anything gets detected and I
will
>> also
>>>>>>> publish the URL of the Jenkins job once it gets created.
>>>>>> 
>>>>>> Thanks Roman. Is there an ETA for this? Also, please file jiras with
>>>>> Blocker priority to catch attention.
>>>>> 
>>>>> The build is up and running (and all green on all of the 9 Linux
>>>>> platforms!):
>>>>>   http://bigtop01.cloudera.org:8080/job/Hadoop-2.1.0/
>>>>> 
>>>>> The immediate benefit here is that we get to see that the
>>>>> build is ok on all these Linuxes and all anybody can easily
>>>>> install packaged Hadoop 2.1.0 nightly builds.
>>>>> 
>>>>> Starting from next week, I'll start running regular tests
>>>>> on these bits and will keep you guys posted!
>>>>> 
>>>>> Thanks,
>>>>> Roman.
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Alejandro
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Alejandro
>> 
>> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message