hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: Planning Hadoop 2.6.1 release
Date Wed, 15 Jul 2015 20:12:13 GMT
Got pinged on a recent thread on this one.

As I mentioned there, I had many offline discussions re 2.6.1.

The biggest problem I found offline was about what bug-fixes are acceptable and what aren’t
for everyone wishing to consume 2.6.1. Given the number of bug-fixes that went into 2.7.x
and into branch-2.8, figuring out a set of patches that is acceptable for everyone is a huge
challenge which kind of stalled my attempts.

Thanks
+Vinod


> On Jul 1, 2015, at 12:41 PM, Sean Busbey <busbey@cloudera.com> wrote:
> 
> Any update on a release plan for 2.6.1?
> 
> On Wed, Jun 10, 2015 at 1:25 AM, Brahma Reddy Battula <
> brahmareddy.battula@huawei.com> wrote:
> 
>> HI vinod
>> 
>> any update on this..? are we planning to give 2.6.1 Or can we make 2.7.1
>> as stable give..?
>> 
>> 
>> Thanks & Regards
>> Brahma Reddy Battula
>> 
>> ________________________________________
>> From: Zhihai Xu [zxu@cloudera.com]
>> Sent: Wednesday, May 13, 2015 12:04 PM
>> To: mapreduce-dev@hadoop.apache.org
>> Cc: common-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org
>> Subject: Re: Planning Hadoop 2.6.1 release
>> 
>> Hi Akira,
>> 
>> Can we also include YARN-3242? YARN-3242 fixed a critical ZKRMStateStore
>> bug.
>> It will work better with YARN-2992.
>> 
>> thanks
>> zhihai
>> 
>> 
>> On Tue, May 12, 2015 at 10:38 PM, Akira AJISAKA <
>> ajisakaa@oss.nttdata.co.jp>
>> wrote:
>> 
>>> Thanks all for collecting jiras for 2.6.1 release. In addition, I'd like
>>> to include the following:
>>> 
>>> * HADOOP-11343. Overflow is not properly handled in calculating final iv
>>> for AES CTR
>>> * YARN-2874. Dead lock in "DelegationTokenRenewer" which blocks RM to
>>> execute any further apps
>>> * YARN-2992. ZKRMStateStore crashes due to session expiry
>>> * YARN-3013. AMRMClientImpl does not update AMRM token properly
>>> * YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to
>>> die
>>> * MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal
>> to
>>> a reducer
>>> 
>>> All of these are marked as blocker bug for 2.7.0 but not fixed in 2.6.0.
>>> 
>>> Regards,
>>> Akira
>>> 
>>> 
>>> On 5/4/15 11:15, Brahma Reddy Battula wrote:
>>> 
>>>> Hello Vinod,
>>>> 
>>>> I am thinking,can we include HADOOP-11491 also..? wihout this jira harfs
>>>> will not be usable when cluster installed in HA mode and try to get
>>>> filecontext like below..
>>>> 
>>>> 
>>>> Path path = new
>>>> 
>> Path("har:///archivedLogs/application_1428917727658_0005-application_1428917727658_0008-1428927448352.har");
>>>> FileSystem fs = path.getFileSystem(new Configuration());
>>>> path = fs.makeQualified(path);
>>>> FileContext fc = FileContext.getFileContext(path.toUri(),new
>>>> Configuration());
>>>> 
>>>> 
>>>> 
>>>> Thanks & Regards
>>>> Brahma Reddy Battula
>>>> ________________________________________
>>>> From: Chris Nauroth [cnauroth@hortonworks.com]
>>>> Sent: Friday, May 01, 2015 4:32 AM
>>>> To: mapreduce-dev@hadoop.apache.org; common-dev@hadoop.apache.org;
>>>> yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>>>> Subject: Re: Planning Hadoop 2.6.1 release
>>>> 
>>>> Thank you, Arpit.  In addition, I suggest we include the following:
>>>> 
>>>> HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification
>>>> pipe is full
>>>> HADOOP-11604. Prevent ConcurrentModificationException while closing
>> domain
>>>> sockets during shutdown of DomainSocketWatcher thread.
>>>> HADOOP-11648. Set DomainSocketWatcher thread name explicitly
>>>> HADOOP-11802. DomainSocketWatcher thread terminates sometimes after
>> there
>>>> is an I/O error during requestShortCircuitShm
>>>> 
>>>> HADOOP-11604 and 11648 are not critical by themselves, but they are
>>>> pre-requisites to getting a clean cherry-pick of 11802, which we believe
>>>> finally fixes the root cause of this issue.
>>>> 
>>>> 
>>>> --Chris Nauroth
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 4/30/15, 3:55 PM, "Arpit Agarwal" <aagarwal@hortonworks.com> wrote:
>>>> 
>>>> HDFS candidates for back-porting to Hadoop 2.6.1. The first two were
>>>>> requested in [1].
>>>>> 
>>>>> HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream
>>>>> should be non static
>>>>> HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt
>>>>> synchronization
>>>>> 
>>>>> HDFS-7009. Active NN and standby NN have different live nodes.
>>>>> HDFS-7035. Make adding a new data directory to the DataNode an atomic
>> and
>>>>> improve error handling
>>>>> HDFS-7425. NameNode block deletion logging uses incorrect appender.
>>>>> HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate
>>>>> block files are present in the same volume.
>>>>> HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang
>> datanodes
>>>>> HDFS-7503. Namenode restart after large deletions can cause slow
>>>>> processReport.
>>>>> HDFS-7575. Upgrade should generate a unique storage ID for each volume.
>>>>> HDFS-7579. Improve log reporting during block report rpc failure.
>>>>> HDFS-7587. Edit log corruption can happen if append fails with a quota
>>>>> violation.
>>>>> HDFS-7596. NameNode should prune dead storages from storageMap.
>>>>> HDFS-7611. deleteSnapshot and delete of a file can leave orphaned
>> blocks
>>>>> in the blocksMap on NameNode restart.
>>>>> HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause
>>>>> DataNode to register successfully with only one NameNode.
>>>>> HDFS-7733. NFS: readdir/readdirplus return null directory attribute on
>>>>> failure.
>>>>> HDFS-7831. Fix the starting index and end condition of the loop in
>>>>> FileDiffList.findEarlierSnapshotBlocks().
>>>>> HDFS-7885. Datanode should not trust the generation stamp provided by
>>>>> client.
>>>>> HDFS-7960. The full block report should prune zombie storages even if
>>>>> they're not empty.
>>>>> HDFS-8072. Reserved RBW space is not released if client terminates
>> while
>>>>> writing block.
>>>>> HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to
>>>>> finalize upgrade.
>>>>> 
>>>>> 
>>>>> Arpit
>>>>> 
>>>>> [1] Will Hadoop 2.6.1 be released soon?
>>>>> http://markmail.org/thread/zlsr6prejyogdyvh
>>>>> 
>>>>> 
>>>>> 
>>>>> On 4/27/15, 11:47 AM, "Vinod Kumar Vavilapalli" <vinodkv@apache.org>
>>>>> wrote:
>>>>> 
>>>>> There were several requests on the user lists [1] for a 2.6.1
>> release. I
>>>>>> got many offline comments too.
>>>>>> 
>>>>>> Planning to do a 2.6.1 release in a few weeks time. We already have
a
>>>>>> bunch
>>>>>> of tickets committed to 2.7.1. I created a filter [2] to tracking
>>>>>> pending
>>>>>> tickets.
>>>>>> 
>>>>>> We need to collectively come up with a list of critical issues. We
can
>>>>>> use
>>>>>> the JIRA Target Version field for the same. I see some but not a
whole
>>>>>> lot
>>>>>> of new work for this release, most of it is likely going to be pulling
>>>>>> in
>>>>>> critical patches from 2.7.1/2.8 etc.
>>>>>> 
>>>>>> Thoughts?
>>>>>> 
>>>>>> Thanks
>>>>>> +Vinod
>>>>>> 
>>>>>> [1] Will Hadoop 2.6.1 be released soon?
>>>>>> http://markmail.org/thread/zlsr6prejyogdyvh
>>>>>> [2] 2.6.1 pending tickets
>>>>>> https://issues.apache.org/jira/issues/?filter=12331711
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Sean

Mime
View raw message