hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohith Sharma K S <rohithsharm...@apache.org>
Subject Re: About 2.7.4 Release
Date Tue, 30 May 2017 06:31:47 GMT
Couple of more JIRAs need to be back ported for 2.7.4 release. These will
solve RM HA unstability issues.
https://issues.apache.org/jira/browse/YARN-5333
https://issues.apache.org/jira/browse/YARN-5988
https://issues.apache.org/jira/browse/YARN-6304

I will raise a JIRAs to back port it.

@Akira , could  you help to add these JIRAs into wiki?

Thanks & Regards
Rohith Sharma K S

On 29 May 2017 at 12:19, Akira Ajisaka <aajisaka@apache.org> wrote:

> Created a page for 2.7.4 release.
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4
>
> If you want to edit this wiki, please ping me.
>
> Regards,
> Akira
>
>
> On 2017/05/23 4:42, Brahma Reddy Battula wrote:
>
>> Hi Konstantin Shvachko
>>
>>
>> how about creating a wiki page for 2.7.4 release status like 2.8 and
>> trunk in following link.??
>>
>>
>> https://cwiki.apache.org/confluence/display/HADOOP
>>
>>
>> ________________________________
>> From: Konstantin Shvachko <shv.hadoop@gmail.com>
>> Sent: Saturday, May 13, 2017 3:58 AM
>> To: Akira Ajisaka
>> Cc: Hadoop Common; Hdfs-dev; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org
>> Subject: Re: About 2.7.4 Release
>>
>> Latest update on the links and filters. Here is the correct link for the
>> filter:
>> https://issues.apache.org/jira/secure/IssueNavigator.jspa?
>> requestId=12340814
>>
>> Also updated: https://s.apache.org/Dzg4
>>
>> Had to do some Jira debugging. Sorry for confusion.
>>
>> Thanks,
>> --Konstantin
>>
>> On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko <
>> shv.hadoop@gmail.com>
>> wrote:
>>
>> Hey Akira,
>>>
>>> I didn't have private filters. Most probably Jira caches something.
>>> Your filter is in the right direction, but for some reason it lists only
>>> 22 issues, while mine has 29.
>>> It misses e.g. YARN-5543 <https://issues.apache.org/jir
>>> a/browse/YARN-5543>
>>> .
>>>
>>> Anyways, I created a Jira filter now "Hadoop 2.7.4 release blockers",
>>> shared it with "everybody", and updated my link to point to that filter.
>>> So
>>> you can use any of the three methods below to get the correct list:
>>> 1. Go to https://s.apache.org/Dzg4
>>> 2. Go to the filter via
>>>     https://issues.apache.org/jira/issues?filter=12340814
>>>    or by finding "Hadoop 2.7.4 release blockers" filter in the jira
>>> 3. On Advanced issues search page paste this:
>>> project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels = release-blocker
>>> AND "Target Version/s" = 2.7.4
>>>
>>> Hope this solves the confusion for which issues are included.
>>> Please LMK if it doesn't, as it is important.
>>>
>>> Thanks,
>>> --Konstantin
>>>
>>> On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <aajisaka@apache.org>
>>> wrote:
>>>
>>> Hi Konstantin,
>>>>
>>>> Thank you for volunteering as release manager!
>>>>
>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>>>
>>>> I couldn't see the link. Maybe is it private filter?
>>>>
>>>> Here is a link I generated: https://s.apache.org/ehKy
>>>> This filter includes resolved issue and excludes fixversion == 2.7.4
>>>>
>>>> Thanks and Regards,
>>>> Akira
>>>>
>>>> On 2017/05/08 19:20, Konstantin Shvachko wrote:
>>>>
>>>> Hi Brahma Reddy Battula,
>>>>>
>>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>>> Your link excludes closed and resolved issues, which needs backporting,
>>>>> and
>>>>> which we cannot reopen, as discussed in this thread earlier.
>>>>>
>>>>> Looked through the issues you proposed:
>>>>>
>>>>> HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311>
>>>>> Seems like a new feature. It helps failover to standby node when
>>>>> primary
>>>>> is
>>>>> under heavy load, but it introduces new APIs, addresses, config
>>>>> parameters.
>>>>> And needs at least one follow up jira.
>>>>> Looks like a backward compatible change, though.
>>>>> Did you have a chance to run it in production?
>>>>>
>>>>> +1 on
>>>>> HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987>
>>>>>
>>>> [HDFS-10987] Make Decommission less expensive when lot of ...<
>> https://issues.apache.org/jira/browse/HDFS-10987>
>> issues.apache.org
>> When user want to decommission a node which having 50M blocks ,it could
>> hold the namesystem lock for long time.We've seen it is taking 36 sec. As
>> we knew during this ...
>>
>>
>>
>> HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902>
>>>>>
>>>> [HDFS-9902] Support different values of dfs.datanode.du ...<
>> https://issues.apache.org/jira/browse/HDFS-9902>
>> issues.apache.org
>> Now Hadoop support different storage type for DISK, SSD, ARCHIVE and
>> RAM_DISK, but they share one configuration dfs.datanode.du.reserved. The
>> DISK size may be several ...
>>
>>
>>
>> HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312>
>>>>>
>>>> Trash does not descent into child directories to check for ...<
>> https://issues.apache.org/jira/browse/HDFS-8312>
>> issues.apache.org
>> HDFS trash does not descent into child directory to check if user has
>> permission to delete files. For example: Run the following command to
>> initialize directory ...
>>
>>
>>
>> HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100>
>>>>>
>>>> Upgrade Jsch jar to latest version to fix vulnerability in ...<
>> https://issues.apache.org/jira/browse/HADOOP-14100>
>> issues.apache.org
>> Recently there was on vulnerability reported on jsch library. Its fixed
>> in latest 0.1.54 version before CVE was made public.
>> https://cve.mitre.org/cgi-bin/cvename.cgi ...
>>
>>
>>
>>
>>>>> Added them to 2.7.4 release. You should see them via the above link
>>>>> now.
>>>>> Would be good if you could attach backport patches for some of them?
>>>>>
>>>>> Appreciate your help,
>>>>> --Konstantin
>>>>>
>>>>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula <
>>>>> brahmareddy.battula@huawei.com> wrote:
>>>>>
>>>>>
>>>>> Looks following link is not correct..
>>>>>>
>>>>>> https://s.apache.org/Dzg4
>>>>>>
>>>>>> It should be like following..?
>>>>>>
>>>>>> https://s.apache.org/wi3U
>>>>>>
>>>>>>
>>>>>> Apart from Konstantin mentioned,Following also good to go..? let
me
>>>>>> know
>>>>>> your thoughts on this.
>>>>>>
>>>>>> For Large Cluster:
>>>>>> =============
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/HDFS-9311===Life Line Protocol
>>>>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommission
>>>>>> Expensive when lot's of blocks are present
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/HDFS-9902===
>>>>>> "dfs.datanode.du.reserved"  per Storage Type
>>>>>>
>>>>>> For Security:
>>>>>> =========
>>>>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash does not
>>>>>> descent
>>>>>> into child directories to check for permission
>>>>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade Jsch
jar
>>>>>> to
>>>>>> latest version to fix vulnerability in old versions
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>> Brahma Reddy Battula
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Erik Krogen [mailto:ekrogen@linkedin.com.INVALID]
>>>>>> Sent: 06 May 2017 02:40
>>>>>> To: Konstantin Shvachko
>>>>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev;
>>>>>> mapreduce-dev@hadoop.apache.org
>>>>>> ;
>>>>>> yarn-dev@hadoop.apache.org
>>>>>> Subject: Re: About 2.7.4 Release
>>>>>>
>>>>>> List LGTM Konstantin!
>>>>>>
>>>>>> Let's say that we will only create a new tracking JIRA for patches
>>>>>> which
>>>>>> do not backport cleanly, to avoid having too many lying around.
>>>>>> Otherwise
>>>>>> we can directly attach to old ticket. If a clean backport does happen
>>>>>> to
>>>>>> break a test the nightly build will help us catch it.
>>>>>>
>>>>>> Erik
>>>>>>
>>>>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko <
>>>>>> shv.hadoop@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Great Zhe. Let's monitor the build.
>>>>>>
>>>>>>>
>>>>>>> I marked all jiras I knew of for inclusion into 2.7.4 as I described
>>>>>>> before.
>>>>>>> Target Version/s: 2.7.4
>>>>>>> Label: release-blocker
>>>>>>>
>>>>>>> Here is the link to the list: https://s.apache.org/Dzg4 Please
let
>>>>>>> me
>>>>>>> know if I missed anything.
>>>>>>> And feel free to pick up any. Most of backports are pretty
>>>>>>> straightforward, but not all.
>>>>>>>
>>>>>>> We can create tracking jiras for backporting if you need to run
>>>>>>> Jenkins on the patch (and since Allen does not allow reopening
them).
>>>>>>> But I think the final patch should be attached to the original
jira.
>>>>>>> Otherwise history will be hard to follow.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> --Konstantin
>>>>>>>
>>>>>>> On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang <zhz@apache.org>
wrote:
>>>>>>>
>>>>>>> Thanks for volunteering as RM Konstantin! The plan LGTM.
>>>>>>>
>>>>>>>>
>>>>>>>> I've created a nightly Jenkins job for branch-2.7 (unit tests):
>>>>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/
>>>>>>>>
>>>>>>>> On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko <
>>>>>>>>
>>>>>>>> shv.hadoop@gmail.com>
>>>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hey guys,
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I and a few of my colleagues would like to help here
and move 2.7.4
>>>>>>>>> release forward. A few points in this regard.
>>>>>>>>>
>>>>>>>>> 1. Reading through this thread since March 1 I see that
Vinod
>>>>>>>>> hinted on managing the release. Vinod, if you still want
the job /
>>>>>>>>> have bandwidth will be happy to work with you.
>>>>>>>>> Otherwise I am glad to volunteer as the release manager.
>>>>>>>>>
>>>>>>>>> 2. In addition to current blockers and criticals, I would
like to
>>>>>>>>>
>>>>>>>>> propose
>>>>>>>>
>>>>>>>
>>>>>>> a
>>>>>>>>
>>>>>>>>> few issues to be included in the release, see the list
below. Those
>>>>>>>>> are mostly bug fixes and optimizations, which we already
have in
>>>>>>>>> our
>>>>>>>>>
>>>>>>>>> internal
>>>>>>>>
>>>>>>>
>>>>>>> branch and run in production. Plus one minor feature "node
>>>>>>>>
>>>>>>>>> labeling", which we found very handy, when you have heterogeneous
>>>>>>>>> environments and mixed workloads, like MR and Spark.
>>>>>>>>>
>>>>>>>>> 3. For marking issues for the release I propose to
>>>>>>>>>  - set the target version to 2.7.4, and
>>>>>>>>>  - add a new label "release-blocker"
>>>>>>>>> That way we will know issues targeted for the release
without
>>>>>>>>> reopening them for backports.
>>>>>>>>>
>>>>>>>>> 4. I see quite a few people are interested in the release.
With all
>>>>>>>>> the help I think we can target to release by the end
of May.
>>>>>>>>>
>>>>>>>>> Other things include fixing CHANGES.txt and fixing Jenkins
build
>>>>>>>>> for
>>>>>>>>>
>>>>>>>>> 2.7.4
>>>>>>>>
>>>>>>>
>>>>>>> branch.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> --Konstantin
>>>>>>>>>
>>>>>>>>> ==========  List of issue for 2.7.4  ===========
>>>>>>>>> ------ Backports
>>>>>>>>> HADOOP-12975 <https://issues.apache.org/jira/browse/HADOOP-12975>.
>>>>>>>>> Add
>>>>>>>>>
>>>>>>>>> du
>>>>>>>>
>>>>>>>
>>>>>>> jitters
>>>>>>>>
>>>>>>>>> HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>.
IBR
>>>>>>>>>
>>>>>>>>> batching
>>>>>>>>
>>>>>>>
>>>>>>> HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715>.
NPE
>>>>>>>>
>>>>>>>>> when applying AvailableSpaceBlockPlacementPolicy
>>>>>>>>> HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538>.
fsck
>>>>>>>>>
>>>>>>>>> removal
>>>>>>>>
>>>>>>>
>>>>>>> of dot printing
>>>>>>>>
>>>>>>>>> HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>.
>>>>>>>>> space-balanced
>>>>>>>>> policy for balancer
>>>>>>>>> HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549>.
abort
>>>>>>>>> balancer if upgrade in progress
>>>>>>>>> HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412>.
skip
>>>>>>>>> small blocks in getBlocks
>>>>>>>>>
>>>>>>>>> YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471>.
SLS
>>>>>>>>> simulator
>>>>>>>>> YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302>.
SLS
>>>>>>>>> YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367>.
SLS
>>>>>>>>> YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>.
SLS
>>>>>>>>>
>>>>>>>>> ----- Node labeling
>>>>>>>>> MAPREDUCE-6304
>>>>>>>>> <https://issues.apache.org/jira/browse/MAPREDUCE-6304>
>>>>>>>>> YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943>
>>>>>>>>> YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109>
>>>>>>>>> YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140>
>>>>>>>>> YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250>
>>>>>>>>> YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>> Zhe Zhang
>>>>>>>> Apache Hadoop Committer
>>>>>>>> http://zhe-thoughts.github.io/about/ | @oldcap
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message