hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akira Ajisaka <aajis...@apache.org>
Subject Re: About 2.7.4 Release
Date Tue, 30 May 2017 07:08:04 GMT
Sure.
If you want to edit the wiki, please tell me your ASF confluence account.

-Akira

On 2017/05/30 15:31, Rohith Sharma K S wrote:
> Couple of more JIRAs need to be back ported for 2.7.4 release. These will
> solve RM HA unstability issues.
> https://issues.apache.org/jira/browse/YARN-5333
> https://issues.apache.org/jira/browse/YARN-5988
> https://issues.apache.org/jira/browse/YARN-6304
>
> I will raise a JIRAs to back port it.
>
> @Akira , could  you help to add these JIRAs into wiki?
>
> Thanks & Regards
> Rohith Sharma K S
>
> On 29 May 2017 at 12:19, Akira Ajisaka <aajisaka@apache.org> wrote:
>
>> Created a page for 2.7.4 release.
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4
>>
>> If you want to edit this wiki, please ping me.
>>
>> Regards,
>> Akira
>>
>>
>> On 2017/05/23 4:42, Brahma Reddy Battula wrote:
>>
>>> Hi Konstantin Shvachko
>>>
>>>
>>> how about creating a wiki page for 2.7.4 release status like 2.8 and
>>> trunk in following link.??
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/HADOOP
>>>
>>>
>>> ________________________________
>>> From: Konstantin Shvachko <shv.hadoop@gmail.com>
>>> Sent: Saturday, May 13, 2017 3:58 AM
>>> To: Akira Ajisaka
>>> Cc: Hadoop Common; Hdfs-dev; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org
>>> Subject: Re: About 2.7.4 Release
>>>
>>> Latest update on the links and filters. Here is the correct link for the
>>> filter:
>>> https://issues.apache.org/jira/secure/IssueNavigator.jspa?
>>> requestId=12340814
>>>
>>> Also updated: https://s.apache.org/Dzg4
>>>
>>> Had to do some Jira debugging. Sorry for confusion.
>>>
>>> Thanks,
>>> --Konstantin
>>>
>>> On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko <
>>> shv.hadoop@gmail.com>
>>> wrote:
>>>
>>> Hey Akira,
>>>>
>>>> I didn't have private filters. Most probably Jira caches something.
>>>> Your filter is in the right direction, but for some reason it lists only
>>>> 22 issues, while mine has 29.
>>>> It misses e.g. YARN-5543 <https://issues.apache.org/jir
>>>> a/browse/YARN-5543>
>>>> .
>>>>
>>>> Anyways, I created a Jira filter now "Hadoop 2.7.4 release blockers",
>>>> shared it with "everybody", and updated my link to point to that filter.
>>>> So
>>>> you can use any of the three methods below to get the correct list:
>>>> 1. Go to https://s.apache.org/Dzg4
>>>> 2. Go to the filter via
>>>>     https://issues.apache.org/jira/issues?filter=12340814
>>>>    or by finding "Hadoop 2.7.4 release blockers" filter in the jira
>>>> 3. On Advanced issues search page paste this:
>>>> project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels = release-blocker
>>>> AND "Target Version/s" = 2.7.4
>>>>
>>>> Hope this solves the confusion for which issues are included.
>>>> Please LMK if it doesn't, as it is important.
>>>>
>>>> Thanks,
>>>> --Konstantin
>>>>
>>>> On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <aajisaka@apache.org>
>>>> wrote:
>>>>
>>>> Hi Konstantin,
>>>>>
>>>>> Thank you for volunteering as release manager!
>>>>>
>>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>>>>
>>>>> I couldn't see the link. Maybe is it private filter?
>>>>>
>>>>> Here is a link I generated: https://s.apache.org/ehKy
>>>>> This filter includes resolved issue and excludes fixversion == 2.7.4
>>>>>
>>>>> Thanks and Regards,
>>>>> Akira
>>>>>
>>>>> On 2017/05/08 19:20, Konstantin Shvachko wrote:
>>>>>
>>>>> Hi Brahma Reddy Battula,
>>>>>>
>>>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>>>> Your link excludes closed and resolved issues, which needs backporting,
>>>>>> and
>>>>>> which we cannot reopen, as discussed in this thread earlier.
>>>>>>
>>>>>> Looked through the issues you proposed:
>>>>>>
>>>>>> HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311>
>>>>>> Seems like a new feature. It helps failover to standby node when
>>>>>> primary
>>>>>> is
>>>>>> under heavy load, but it introduces new APIs, addresses, config
>>>>>> parameters.
>>>>>> And needs at least one follow up jira.
>>>>>> Looks like a backward compatible change, though.
>>>>>> Did you have a chance to run it in production?
>>>>>>
>>>>>> +1 on
>>>>>> HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987>
>>>>>>
>>>>> [HDFS-10987] Make Decommission less expensive when lot of ...<
>>> https://issues.apache.org/jira/browse/HDFS-10987>
>>> issues.apache.org
>>> When user want to decommission a node which having 50M blocks ,it could
>>> hold the namesystem lock for long time.We've seen it is taking 36 sec. As
>>> we knew during this ...
>>>
>>>
>>>
>>> HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902>
>>>>>>
>>>>> [HDFS-9902] Support different values of dfs.datanode.du ...<
>>> https://issues.apache.org/jira/browse/HDFS-9902>
>>> issues.apache.org
>>> Now Hadoop support different storage type for DISK, SSD, ARCHIVE and
>>> RAM_DISK, but they share one configuration dfs.datanode.du.reserved. The
>>> DISK size may be several ...
>>>
>>>
>>>
>>> HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312>
>>>>>>
>>>>> Trash does not descent into child directories to check for ...<
>>> https://issues.apache.org/jira/browse/HDFS-8312>
>>> issues.apache.org
>>> HDFS trash does not descent into child directory to check if user has
>>> permission to delete files. For example: Run the following command to
>>> initialize directory ...
>>>
>>>
>>>
>>> HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100>
>>>>>>
>>>>> Upgrade Jsch jar to latest version to fix vulnerability in ...<
>>> https://issues.apache.org/jira/browse/HADOOP-14100>
>>> issues.apache.org
>>> Recently there was on vulnerability reported on jsch library. Its fixed
>>> in latest 0.1.54 version before CVE was made public.
>>> https://cve.mitre.org/cgi-bin/cvename.cgi ...
>>>
>>>
>>>
>>>
>>>>>> Added them to 2.7.4 release. You should see them via the above link
>>>>>> now.
>>>>>> Would be good if you could attach backport patches for some of them?
>>>>>>
>>>>>> Appreciate your help,
>>>>>> --Konstantin
>>>>>>
>>>>>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula <
>>>>>> brahmareddy.battula@huawei.com> wrote:
>>>>>>
>>>>>>
>>>>>> Looks following link is not correct..
>>>>>>>
>>>>>>> https://s.apache.org/Dzg4
>>>>>>>
>>>>>>> It should be like following..?
>>>>>>>
>>>>>>> https://s.apache.org/wi3U
>>>>>>>
>>>>>>>
>>>>>>> Apart from Konstantin mentioned,Following also good to go..?
let me
>>>>>>> know
>>>>>>> your thoughts on this.
>>>>>>>
>>>>>>> For Large Cluster:
>>>>>>> =============
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/HDFS-9311===Life Line Protocol
>>>>>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommission
>>>>>>> Expensive when lot's of blocks are present
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/HDFS-9902===
>>>>>>> "dfs.datanode.du.reserved"  per Storage Type
>>>>>>>
>>>>>>> For Security:
>>>>>>> =========
>>>>>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash does
not
>>>>>>> descent
>>>>>>> into child directories to check for permission
>>>>>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade
Jsch jar
>>>>>>> to
>>>>>>> latest version to fix vulnerability in old versions
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards
>>>>>>> Brahma Reddy Battula
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Erik Krogen [mailto:ekrogen@linkedin.com.INVALID]
>>>>>>> Sent: 06 May 2017 02:40
>>>>>>> To: Konstantin Shvachko
>>>>>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev;
>>>>>>> mapreduce-dev@hadoop.apache.org
>>>>>>> ;
>>>>>>> yarn-dev@hadoop.apache.org
>>>>>>> Subject: Re: About 2.7.4 Release
>>>>>>>
>>>>>>> List LGTM Konstantin!
>>>>>>>
>>>>>>> Let's say that we will only create a new tracking JIRA for patches
>>>>>>> which
>>>>>>> do not backport cleanly, to avoid having too many lying around.
>>>>>>> Otherwise
>>>>>>> we can directly attach to old ticket. If a clean backport does
happen
>>>>>>> to
>>>>>>> break a test the nightly build will help us catch it.
>>>>>>>
>>>>>>> Erik
>>>>>>>
>>>>>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko <
>>>>>>> shv.hadoop@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Great Zhe. Let's monitor the build.
>>>>>>>
>>>>>>>>
>>>>>>>> I marked all jiras I knew of for inclusion into 2.7.4 as
I described
>>>>>>>> before.
>>>>>>>> Target Version/s: 2.7.4
>>>>>>>> Label: release-blocker
>>>>>>>>
>>>>>>>> Here is the link to the list: https://s.apache.org/Dzg4 Please
let
>>>>>>>> me
>>>>>>>> know if I missed anything.
>>>>>>>> And feel free to pick up any. Most of backports are pretty
>>>>>>>> straightforward, but not all.
>>>>>>>>
>>>>>>>> We can create tracking jiras for backporting if you need
to run
>>>>>>>> Jenkins on the patch (and since Allen does not allow reopening
them).
>>>>>>>> But I think the final patch should be attached to the original
jira.
>>>>>>>> Otherwise history will be hard to follow.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> --Konstantin
>>>>>>>>
>>>>>>>> On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang <zhz@apache.org>
wrote:
>>>>>>>>
>>>>>>>> Thanks for volunteering as RM Konstantin! The plan LGTM.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I've created a nightly Jenkins job for branch-2.7 (unit
tests):
>>>>>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/
>>>>>>>>>
>>>>>>>>> On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko <
>>>>>>>>>
>>>>>>>>> shv.hadoop@gmail.com>
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hey guys,
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I and a few of my colleagues would like to help here
and move 2.7.4
>>>>>>>>>> release forward. A few points in this regard.
>>>>>>>>>>
>>>>>>>>>> 1. Reading through this thread since March 1 I see
that Vinod
>>>>>>>>>> hinted on managing the release. Vinod, if you still
want the job /
>>>>>>>>>> have bandwidth will be happy to work with you.
>>>>>>>>>> Otherwise I am glad to volunteer as the release manager.
>>>>>>>>>>
>>>>>>>>>> 2. In addition to current blockers and criticals,
I would like to
>>>>>>>>>>
>>>>>>>>>> propose
>>>>>>>>>
>>>>>>>>
>>>>>>>> a
>>>>>>>>>
>>>>>>>>>> few issues to be included in the release, see the
list below. Those
>>>>>>>>>> are mostly bug fixes and optimizations, which we
already have in
>>>>>>>>>> our
>>>>>>>>>>
>>>>>>>>>> internal
>>>>>>>>>
>>>>>>>>
>>>>>>>> branch and run in production. Plus one minor feature "node
>>>>>>>>>
>>>>>>>>>> labeling", which we found very handy, when you have
heterogeneous
>>>>>>>>>> environments and mixed workloads, like MR and Spark.
>>>>>>>>>>
>>>>>>>>>> 3. For marking issues for the release I propose to
>>>>>>>>>>  - set the target version to 2.7.4, and
>>>>>>>>>>  - add a new label "release-blocker"
>>>>>>>>>> That way we will know issues targeted for the release
without
>>>>>>>>>> reopening them for backports.
>>>>>>>>>>
>>>>>>>>>> 4. I see quite a few people are interested in the
release. With all
>>>>>>>>>> the help I think we can target to release by the
end of May.
>>>>>>>>>>
>>>>>>>>>> Other things include fixing CHANGES.txt and fixing
Jenkins build
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>> 2.7.4
>>>>>>>>>
>>>>>>>>
>>>>>>>> branch.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> --Konstantin
>>>>>>>>>>
>>>>>>>>>> ==========  List of issue for 2.7.4  ===========
>>>>>>>>>> ------ Backports
>>>>>>>>>> HADOOP-12975 <https://issues.apache.org/jira/browse/HADOOP-12975>.
>>>>>>>>>> Add
>>>>>>>>>>
>>>>>>>>>> du
>>>>>>>>>
>>>>>>>>
>>>>>>>> jitters
>>>>>>>>>
>>>>>>>>>> HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>.
IBR
>>>>>>>>>>
>>>>>>>>>> batching
>>>>>>>>>
>>>>>>>>
>>>>>>>> HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715>.
NPE
>>>>>>>>>
>>>>>>>>>> when applying AvailableSpaceBlockPlacementPolicy
>>>>>>>>>> HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538>.
fsck
>>>>>>>>>>
>>>>>>>>>> removal
>>>>>>>>>
>>>>>>>>
>>>>>>>> of dot printing
>>>>>>>>>
>>>>>>>>>> HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>.
>>>>>>>>>> space-balanced
>>>>>>>>>> policy for balancer
>>>>>>>>>> HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549>.
abort
>>>>>>>>>> balancer if upgrade in progress
>>>>>>>>>> HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412>.
skip
>>>>>>>>>> small blocks in getBlocks
>>>>>>>>>>
>>>>>>>>>> YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471>.
SLS
>>>>>>>>>> simulator
>>>>>>>>>> YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302>.
SLS
>>>>>>>>>> YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367>.
SLS
>>>>>>>>>> YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>.
SLS
>>>>>>>>>>
>>>>>>>>>> ----- Node labeling
>>>>>>>>>> MAPREDUCE-6304
>>>>>>>>>> <https://issues.apache.org/jira/browse/MAPREDUCE-6304>
>>>>>>>>>> YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943>
>>>>>>>>>> YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109>
>>>>>>>>>> YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140>
>>>>>>>>>> YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250>
>>>>>>>>>> YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>> Zhe Zhang
>>>>>>>>> Apache Hadoop Committer
>>>>>>>>> http://zhe-thoughts.github.io/about/ | @oldcap
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message