hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akira Ajisaka <aajis...@apache.org>
Subject Re: About 2.7.4 Release
Date Mon, 29 May 2017 06:49:45 GMT
Created a page for 2.7.4 release.
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4

If you want to edit this wiki, please ping me.

Regards,
Akira

On 2017/05/23 4:42, Brahma Reddy Battula wrote:
> Hi Konstantin Shvachko
>
>
> how about creating a wiki page for 2.7.4 release status like 2.8 and trunk in following
link.??
>
>
> https://cwiki.apache.org/confluence/display/HADOOP
>
>
> ________________________________
> From: Konstantin Shvachko <shv.hadoop@gmail.com>
> Sent: Saturday, May 13, 2017 3:58 AM
> To: Akira Ajisaka
> Cc: Hadoop Common; Hdfs-dev; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: About 2.7.4 Release
>
> Latest update on the links and filters. Here is the correct link for the
> filter:
> https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12340814
>
> Also updated: https://s.apache.org/Dzg4
>
> Had to do some Jira debugging. Sorry for confusion.
>
> Thanks,
> --Konstantin
>
> On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko <shv.hadoop@gmail.com>
> wrote:
>
>> Hey Akira,
>>
>> I didn't have private filters. Most probably Jira caches something.
>> Your filter is in the right direction, but for some reason it lists only
>> 22 issues, while mine has 29.
>> It misses e.g. YARN-5543 <https://issues.apache.org/jira/browse/YARN-5543>
>> .
>>
>> Anyways, I created a Jira filter now "Hadoop 2.7.4 release blockers",
>> shared it with "everybody", and updated my link to point to that filter. So
>> you can use any of the three methods below to get the correct list:
>> 1. Go to https://s.apache.org/Dzg4
>> 2. Go to the filter via
>>     https://issues.apache.org/jira/issues?filter=12340814
>>    or by finding "Hadoop 2.7.4 release blockers" filter in the jira
>> 3. On Advanced issues search page paste this:
>> project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels = release-blocker
>> AND "Target Version/s" = 2.7.4
>>
>> Hope this solves the confusion for which issues are included.
>> Please LMK if it doesn't, as it is important.
>>
>> Thanks,
>> --Konstantin
>>
>> On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <aajisaka@apache.org> wrote:
>>
>>> Hi Konstantin,
>>>
>>> Thank you for volunteering as release manager!
>>>
>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>> I couldn't see the link. Maybe is it private filter?
>>>
>>> Here is a link I generated: https://s.apache.org/ehKy
>>> This filter includes resolved issue and excludes fixversion == 2.7.4
>>>
>>> Thanks and Regards,
>>> Akira
>>>
>>> On 2017/05/08 19:20, Konstantin Shvachko wrote:
>>>
>>>> Hi Brahma Reddy Battula,
>>>>
>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>> Your link excludes closed and resolved issues, which needs backporting,
>>>> and
>>>> which we cannot reopen, as discussed in this thread earlier.
>>>>
>>>> Looked through the issues you proposed:
>>>>
>>>> HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311>
>>>> Seems like a new feature. It helps failover to standby node when primary
>>>> is
>>>> under heavy load, but it introduces new APIs, addresses, config
>>>> parameters.
>>>> And needs at least one follow up jira.
>>>> Looks like a backward compatible change, though.
>>>> Did you have a chance to run it in production?
>>>>
>>>> +1 on
>>>> HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987>
> [HDFS-10987] Make Decommission less expensive when lot of ...<https://issues.apache.org/jira/browse/HDFS-10987>
> issues.apache.org
> When user want to decommission a node which having 50M blocks ,it could hold the namesystem
lock for long time.We've seen it is taking 36 sec. As we knew during this ...
>
>
>
>>>> HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902>
> [HDFS-9902] Support different values of dfs.datanode.du ...<https://issues.apache.org/jira/browse/HDFS-9902>
> issues.apache.org
> Now Hadoop support different storage type for DISK, SSD, ARCHIVE and RAM_DISK, but they
share one configuration dfs.datanode.du.reserved. The DISK size may be several ...
>
>
>
>>>> HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312>
> Trash does not descent into child directories to check for ...<https://issues.apache.org/jira/browse/HDFS-8312>
> issues.apache.org
> HDFS trash does not descent into child directory to check if user has permission to delete
files. For example: Run the following command to initialize directory ...
>
>
>
>>>> HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100>
> Upgrade Jsch jar to latest version to fix vulnerability in ...<https://issues.apache.org/jira/browse/HADOOP-14100>
> issues.apache.org
> Recently there was on vulnerability reported on jsch library. Its fixed in latest 0.1.54
version before CVE was made public. https://cve.mitre.org/cgi-bin/cvename.cgi ...
>
>
>
>>>>
>>>> Added them to 2.7.4 release. You should see them via the above link now.
>>>> Would be good if you could attach backport patches for some of them?
>>>>
>>>> Appreciate your help,
>>>> --Konstantin
>>>>
>>>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula <
>>>> brahmareddy.battula@huawei.com> wrote:
>>>>
>>>>
>>>>> Looks following link is not correct..
>>>>>
>>>>> https://s.apache.org/Dzg4
>>>>>
>>>>> It should be like following..?
>>>>>
>>>>> https://s.apache.org/wi3U
>>>>>
>>>>>
>>>>> Apart from Konstantin mentioned,Following also good to go..? let me know
>>>>> your thoughts on this.
>>>>>
>>>>> For Large Cluster:
>>>>> =============
>>>>>
>>>>> https://issues.apache.org/jira/browse/HDFS-9311===Life Line Protocol
>>>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommission
>>>>> Expensive when lot's of blocks are present
>>>>>
>>>>> https://issues.apache.org/jira/browse/HDFS-9902===
>>>>> "dfs.datanode.du.reserved"  per Storage Type
>>>>>
>>>>> For Security:
>>>>> =========
>>>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash does not
>>>>> descent
>>>>> into child directories to check for permission
>>>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade Jsch jar
>>>>> to
>>>>> latest version to fix vulnerability in old versions
>>>>>
>>>>>
>>>>>
>>>>> Regards
>>>>> Brahma Reddy Battula
>>>>>
>>>>> -----Original Message-----
>>>>> From: Erik Krogen [mailto:ekrogen@linkedin.com.INVALID]
>>>>> Sent: 06 May 2017 02:40
>>>>> To: Konstantin Shvachko
>>>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev; mapreduce-dev@hadoop.apache.org
>>>>> ;
>>>>> yarn-dev@hadoop.apache.org
>>>>> Subject: Re: About 2.7.4 Release
>>>>>
>>>>> List LGTM Konstantin!
>>>>>
>>>>> Let's say that we will only create a new tracking JIRA for patches which
>>>>> do not backport cleanly, to avoid having too many lying around.
>>>>> Otherwise
>>>>> we can directly attach to old ticket. If a clean backport does happen
to
>>>>> break a test the nightly build will help us catch it.
>>>>>
>>>>> Erik
>>>>>
>>>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko <
>>>>> shv.hadoop@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Great Zhe. Let's monitor the build.
>>>>>>
>>>>>> I marked all jiras I knew of for inclusion into 2.7.4 as I described
>>>>>> before.
>>>>>> Target Version/s: 2.7.4
>>>>>> Label: release-blocker
>>>>>>
>>>>>> Here is the link to the list: https://s.apache.org/Dzg4 Please let
me
>>>>>> know if I missed anything.
>>>>>> And feel free to pick up any. Most of backports are pretty
>>>>>> straightforward, but not all.
>>>>>>
>>>>>> We can create tracking jiras for backporting if you need to run
>>>>>> Jenkins on the patch (and since Allen does not allow reopening them).
>>>>>> But I think the final patch should be attached to the original jira.
>>>>>> Otherwise history will be hard to follow.
>>>>>>
>>>>>> Thanks,
>>>>>> --Konstantin
>>>>>>
>>>>>> On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang <zhz@apache.org>
wrote:
>>>>>>
>>>>>> Thanks for volunteering as RM Konstantin! The plan LGTM.
>>>>>>>
>>>>>>> I've created a nightly Jenkins job for branch-2.7 (unit tests):
>>>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/
>>>>>>>
>>>>>>> On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko <
>>>>>>>
>>>>>> shv.hadoop@gmail.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hey guys,
>>>>>>>>
>>>>>>>> I and a few of my colleagues would like to help here and
move 2.7.4
>>>>>>>> release forward. A few points in this regard.
>>>>>>>>
>>>>>>>> 1. Reading through this thread since March 1 I see that Vinod
>>>>>>>> hinted on managing the release. Vinod, if you still want
the job /
>>>>>>>> have bandwidth will be happy to work with you.
>>>>>>>> Otherwise I am glad to volunteer as the release manager.
>>>>>>>>
>>>>>>>> 2. In addition to current blockers and criticals, I would
like to
>>>>>>>>
>>>>>>> propose
>>>>>>
>>>>>>> a
>>>>>>>> few issues to be included in the release, see the list below.
Those
>>>>>>>> are mostly bug fixes and optimizations, which we already
have in
>>>>>>>> our
>>>>>>>>
>>>>>>> internal
>>>>>>
>>>>>>> branch and run in production. Plus one minor feature "node
>>>>>>>> labeling", which we found very handy, when you have heterogeneous
>>>>>>>> environments and mixed workloads, like MR and Spark.
>>>>>>>>
>>>>>>>> 3. For marking issues for the release I propose to
>>>>>>>>  - set the target version to 2.7.4, and
>>>>>>>>  - add a new label "release-blocker"
>>>>>>>> That way we will know issues targeted for the release without
>>>>>>>> reopening them for backports.
>>>>>>>>
>>>>>>>> 4. I see quite a few people are interested in the release.
With all
>>>>>>>> the help I think we can target to release by the end of May.
>>>>>>>>
>>>>>>>> Other things include fixing CHANGES.txt and fixing Jenkins
build
>>>>>>>> for
>>>>>>>>
>>>>>>> 2.7.4
>>>>>>
>>>>>>> branch.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> --Konstantin
>>>>>>>>
>>>>>>>> ==========  List of issue for 2.7.4  ===========
>>>>>>>> ------ Backports
>>>>>>>> HADOOP-12975 <https://issues.apache.org/jira/browse/HADOOP-12975>.
>>>>>>>> Add
>>>>>>>>
>>>>>>> du
>>>>>>
>>>>>>> jitters
>>>>>>>> HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>.
IBR
>>>>>>>>
>>>>>>> batching
>>>>>>
>>>>>>> HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715>.
NPE
>>>>>>>> when applying AvailableSpaceBlockPlacementPolicy
>>>>>>>> HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538>.
fsck
>>>>>>>>
>>>>>>> removal
>>>>>>
>>>>>>> of dot printing
>>>>>>>> HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>.
>>>>>>>> space-balanced
>>>>>>>> policy for balancer
>>>>>>>> HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549>.
abort
>>>>>>>> balancer if upgrade in progress
>>>>>>>> HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412>.
skip
>>>>>>>> small blocks in getBlocks
>>>>>>>>
>>>>>>>> YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471>.
SLS
>>>>>>>> simulator
>>>>>>>> YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302>.
SLS
>>>>>>>> YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367>.
SLS
>>>>>>>> YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>.
SLS
>>>>>>>>
>>>>>>>> ----- Node labeling
>>>>>>>> MAPREDUCE-6304
>>>>>>>> <https://issues.apache.org/jira/browse/MAPREDUCE-6304>
>>>>>>>> YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943>
>>>>>>>> YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109>
>>>>>>>> YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140>
>>>>>>>> YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250>
>>>>>>>> YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925>
>>>>>>>>
>>>>>>>> --
>>>>>>> Zhe Zhang
>>>>>>> Apache Hadoop Committer
>>>>>>> http://zhe-thoughts.github.io/about/ | @oldcap
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message