hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anu Engineer <aengin...@hortonworks.com>
Subject Re: LinkedIn Dynamometer Tool (was About 2.7.4 Release)
Date Thu, 20 Jul 2017 17:56:33 GMT
Hi Erik,

Looking forward to the release of this tool. Thank you very much for the contribution.

Had a couple of questions about how the tool works.

1. Would you be able to provide the traces along with this tool? In other words, would I be
able to use this out of the box, or do I have to build up traces myself? 

2. Could you explain how the “fake out DNs into thinking they are storing data” — works?
Or I can be patient and read your blog post too.

Thanks
Anu






On 7/20/17, 10:42 AM, "Erik Krogen" <ekrogen@linkedin.com.INVALID> wrote:

>forking off of the 2.7.4 release thread to answer this question about
>Dynamometer
>
>Dynamometer is a tool developed at LinkedIn for scale testing HDFS,
>specifically the NameNode. We have been using it for some time now and have
>recently been making some enhancements to ease of use and reproducibility.
>We hope to post a blog post sometime in the not-too-distant future, and
>also to open source it. I can provide some details here given that we have
>been leveraging it as part of our 2.7.4 release / upgrade process (in
>addition to previous upgrades).
>
>The basic idea is to get full-scale black-box testing of the HDFS NN while
>using significantly less (~10%) hardware than a real cluster of that size
>would require. We use real NN images from our at-scale clusters paired with
>some logic to fake out DNs into thinking they are storing data when they
>are not, allowing us to stuff more DNs onto each machine. Since we use a
>real image, we can replay real traces (collected from audit logs) to
>compare actual production performance vs. performance on this simulated
>cluster (with additional tuning, different version, etc.). We leverage YARN
>to manage setting up this cluster and to replay the traces.
>
>Happy to answer questions.
>
>Erik
>
>On Wed, Jul 19, 2017 at 5:05 PM, Konstantin Shvachko <shv.hadoop@gmail.com>
>wrote:
>
>> Hi Tianyi,
>>
>> Glad you are interested in Dynamometer. Erik (CC-ed) is actively working
>> on this project right now, I'll let him elaborate.
>> Erik, you should probably respond on Apache dev list, as I think it could
>> be interesting for other people as well, asince we planned to open source
>> it. You can fork the "About 2.7.4 Release" thread with a new subject and
>> give some details about Dynamometer there.
>>
>> Thanks,
>> --Konstantin
>>
>> On Wed, Jul 19, 2017 at 1:40 AM, 何天一 <hetianyi@bytedance.com> wrote:
>>
>>> Hi, Shavachko.
>>>
>>> You mentioned an internal tool called Dynamometer to test NameNode
>>> performance earlier in the 2.7.4 release thread.
>>> I wonder if you could share some ideas behind the tool. Or is there a
>>> plan to bring Dynamometer to open source community?
>>>
>>> Thanks.
>>>
>>> BR,
>>> Tianyi
>>>
>>> On Fri, Jul 14, 2017 at 8:45 AM Konstantin Shvachko <shv.hadoop@gmail.com>
>>> wrote:
>>>
>>>> Hi everybody.
>>>>
>>>> We have been doing some internal testing of Hadoop 2.7.4. The testing is
>>>> going well.
>>>> Did not find any major issues on our workloads.
>>>> Used an internal tool called Dynamometer to check NameNode performance on
>>>> real cluster traces. Good.
>>>> Overall test cluster performance looks good.
>>>> Some more testing is still going on.
>>>>
>>>> I plan to build an RC next week. If there are no objection.
>>>>
>>>> Thanks,
>>>> --Konst
>>>>
>>>> On Thu, Jun 15, 2017 at 4:42 PM, Konstantin Shvachko <
>>>> shv.hadoop@gmail.com>
>>>> wrote:
>>>>
>>>> > Hey guys.
>>>> >
>>>> > An update on 2.7.4 progress.
>>>> > We are down to 4 blockers. There is some work remaining on those.
>>>> > https://issues.apache.org/jira/browse/HDFS-11896?filter=12340814
>>>> > Would be good if people could follow up on review comments.
>>>> >
>>>> > I looked through nightly Jenkins build results for 2.7.4 both on Apache
>>>> > Jenkins and internal.
>>>> > Some test fail intermittently, but there no consistent failures. I
>>>> filed
>>>> > HDFS-11985 to track some of them.
>>>> > https://issues.apache.org/jira/browse/HDFS-11985
>>>> > I do not currently consider these failures as blockers. LMK if some
of
>>>> > them are.
>>>> >
>>>> > We started internal testing of branch-2.7 on one of our smallish (100+
>>>> > nodes) test clusters.
>>>> > Will update on the results.
>>>> >
>>>> > There is a plan to enable BigTop for 2.7.4 testing.
>>>> >
>>>> > Akira, Brahma thank you for setting up a wiki page for 2.7.4 release.
>>>> > Thank you everybody for contributing to this effort.
>>>> >
>>>> > Regards,
>>>> > --Konstantin
>>>> >
>>>> >
>>>> > On Tue, May 30, 2017 at 12:08 AM, Akira Ajisaka <aajisaka@apache.org>
>>>> > wrote:
>>>> >
>>>> >> Sure.
>>>> >> If you want to edit the wiki, please tell me your ASF confluence
>>>> account.
>>>> >>
>>>> >> -Akira
>>>> >>
>>>> >> On 2017/05/30 15:31, Rohith Sharma K S wrote:
>>>> >>
>>>> >>> Couple of more JIRAs need to be back ported for 2.7.4 release.
These
>>>> will
>>>> >>> solve RM HA unstability issues.
>>>> >>> https://issues.apache.org/jira/browse/YARN-5333
>>>> >>> https://issues.apache.org/jira/browse/YARN-5988
>>>> >>> https://issues.apache.org/jira/browse/YARN-6304
>>>> >>>
>>>> >>> I will raise a JIRAs to back port it.
>>>> >>>
>>>> >>> @Akira , could  you help to add these JIRAs into wiki?
>>>> >>>
>>>> >>> Thanks & Regards
>>>> >>> Rohith Sharma K S
>>>> >>>
>>>> >>> On 29 May 2017 at 12:19, Akira Ajisaka <aajisaka@apache.org>
wrote:
>>>> >>>
>>>> >>> Created a page for 2.7.4 release.
>>>> >>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4
>>>> >>>>
>>>> >>>> If you want to edit this wiki, please ping me.
>>>> >>>>
>>>> >>>> Regards,
>>>> >>>> Akira
>>>> >>>>
>>>> >>>>
>>>> >>>> On 2017/05/23 4:42, Brahma Reddy Battula wrote:
>>>> >>>>
>>>> >>>> Hi Konstantin Shvachko
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> how about creating a wiki page for 2.7.4 release status
like 2.8
>>>> and
>>>> >>>>> trunk in following link.??
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> https://cwiki.apache.org/confluence/display/HADOOP
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> ________________________________
>>>> >>>>> From: Konstantin Shvachko <shv.hadoop@gmail.com>
>>>> >>>>> Sent: Saturday, May 13, 2017 3:58 AM
>>>> >>>>> To: Akira Ajisaka
>>>> >>>>> Cc: Hadoop Common; Hdfs-dev; mapreduce-dev@hadoop.apache.org;
>>>> >>>>> yarn-dev@hadoop.apache.org
>>>> >>>>> Subject: Re: About 2.7.4 Release
>>>> >>>>>
>>>> >>>>> Latest update on the links and filters. Here is the
correct link
>>>> for
>>>> >>>>> the
>>>> >>>>> filter:
>>>> >>>>> https://issues.apache.org/jira/secure/IssueNavigator.jspa?
>>>> >>>>> requestId=12340814
>>>> >>>>>
>>>> >>>>> Also updated: https://s.apache.org/Dzg4
>>>> >>>>>
>>>> >>>>> Had to do some Jira debugging. Sorry for confusion.
>>>> >>>>>
>>>> >>>>> Thanks,
>>>> >>>>> --Konstantin
>>>> >>>>>
>>>> >>>>> On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko
<
>>>> >>>>> shv.hadoop@gmail.com>
>>>> >>>>> wrote:
>>>> >>>>>
>>>> >>>>> Hey Akira,
>>>> >>>>>
>>>> >>>>>>
>>>> >>>>>> I didn't have private filters. Most probably Jira
caches
>>>> something.
>>>> >>>>>> Your filter is in the right direction, but for some
reason it
>>>> lists
>>>> >>>>>> only
>>>> >>>>>> 22 issues, while mine has 29.
>>>> >>>>>> It misses e.g. YARN-5543 <https://issues.apache.org/jir
>>>> >>>>>> a/browse/YARN-5543>
>>>> >>>>>> .
>>>> >>>>>>
>>>> >>>>>> Anyways, I created a Jira filter now "Hadoop 2.7.4
release
>>>> blockers",
>>>> >>>>>> shared it with "everybody", and updated my link
to point to that
>>>> >>>>>> filter.
>>>> >>>>>> So
>>>> >>>>>> you can use any of the three methods below to get
the correct
>>>> list:
>>>> >>>>>> 1. Go to https://s.apache.org/Dzg4
>>>> >>>>>> 2. Go to the filter via
>>>> >>>>>>     https://issues.apache.org/jira/issues?filter=12340814
>>>> >>>>>>    or by finding "Hadoop 2.7.4 release blockers"
filter in the
>>>> jira
>>>> >>>>>> 3. On Advanced issues search page paste this:
>>>> >>>>>> project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels
=
>>>> >>>>>> release-blocker
>>>> >>>>>> AND "Target Version/s" = 2.7.4
>>>> >>>>>>
>>>> >>>>>> Hope this solves the confusion for which issues
are included.
>>>> >>>>>> Please LMK if it doesn't, as it is important.
>>>> >>>>>>
>>>> >>>>>> Thanks,
>>>> >>>>>> --Konstantin
>>>> >>>>>>
>>>> >>>>>> On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <
>>>> aajisaka@apache.org>
>>>> >>>>>> wrote:
>>>> >>>>>>
>>>> >>>>>> Hi Konstantin,
>>>> >>>>>>
>>>> >>>>>>>
>>>> >>>>>>> Thank you for volunteering as release manager!
>>>> >>>>>>>
>>>> >>>>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>> >>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> I couldn't see the link. Maybe is it private
filter?
>>>> >>>>>>>
>>>> >>>>>>> Here is a link I generated: https://s.apache.org/ehKy
>>>> >>>>>>> This filter includes resolved issue and excludes
fixversion ==
>>>> 2.7.4
>>>> >>>>>>>
>>>> >>>>>>> Thanks and Regards,
>>>> >>>>>>> Akira
>>>> >>>>>>>
>>>> >>>>>>> On 2017/05/08 19:20, Konstantin Shvachko wrote:
>>>> >>>>>>>
>>>> >>>>>>> Hi Brahma Reddy Battula,
>>>> >>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> Actually the original link works fine:
>>>> https://s.apache.org/Dzg4
>>>> >>>>>>>> Your link excludes closed and resolved issues,
which needs
>>>> >>>>>>>> backporting,
>>>> >>>>>>>> and
>>>> >>>>>>>> which we cannot reopen, as discussed in
this thread earlier.
>>>> >>>>>>>>
>>>> >>>>>>>> Looked through the issues you proposed:
>>>> >>>>>>>>
>>>> >>>>>>>> HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311>
>>>> >>>>>>>> Seems like a new feature. It helps failover
to standby node when
>>>> >>>>>>>> primary
>>>> >>>>>>>> is
>>>> >>>>>>>> under heavy load, but it introduces new
APIs, addresses, config
>>>> >>>>>>>> parameters.
>>>> >>>>>>>> And needs at least one follow up jira.
>>>> >>>>>>>> Looks like a backward compatible change,
though.
>>>> >>>>>>>> Did you have a chance to run it in production?
>>>> >>>>>>>>
>>>> >>>>>>>> +1 on
>>>> >>>>>>>> HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987>
>>>> >>>>>>>>
>>>> >>>>>>>> [HDFS-10987] Make Decommission less expensive
when lot of ...<
>>>> >>>>>>>
>>>> >>>>>> https://issues.apache.org/jira/browse/HDFS-10987>
>>>> >>>>> issues.apache.org
>>>> >>>>> When user want to decommission a node which having 50M
blocks ,it
>>>> could
>>>> >>>>> hold the namesystem lock for long time.We've seen it
is taking 36
>>>> sec.
>>>> >>>>> As
>>>> >>>>> we knew during this ...
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902>
>>>> >>>>>
>>>> >>>>>>
>>>> >>>>>>>> [HDFS-9902] Support different values of
dfs.datanode.du ...<
>>>> >>>>>>>
>>>> >>>>>> https://issues.apache.org/jira/browse/HDFS-9902>
>>>> >>>>> issues.apache.org
>>>> >>>>> Now Hadoop support different storage type for DISK,
SSD, ARCHIVE
>>>> and
>>>> >>>>> RAM_DISK, but they share one configuration
>>>> dfs.datanode.du.reserved.
>>>> >>>>> The
>>>> >>>>> DISK size may be several ...
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312>
>>>> >>>>>
>>>> >>>>>>
>>>> >>>>>>>> Trash does not descent into child directories
to check for ...<
>>>> >>>>>>>
>>>> >>>>>> https://issues.apache.org/jira/browse/HDFS-8312>
>>>> >>>>> issues.apache.org
>>>> >>>>> HDFS trash does not descent into child directory to
check if user
>>>> has
>>>> >>>>> permission to delete files. For example: Run the following
command
>>>> to
>>>> >>>>> initialize directory ...
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100>
>>>> >>>>>
>>>> >>>>>>
>>>> >>>>>>>> Upgrade Jsch jar to latest version to fix
vulnerability in ...<
>>>> >>>>>>>
>>>> >>>>>> https://issues.apache.org/jira/browse/HADOOP-14100>
>>>> >>>>> issues.apache.org
>>>> >>>>> Recently there was on vulnerability reported on jsch
library. Its
>>>> fixed
>>>> >>>>> in latest 0.1.54 version before CVE was made public.
>>>> >>>>> https://cve.mitre.org/cgi-bin/cvename.cgi ...
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> Added them to 2.7.4 release. You should see them via
the above link
>>>> >>>>>>>> now.
>>>> >>>>>>>> Would be good if you could attach backport
patches for some of
>>>> them?
>>>> >>>>>>>>
>>>> >>>>>>>> Appreciate your help,
>>>> >>>>>>>> --Konstantin
>>>> >>>>>>>>
>>>> >>>>>>>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy
Battula <
>>>> >>>>>>>> brahmareddy.battula@huawei.com> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> Looks following link is not correct..
>>>> >>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> https://s.apache.org/Dzg4
>>>> >>>>>>>>>
>>>> >>>>>>>>> It should be like following..?
>>>> >>>>>>>>>
>>>> >>>>>>>>> https://s.apache.org/wi3U
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> Apart from Konstantin mentioned,Following
also good to go..?
>>>> let me
>>>> >>>>>>>>> know
>>>> >>>>>>>>> your thoughts on this.
>>>> >>>>>>>>>
>>>> >>>>>>>>> For Large Cluster:
>>>> >>>>>>>>> =============
>>>> >>>>>>>>>
>>>> >>>>>>>>> https://issues.apache.org/jira/browse/HDFS-9311===Life
Line
>>>> >>>>>>>>> Protocol
>>>> >>>>>>>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommis
>>>> sion
>>>> >>>>>>>>> Expensive when lot's of blocks are present
>>>> >>>>>>>>>
>>>> >>>>>>>>> https://issues.apache.org/jira/browse/HDFS-9902===
>>>> >>>>>>>>> "dfs.datanode.du.reserved"  per Storage
Type
>>>> >>>>>>>>>
>>>> >>>>>>>>> For Security:
>>>> >>>>>>>>> =========
>>>> >>>>>>>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash
does
>>>> not
>>>> >>>>>>>>> descent
>>>> >>>>>>>>> into child directories to check for
permission
>>>> >>>>>>>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade
>>>> Jsch
>>>> >>>>>>>>> jar
>>>> >>>>>>>>> to
>>>> >>>>>>>>> latest version to fix vulnerability
in old versions
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> Regards
>>>> >>>>>>>>> Brahma Reddy Battula
>>>> >>>>>>>>>
>>>> >>>>>>>>> -----Original Message-----
>>>> >>>>>>>>> From: Erik Krogen [mailto:ekrogen@linkedin.com.INVALID]
>>>> >>>>>>>>> Sent: 06 May 2017 02:40
>>>> >>>>>>>>> To: Konstantin Shvachko
>>>> >>>>>>>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev;
>>>> >>>>>>>>> mapreduce-dev@hadoop.apache.org
>>>> >>>>>>>>> ;
>>>> >>>>>>>>> yarn-dev@hadoop.apache.org
>>>> >>>>>>>>> Subject: Re: About 2.7.4 Release
>>>> >>>>>>>>>
>>>> >>>>>>>>> List LGTM Konstantin!
>>>> >>>>>>>>>
>>>> >>>>>>>>> Let's say that we will only create a
new tracking JIRA for
>>>> patches
>>>> >>>>>>>>> which
>>>> >>>>>>>>> do not backport cleanly, to avoid having
too many lying around.
>>>> >>>>>>>>> Otherwise
>>>> >>>>>>>>> we can directly attach to old ticket.
If a clean backport does
>>>> >>>>>>>>> happen
>>>> >>>>>>>>> to
>>>> >>>>>>>>> break a test the nightly build will
help us catch it.
>>>> >>>>>>>>>
>>>> >>>>>>>>> Erik
>>>> >>>>>>>>>
>>>> >>>>>>>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin
Shvachko <
>>>> >>>>>>>>> shv.hadoop@gmail.com>
>>>> >>>>>>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>> Great Zhe. Let's monitor the build.
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>> I marked all jiras I knew of for
inclusion into 2.7.4 as I
>>>> >>>>>>>>>> described
>>>> >>>>>>>>>> before.
>>>> >>>>>>>>>> Target Version/s: 2.7.4
>>>> >>>>>>>>>> Label: release-blocker
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Here is the link to the list: https://s.apache.org/Dzg4
>>>> Please
>>>> >>>>>>>>>> let
>>>> >>>>>>>>>> me
>>>> >>>>>>>>>> know if I missed anything.
>>>> >>>>>>>>>> And feel free to pick up any. Most
of backports are pretty
>>>> >>>>>>>>>> straightforward, but not all.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> We can create tracking jiras for
backporting if you need to
>>>> run
>>>> >>>>>>>>>> Jenkins on the patch (and since
Allen does not allow reopening
>>>> >>>>>>>>>> them).
>>>> >>>>>>>>>> But I think the final patch should
be attached to the original
>>>> >>>>>>>>>> jira.
>>>> >>>>>>>>>> Otherwise history will be hard to
follow.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Thanks,
>>>> >>>>>>>>>> --Konstantin
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> On Wed, May 3, 2017 at 4:53 PM,
Zhe Zhang <zhz@apache.org>
>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Thanks for volunteering as RM Konstantin!
The plan LGTM.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>> I've created a nightly Jenkins
job for branch-2.7 (unit
>>>> tests):
>>>> >>>>>>>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> On Wed, May 3, 2017 at 12:42
AM Konstantin Shvachko <
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> shv.hadoop@gmail.com>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Hey guys,
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>> I and a few of my colleagues
would like to help here and
>>>> move
>>>> >>>>>>>>>>>> 2.7.4
>>>> >>>>>>>>>>>> release forward. A few points
in this regard.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> 1. Reading through this
thread since March 1 I see that
>>>> Vinod
>>>> >>>>>>>>>>>> hinted on managing the release.
Vinod, if you still want the
>>>> >>>>>>>>>>>> job /
>>>> >>>>>>>>>>>> have bandwidth will be happy
to work with you.
>>>> >>>>>>>>>>>> Otherwise I am glad to volunteer
as the release manager.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> 2. In addition to current
blockers and criticals, I would
>>>> like
>>>> >>>>>>>>>>>> to
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> propose
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>> a
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> few issues to be included in
the release, see the list below.
>>>> >>>>>>>>>>>> Those
>>>> >>>>>>>>>>>> are mostly bug fixes and
optimizations, which we already
>>>> have in
>>>> >>>>>>>>>>>> our
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> internal
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>> branch and run in production. Plus
one minor feature "node
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> labeling", which we found very
handy, when you have
>>>> heterogeneous
>>>> >>>>>>>>>>>> environments and mixed workloads,
like MR and Spark.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> 3. For marking issues for
the release I propose to
>>>> >>>>>>>>>>>>  - set the target version
to 2.7.4, and
>>>> >>>>>>>>>>>>  - add a new label "release-blocker"
>>>> >>>>>>>>>>>> That way we will know issues
targeted for the release
>>>> without
>>>> >>>>>>>>>>>> reopening them for backports.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> 4. I see quite a few people
are interested in the release.
>>>> With
>>>> >>>>>>>>>>>> all
>>>> >>>>>>>>>>>> the help I think we can
target to release by the end of May.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> Other things include fixing
CHANGES.txt and fixing Jenkins
>>>> build
>>>> >>>>>>>>>>>> for
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> 2.7.4
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>> branch.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>> --Konstantin
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> ==========  List of issue
for 2.7.4  ===========
>>>> >>>>>>>>>>>> ------ Backports
>>>> >>>>>>>>>>>> HADOOP-12975 <https://issues.apache.org/jir
>>>> >>>>>>>>>>>> a/browse/HADOOP-12975>.
>>>> >>>>>>>>>>>> Add
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> du
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>> jitters
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>.
>>>> IBR
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> batching
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>> HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715
>>>> >.
>>>> >>>>>>>>>> NPE
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> when applying AvailableSpaceBlockPlacementPolicy
>>>> >>>>>>>>>>>> HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538
>>>> >.
>>>> >>>>>>>>>>>> fsck
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> removal
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>> of dot printing
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>.
>>>> >>>>>>>>>>>> space-balanced
>>>> >>>>>>>>>>>> policy for balancer
>>>> >>>>>>>>>>>> HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549
>>>> >.
>>>> >>>>>>>>>>>> abort
>>>> >>>>>>>>>>>> balancer if upgrade in progress
>>>> >>>>>>>>>>>> HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412
>>>> >.
>>>> >>>>>>>>>>>> skip
>>>> >>>>>>>>>>>> small blocks in getBlocks
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471
>>>> >.
>>>> >>>>>>>>>>>> SLS
>>>> >>>>>>>>>>>> simulator
>>>> >>>>>>>>>>>> YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302
>>>> >.
>>>> >>>>>>>>>>>> SLS
>>>> >>>>>>>>>>>> YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367
>>>> >.
>>>> >>>>>>>>>>>> SLS
>>>> >>>>>>>>>>>> YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612
>>>> >.
>>>> >>>>>>>>>>>> SLS
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> ----- Node labeling
>>>> >>>>>>>>>>>> MAPREDUCE-6304
>>>> >>>>>>>>>>>> <https://issues.apache.org/jira/browse/MAPREDUCE-6304>
>>>> >>>>>>>>>>>> YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943>
>>>> >>>>>>>>>>>> YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109>
>>>> >>>>>>>>>>>> YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140>
>>>> >>>>>>>>>>>> YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250>
>>>> >>>>>>>>>>>> YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925>
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> --
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> Zhe Zhang
>>>> >>>>>>>>>>> Apache Hadoop Committer
>>>> >>>>>>>>>>> http://zhe-thoughts.github.io/about/
| @oldcap
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>
>>>> >>>>> ------------------------------------------------------------
>>>> ---------
>>>> >>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>>> >>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>
>>>> >
>>>>
>>>
>>
Mime
View raw message