hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Junping Du <...@hortonworks.com>
Subject Re: [Continued] [Release thread] 2.8.0 release activities
Date Wed, 04 Jan 2017 09:41:36 GMT
Hi all Hadoopers,
     I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I
will kick off process to prepare the first RC for our 2.8.0 release. This release will include
at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k
for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with
2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and
fix the versions before kicking off the first RC.
     Our progress is slightly behind my estimation weeks ago. However, considering we just
go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.),
our achievement here is still great! Thanks everyone for keep calm and carry on the help for
the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard
on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge
and practice (include scripts for auto version check) for releasing effort.
     Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!

Thanks,

Junping

________________________________________
From: Junping Du <jdu@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities

Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before
last weekend, but it get failed for latest several runs, probably affected by recently Jenkins
down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/.
I will keep an eye for today's nightly run to see if it back to normal.


Thanks,

Junping

________________________________________
From: Akira Ajisaka <ajisakaa@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities

Thanks Junping and Andrew!

HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.

https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html

Regards,
Akira

On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <andrew.wang@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job
which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com<mailto:jdu@hortonworks.com>>
wrote:
> Hi Sangjin and all,
>
>      That sounds good. If anyone have priority fix to backport , please nominate it by
following this email thread or ping me on JIRA directly. And I agree that we should keep in
mind that our bar for 2.8 release should be high now as we want to move quickly on this release.
Non-critical fixes can wait for next one.
>
>       Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<mailto:sjlee0@gmail.com> <sjlee0@gmail.com<mailto:sjlee0@gmail.com>>
on behalf of Sangjin Lee <sjlee@apache.org<mailto:sjlee@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>; yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>;
Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting
some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to
do that, but it might be good to have folks suggest some critical fixes that need to be released
sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:jdu@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com>>>
wrote:
> Hi folks,
>      Per asked by Vinod offline, I would like to continue the effort to push 2.8 release
out in next several weeks. The plan is as following:
>      - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far,
and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release)
and will do daily monitor and update on these issues in next couple of weeks.
>      - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push
out all major/minor issues that is open. So if you are working on some major issues and target
for release in 2.8, please keep track of our release status and your work progress.
>      - I will create RC immediately once our 2.8 target issues are clean (especially
for blocker and critical issues) for voting process.
>
>      I hope we can make it within year 2016, best before Xmas holiday. And I need help
from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>>;
hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>>;
common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et
al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985.
Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jhe@hortonworks.com><mailto:jhe@hortonworks.com<mailto:jhe@hortonworks.com>>>
wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for
2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release
blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort,
resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com>>>>
wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org>>>>
wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com>>>>
wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org>>>>
wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<mailto:stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:stevel@hortonworks.com>><mailto:stevel@hortonworks.com<mailto:stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:stevel@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<mailto:vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org>><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-unsubscribe@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:yarn-dev-help@hadoop.apache.org><mailto:yarn-dev-help@hadoop.apache.org<mailto:yarn-dev-help@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:mapreduce-dev-unsubscribe@hadoop.apache.org><mailto:mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:mapreduce-dev-unsubscribe@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<mailto:mapreduce-dev-help@hadoop.apache.org><mailto:mapreduce-dev-help@hadoop.apache.org<mailto:mapreduce-dev-help@hadoop.apache.org>>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message