hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian He <...@hortonworks.com>
Subject Re: [DISCUSS] Merge yarn-native-services branch into trunk
Date Fri, 08 Sep 2017 22:23:06 GMT
Hi Arun

Sorry for late reply.
* Is there a branch-2 merge planned for this ?
Branch-2 is not planned for this merge.

* I understand YARN-7126 has some introductory documentation, But I think we need to flesh
it up a bit more before release, I would also like to see steps to deploy a sample service.
We have added more documentations, QuickStart.md, Overview.md and others in the same folder.
YarnCommands.md is also updated to document the new shell commands.

I encourage everyone to try  and share suggestions.

As said in another email thread, we decided to drop this for beta and re-target it for GA.

Thanks,
Jian

On Sep 5, 2017, at 1:37 PM, Arun Suresh <asuresh@apache.org<mailto:asuresh@apache.org>>
wrote:

Thanks for all the work on this folks.
I know the VOTE thread has started for this.

But I did have a couple of questions:
* Is there a branch-2 merge planned for this ?
* I understand YARN-7126 has some introductory documentation, But I think we need to flesh
it up a bit more before release, I would also like to see steps to deploy a sample service.

Cheers
-Arun

On Thu, Aug 31, 2017 at 12:40 AM, Jian He <jhe@hortonworks.com<mailto:jhe@hortonworks.com>>
wrote:
Update:
I’ve chatted with Andrew offline, we’ll proceed with merging yarn-native-services into
trunk for beta.
We’ll advertise this feature as “alpha"
Currently, we have completed all the jiras for this merge - I’ve also moved out the subtasks
that are not blocking this merge.

I’ve created YARN-7127 to run the entire patch against trunk, once that goes green, I plan
to start a formal vote.

Thanks,
Jian

On Aug 18, 2017, at 2:48 PM, Andrew Wang <andrew.wang@cloudera.com<mailto:andrew.wang@cloudera.com><mailto:andrew.wang@cloudera.com<mailto:andrew.wang@cloudera.com>>>
wrote:

Hi Jian, thanks for the reply,

On Thu, Aug 17, 2017 at 1:03 PM, Jian He <jhe@hortonworks.com<mailto:jhe@hortonworks.com><mailto:jhe@hortonworks.com<mailto:jhe@hortonworks.com>>>
wrote:
Thanks Andrew for the comments. Answers below:

- There are no new APIs added in YARN/Hadoop core. In fact, all the new code are running outside
of existing system and they are optional and require users to explicitly opt in. The new system’s
own rest API is not stable and will be evolving.

Great! That adds a lot more confidence that this is safe to merge.

Are these new APIs listed in user documentation, and described as unstable?

- We have been running/testing a version of the entire system internally for quite a while.

Do you mind elaborating on the level of testing? Number of nodes, types of applications, production
or test workload, etc. It'd help us build confidence.

- I’d like to see this in hadoop3-beta1. Of course, we’ll take responsibility of moving
fast and not block the potential timeline.

Few more questions:

How should we advertise this feature in the release? Since the APIs are unstable, I'd propose
calling it "alpha" in the release notes, like we do the TSv2.

Could you move out subtasks from YARN-5079 that are not blocking the merge? This would make
it easier to understand what's remaining.

Thanks,
Andrew



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message