hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlo Curino <ccur...@microsoft.com>
Subject RE: "Reservation" ambiguity
Date Wed, 05 Aug 2015 17:16:16 GMT
+1 on keeping the name "reservation" for the user-visible (2). 

On top of the external/internal argument that Chris makes (which I completely agree with),
I noticed the following:

While developing (2)  we spoke with lots and lots of folks both in industry and academia,
and the term 
"reservation" was very evocative and intuitive. Within seconds people were using it to refer
to the functionality 
and easily grasping the idea.  On the other hand, every time I spoke about (1) using the keyword
"reservation",
I had to add a bunch of context, expand, explain, and even then people were naturally drawn
to refer to it 
as "hoarding of resources for large containers", or "large container management".

Other alternative names for (1) could be: "hoarded" or "prefecthed" resources.

My 2 cents...

Cheers,
Carlo

-----Original Message-----
From: Karthik Kambatla [mailto:kasha@cloudera.com] 
Sent: Wednesday, August 5, 2015 8:20 AM
To: yarn-dev@hadoop.apache.org
Subject: Re: "Reservation" ambiguity

Inline.

On Tue, Aug 4, 2015 at 6:48 PM, Chris Douglas <cdouglas@apache.org> wrote:

> How visible are (1) reservations? They're an internal, implementation 
> detail exposed in metrics only to explain the edge cases they create.
> Are users typically aware of them?
>

This is internal, and I don't think users are aware of the mechanics.
However, they do see metrics for "reserved" resources.


>
> SLA reservations (2) are user-visible, and express the contract with 
> users/operators symmetrically. While (1) is a concept, renaming (2) 
> would require user-breaking code changes.
>

Yes, I don't think we should rename (2).


>
> Unless you're discussing the intersection- the effect of reservations
> (1) on a reservation (2)- it's usually clear from context... I'd 
> rather avoid breaking anyone listening to the metrics in Hadoop-3.
>

I propose to add new metrics holdMB, holdCores for reservedMB, reseveredCores. Could we deprecate
the older metrics in Hadoop-2 and Hadoop-3, and remove them in Hadoop-4?


>
> Maybe reservations (2) could have been named "sessions", but that 
> collided with applications that already used it for a similar concept.
> -C
>
> On Wed, Jul 29, 2015 at 10:37 AM, Karthik Kambatla 
> <kasha@cloudera.com>
> wrote:
> > Hi folks
> >
> > We use the word "reservation" to mean both (1) reservations on nodes 
> > to avoid starvation of big container asks, and (2) the recent SLA 
> > work. This is confusing both to developers and end-users.
> >
> > I was wondering if people are open to calling the first one a "hold" 
> > and the second one a "reservation". We can change the terminology in 
> > the code and add new metrics for hold in branch-2 and remove the 
> > metrics for
> > reserved* in Hadoop-3?
> >
> > Thoughts?
>
Mime
View raw message