reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Weimer <>
Subject YARN Request IDs are here
Date Wed, 04 Oct 2017 07:16:36 GMT

the long timers might remember, but this has been a need of ours for a long
time: YARN now allows you to tag resource requests, and the allocations
that come back will have those tags:

This will greatly simplify the implementation of stuff like IMRU, where
there are different resource allocations for different Evaluators. Now it
is up to us to support that new API. I believe that the
`SchedulingConstraints` approach taken in REEF-1750 should be a workable
way to get this new feature into the `EvaluatorRequestor` APIs. For the
`AllocatedEvaluator`, we could add a new field `requestId` or such to
`EvaluatorDescriptor`. However, I think that we've reached the point where
that class might be better served by a `Map<String,String>` of metadata
about that Evaluator. That way, runtimes can communicate an open set of
metadata to the Driver.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message