mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Erlandson <...@redhat.com>
Subject Re: Adding GPUs/FPGAs into the resources
Date Wed, 26 Mar 2014 22:06:52 GMT


----- Original Message -----
> I think many can agree that *this is a useful feature, but be forewarned,
> "Thar be dragons in this space!"
> 
> There is a high degree of variability across GPU's, their abilities, and
> apis. That shouldn't deter folks, but it may be wise to leverage other
> communities on how they approached this problem, and learn from their wisdom
> as well as mistakes.
> 
> - Simpler is better
> - Think non-fungible
> - Script for discovery
> 
> A couple of quick refs:
> older ref: https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToManageGpus
> http://techblog.netflix.com/2014/02/distributed-neural-networks-with-gpus.html
> I'll let Erik point out the other refs, he has spent a large amount of time
> in this area, and has looked @ (N) resources.




+1 to thinking about 'non-fungible' resources as early as possible.  Allowing resources to
be assigned with individual identities requires more machinery.  HTCondor has recently been
adding support for non-fungible resources:
https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=4141

I've spent more time thinking about managing heterogeneous resources, than non-fungible resources,
per se.   Some possibly useful links to my thoughts (with a htcondor slant) are:

http://erikerlandson.github.io/blog/2012/11/13/rethinking-the-semantics-of-group-quotas-and-slot-weights-for-heterogeneous-and-multidimensional-compute-resources/
http://erikerlandson.github.io/blog/2012/11/15/rethinking-the-semantics-of-group-quotas-and-slot-weights-claim-capacity-model/
http://erikerlandson.github.io/blog/2012/11/26/rethinking-the-semantics-of-group-quotas-and-slot-weights-computing-claim-capacity-from-consumption-policy/

https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=ConsumptionPolicies


Mime
View raw message