airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shameera Rathnayaka <shameerai...@gmail.com>
Subject Orchestrator Real time Job submission improvement
Date Thu, 11 Sep 2014 14:29:13 GMT
Hi devs,


I am going to implement the $Subject


Requirement: Introduce a max job submission count for a given resource
under a given username.


Abstraction: When user submits a new experiment to the airavata, user
selects the resource (Machine) where airavata should run that experiment
(Job). That resource may have job count restriction like under one user
there can only be have X number of jobs either in Q or R state. So we need
to handle this at Orchestrator level rather than handing over the
experiment to GFac to submit the jobs where it gets rejected because of
that restriction. To do that Orchestrator need to know the job count of
particular user in that given resource.


Implementation:  HPCPullMonitor will write stat data to zookeeper,
zookeeper path would be something like
/stat/{username}/{machine}/jobs/{count}. Orchestrator will register a
watcher for this data change and that watcher will trigger when any GFac
node(Monitor component) update the job status realtime. Finished jobs will
immediately decrement the count and these changes will replicate in
Orchestrator with ZK watches.

Thanks,
Shameera.

-- 
Best Regards,
Shameera Rathnayaka.

email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/

Mime
View raw message