mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharma Podila <spod...@netflix.com>
Subject Re: [Junk released by User action] Re: Batch Scheduler with dependency support
Date Thu, 14 May 2015 16:45:10 GMT
Hi Aaron, yeah, an in-house one.


On Thu, May 14, 2015 at 1:09 AM, Aaron Carey <acarey@ilm.com> wrote:

>  Hi Sharma,
>
> This sounds eerily familiar! Was this an in-house system you were working
> on, or a commercial product?
>
> Thanks,
> Aaron
>
>  ------------------------------
> *From:* Sharma Podila [spodila@netflix.com]
> *Sent:* 13 May 2015 23:49
> *To:* user@mesos.apache.org
> *Cc:* Douglas Thain; Brian Bockelman
> *Subject:* [Junk released by User action] Re: Batch Scheduler with
> dependency support
>
>    ​I keep longing for folks with decades of experience in HTC&HPC to
>> chime in "on-list".
>
>
>  FWIW, I come from that background, but, am not in that space at this
> time. My prior life was in developing a (not open source) distributed job
> scheduler and management system for batch and interactive jobs that handled
> dependencies, deadlines, preemptions, advance reservation of resources,
> etc. with multi-level priority and share tree hierarchy based allocation.
> Typically, dependencies and deadlines are handled outside of schedulers and
> fed into schedulers as task submission after dependencies have been met. We
> found it more optimal to have the scheduler resolve dependencies and
> deadlines inherently. This way, a high priority job dependent on another
> low priority job can induce higher priority on that dependent job.
> Similarly, a job with a deadline depending on another job's completion can
> induce an earlier launch of the latter job in order to meet it's deadline.
> Also, a dependent job can reserve its resources in advance, knowing the
> expected completion time of its dependent jobs. This was important because
> in that environment we always had more jobs to run than can run on
> available resources. It wasn't unusual to have 10s of 1000s of jobs waiting
> in queue to run during the day.
>
>  Not sure if this helps the original question in this thread in any way.
> But, I am glad to share my learning, if that helps.
>
>  Sharma
>
>
> On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <tstclair@redhat.com> wrote:
>
>>  Hi Alex,
>>
>>  Have you by chance integrated with any of the tradition batch DAG
>> systems?
>>
>>  http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/
>>
>>   ​​
>> I keep longing for folks with decades of experience in HTC&HPC to chime
>> in "on-list".
>>
>>  Subtle nudge ;-)
>> Tim
>>
>>  ------------------------------
>>
>> *From: *"Alex Gaudio" <adgaudio@gmail.com>
>> *To: *user@mesos.apache.org
>> *Sent: *Wednesday, May 13, 2015 3:04:20 PM
>>
>> *Subject: *Re: Batch Scheduler with dependency support
>>
>>  Hi Tim (and everyone else!),
>>
>>  I am the primary author of Stolos.  We use Stolos to run all of our
>> batch jobs on Mesos.  The batch jobs are scripts we can run from the
>> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>>
>>  It's a great tool for us because, unlike Chronos, it lets us define a
>> script as stage in a dependency chain, where the script can run with
>> different parameters for different dependency contexts.  (The closest usage
>> of this would be to have many Chronos servers, though this does not work in
>> all cases).
>>
>>  The tool is a critical component of Sailthru's data science
>> infrastructure, but I believe we are the only people who use the tool right
>> now.
>>
>>  If you are interested in learning more, I'm happy to invest time to
>> talk more about Stolos, what it does and how we use it!
>>
>>  Alex
>>
>> On Wed, May 13, 2015 at 2:02 PM Tim Chen <tim@mesosphere.io> wrote:
>>
>>> How are you running your batch jobs? Is the batch job script/executable
>>> an in-house app?
>>>
>>>  Tim
>>>
>>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>>> andras.kerekes@ishisystems.com> wrote:
>>>
>>>>  You might want to have a look at stolos too:
>>>>
>>>>
>>>>
>>>> https://github.com/sailthru/stolos
>>>>
>>>>
>>>>
>>>> Andras
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* RE: Batch Scheduler with dependency support
>>>>
>>>>
>>>>
>>>> Thanks! I hadn't come across that one before :)
>>>>  ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>> *Sent:* 13 May 2015 16:39
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>
>>>> Lookup Hubspot's Singularity
>>>>
>>>>  On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com> wrote:
>>>>
>>>> Thanks Jeff,
>>>>
>>>>  Any other options around as well?
>>>>  ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>>>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>>>> *Sent:* 13 May 2015 14:12
>>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>>> *Subject:* Batch Scheduler with dependency support
>>>>
>>>> It does both just as well, along with cron-like functionality. It is
>>>> harder to install and takes a bit more understanding however. The official
>>>> tutorial is a process that loops 100 times and then exits.
>>>>
>>>>
>>>>
>>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>>
>>>>  Aurora is pretty much a superset of most other generic frameworks
>>>> sans maybe hubspot's singularity.
>>>>
>>>>
>>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>>> <http://UrlBlockedError.aspx>> wrote:
>>>>
>>>> I was under the impression Aurora was for long running services? Is it
>>>> suitable for scheduling one of batch processes too?
>>>>
>>>>  thanks,
>>>> Aaron
>>>>  ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>> *Sent:* 13 May 2015 13:12
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>
>>>> Apache Aurora does this and you can be explicit about the ordering
>>>>
>>>>  On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com> wrote:
>>>>
>>>> Hi All,
>>>>
>>>>  I was just wondering if anyone out there knew of a good mesos batch
>>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>>> until Task A is complete)
>>>>
>>>>  Thanks,
>>>> Aaron
>>>>
>>>>
>>>>
>>>>  --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>>
>>>>  --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>>
>>>>  --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>
>>>
>>
>>
>>  --
>>  Cheers,
>> Timothy St. Clair
>> Red Hat Inc.
>>
>
>

Mime
View raw message