mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-943) Provide an abstraction for asynchronous launching of subprocesses.
Date Thu, 23 Jan 2014 22:11:39 GMT

    [ https://issues.apache.org/jira/browse/MESOS-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880425#comment-13880425
] 

Jie Yu commented on MESOS-943:
------------------------------

A good reference:
https://github.com/facebook/folly/blob/master/folly/Subprocess.h

> Provide an abstraction for asynchronous launching of subprocesses.
> ------------------------------------------------------------------
>
>                 Key: MESOS-943
>                 URL: https://issues.apache.org/jira/browse/MESOS-943
>             Project: Mesos
>          Issue Type: Improvement
>            Reporter: Benjamin Mahler
>            Assignee: Benjamin Mahler
>
> This has come up during [~idownes] changes to add containerization.
> We would like to be able to run commands asynchronously like:
> {{curl -O http://foo.com/bigfile.zip}}
> Currently, there is not an easy way to do this while having:
> 1. A Future handle on the exit status of the subprocess.
> 2. The means to 'discard' the future and consequently kill the subprocess (e.g. stalled
hadoop command).
> 3. Handles to stdin, stdout, stderr of the subprocess.
> The first issue is that we need to re-work the Reaper to not reap _all_ subprocesses.
Rather, we need to allow other components to reap their own forked subprocesses without the
slave's Reaper "stealing" the exit status information. I've proposed that we move the Reaper
into libprocess initially with the only change being to reap the desired pids. (We can optimize
this later using a per-pid blocking thread or SIGCHLD).
> One concern is that if we 'leak' child processes by accidentally not reaping, we may
fill the process table with zombie processes. However, we have tight control over where our
code performs forks, and can enforce proper reaping.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message