mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jessica" <jessicaest...@gmail.com>
Subject Re: Review Request: Updates and additions to the MPI framework
Date Tue, 08 May 2012 12:19:45 GMT


> On 2012-05-08 03:39:29, Benjamin Hindman wrote:
> > I'll get this checked in provided Jessica gives it a "Ship It". Thanks the the good
work here, I intend to make it a demonstration of how to write frameworks on Mesos!

Scratch that. I voted to ship it and then remembered an issue that I don't think has been
addressed yet. I posted this on the jira, but I haven't seen any changes for it: 

I'm running into the setuptools issue addressed in the test python framework: https://issues.apache.org/jira/browse/MESOS-130.
The locations of the eggs added to PYTHONPATH in nmpiexec [now mpiexec-mesos?] need to be
updated so that the Mesos/protobuf libraries (and setuptools) don't have to be installed on
every node. 

There also seems to be an issue with Python detecting the Mesos module from the egg in src/python/dist--I
couldn't import mesos until I unzipped the egg, no matter what directory I was in or how I
modified the PYTHONPATH. [Update: I believe it's related to the fact that the mesos egg uses
C/C++ extensions. I think it needs to use a setuptools module to list the package contents.]


- Jessica


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4768/#review7666
-----------------------------------------------------------


On 2012-05-08 01:29:06, Harvey Feng wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4768/
> -----------------------------------------------------------
> 
> (Updated 2012-05-08 01:29:06)
> 
> 
> Review request for mesos, Benjamin Hindman, Charles Reiss, and Jessica.
> 
> 
> Summary
> -------
> 
> Some updates to point out:
> 
> -nmpiexec.py
>   -> 'mpdallexit' should terminate all slaves' mpds in the ring. I moved 'driver.stop()'
to statusUpdate() so that it stops when all tasks have been finished, which occurs when the
executor's launched mpd processes have all exited. 
> -startmpd.py
>   -> Didn't remove cleanup(), and added code in shutdown() that manually kills mpd
processes. They might be useful during abnormal (cleanup) and normal (shutdown) framework/executor
termination...I think. cleanup() still terminates all mpd's in the slave, but shutdown doesn't.

>   -> killtask() stops the mpd associated with the given tid. 
>   -> Task states update nicely now. They correspond to the state of a task's associated
mpd process.
> -Readme
>   -> Included additional info on how to setup and run MPICH2 1.2 and nmpiexec on OS
X and Ubuntu/Linux
> 
> 
> This addresses bug MESOS-183.
>     https://issues.apache.org/jira/browse/MESOS-183
> 
> 
> Diffs
> -----
> 
>   frameworks/mpi/README.txt cdb4553 
>   frameworks/mpi/nmpiexec 517bdbc 
>   frameworks/mpi/nmpiexec.py a5db9c0 
>   frameworks/mpi/startmpd.py 8eeba5e 
>   frameworks/mpi/startmpd.sh 44faa05 
> 
> Diff: https://reviews.apache.org/r/4768/diff
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Harvey
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message