mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Timothy Chen" <tnac...@apache.org>
Subject Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.
Date Mon, 03 Nov 2014 23:23:18 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review59674
-----------------------------------------------------------



src/launcher/fetcher.cpp
<https://reviews.apache.org/r/27483/#comment100969>

    I think we should leave some comments why we're trying to use HDFS (I think more specifically
the HDFS client right?) to fetch all other URIs, the original motivation of the refactor.


- Timothy Chen


On Nov. 3, 2014, 11:09 p.m., Ankur Chauhan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> -----------------------------------------------------------
> 
> (Updated Nov. 3, 2014, 11:09 p.m.)
> 
> 
> Review request for mesos and Timothy St. Clair.
> 
> 
> Bugs: MESOS-1711
>     https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what URIs could
be fetched by hadoop (if available). This is now changed such that we first check if hadoop
can fetch them for us and then we fallback to the os::net and then a local copy method (same
as it used to be). This allows users to fetch artifacts from arbitrary filesystems as long
as hadoop is correctly configured (in core-site.xml).
> 
> 
> Diffs
> -----
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> -------
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message