mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankur Chauhan" <an...@malloc64.com>
Subject Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.
Date Mon, 03 Nov 2014 23:09:35 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/
-----------------------------------------------------------

(Updated Nov. 3, 2014, 11:09 p.m.)


Review request for mesos and Timothy St. Clair.


Changes
-------

Prefer the libcurl fetcher to the HDFS fetcher.


Bugs: MESOS-1711
    https://issues.apache.org/jira/browse/MESOS-1711


Repository: mesos-git


Description
-------

Previously, the fetcher used a hardcoded list of schemes to determine what URIs could be fetched
by hadoop (if available). This is now changed such that we first check if hadoop can fetch
them for us and then we fallback to the os::net and then a local copy method (same as it used
to be). This allows users to fetch artifacts from arbitrary filesystems as long as hadoop
is correctly configured (in core-site.xml).


Diffs (updated)
-----

  src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 

Diff: https://reviews.apache.org/r/27483/diff/


Testing
-------

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message