Return-Path: X-Original-To: apmail-mesos-dev-archive@www.apache.org Delivered-To: apmail-mesos-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 971CD109B3 for ; Sun, 2 Nov 2014 03:55:23 +0000 (UTC) Received: (qmail 45257 invoked by uid 500); 2 Nov 2014 03:55:23 -0000 Delivered-To: apmail-mesos-dev-archive@mesos.apache.org Received: (qmail 45185 invoked by uid 500); 2 Nov 2014 03:55:23 -0000 Mailing-List: contact dev-help@mesos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mesos.apache.org Delivered-To: mailing list dev@mesos.apache.org Received: (qmail 45173 invoked by uid 99); 2 Nov 2014 03:55:22 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 02 Nov 2014 03:55:22 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 7D0461DFA39; Sun, 2 Nov 2014 03:55:26 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============1297310261524480808==" MIME-Version: 1.0 Subject: Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme. From: "Ankur Chauhan" To: "Ankur Chauhan" , "mesos" Date: Sun, 02 Nov 2014 03:55:26 -0000 Message-ID: <20141102035526.7137.83888@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Ankur Chauhan" X-ReviewGroup: mesos X-ReviewRequest-URL: https://reviews.apache.org/r/27483/ X-Sender: "Ankur Chauhan" References: <20141102034751.7137.44455@reviews.apache.org> In-Reply-To: <20141102034751.7137.44455@reviews.apache.org> Reply-To: "Ankur Chauhan" X-ReviewRequest-Repository: mesos-git --===============1297310261524480808== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27483/ ----------------------------------------------------------- (Updated Nov. 2, 2014, 3:55 a.m.) Review request for mesos. Changes ------- Change fetcher to use hdfs regardless of scheme and added available method to hdfs Bugs: MESOS-1711 https://issues.apache.org/jira/browse/MESOS-1711 Repository: mesos-git Description ------- Previously, the fetcher used a hardcoded list of schemes to determine what URIs could be fetched by hadoop (if available). This is now changed such that we first check if hadoop can fetch them for us and then we fallback to the os::net and then a local copy method (same as it used to be). This allows users to fetch artifacts from arbitrary filesystems as long as hadoop is correctly configured (in core-site.xml). Diffs (updated) ----- src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 Diff: https://reviews.apache.org/r/27483/diff/ Testing ------- make check sudo bin/mesos-tests.sh --verbose support/mesos-style.py Thanks, Ankur Chauhan --===============1297310261524480808==--