mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Park (JIRA)" <>
Subject [jira] [Commented] (MESOS-4304) hdfs operations fail due to prepended / on path for non-hdfs hadoop clients.
Date Wed, 13 Jan 2016 06:11:40 GMT


Michael Park commented on MESOS-4304:

[~JTCunning], [~jieyu] Is this a blocker for 0.27.0? Is anyone working on this currently?

> hdfs operations fail due to prepended / on path for non-hdfs hadoop clients.
> ----------------------------------------------------------------------------
>                 Key: MESOS-4304
>                 URL:
>             Project: Mesos
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 0.26.0
>            Reporter: James Cunningham
> This bug was resolved for the hdfs protocol for MESOS-3602 but since the process checks
for the "hdfs" protocol at the beginning of the URI, the fix does not extend itself to non-hdfs
hadoop clients.
> {code}
> I0107 01:22:01.259490 17678 logging.cpp:172] INFO level logging started!
> I0107 01:22:01.259856 17678 fetcher.cpp:422] Fetcher Info: {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/530dda5a-481a-4117-8154-3aee637d3b38-S3\/root","items":[{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"maprfs:\/\/\/mesos\/storm-mesos-0.9.3.tgz"}},{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"http:\/\/\/conf\/storm.yaml"}}],"sandbox_directory":"\/mnt\/data\/mesos\/slaves\/530dda5a-481a-4117-8154-3aee637d3b38-S3\/frameworks\/530dda5a-481a-4117-8154-3aee637d3b38-0000\/executors\/word-count-1-1452129714\/runs\/4443d5ac-d034-49b3-bf12-08fb9b0d92d0","user":"root"}
> I0107 01:22:01.262171 17678 fetcher.cpp:377] Fetching URI 'maprfs:///mesos/storm-mesos-0.9.3.tgz'
> I0107 01:22:01.262212 17678 fetcher.cpp:248] Fetching directly into the sandbox directory
> I0107 01:22:01.262243 17678 fetcher.cpp:185] Fetching URI 'maprfs:///mesos/storm-mesos-0.9.3.tgz'
> I0107 01:22:01.671777 17678 fetcher.cpp:110] Downloading resource with Hadoop client
from 'maprfs:///mesos/storm-mesos-0.9.3.tgz' to '/mnt/data/mesos/slaves/530dda5a-481a-4117-8154-3aee637d3b38-S3/frameworks/530dda5a-481a-4117-8154-3aee637d3b38-0000/executors/word-count-1-1452129714/runs/4443d5ac-d034-49b3-bf12-08fb9b0d92d0/storm-mesos-0.9.3.tgz'
> copyToLocal: Expected scheme-specific part at index 7: maprfs:
> Usage: java FsShell [-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>]
> E0107 01:22:02.435556 17678 shell.hpp:90] Command 'hadoop fs -copyToLocal '/maprfs:///mesos/storm-mesos-0.9.3.tgz'
failed; this is the output:
> Failed to fetch 'maprfs:///mesos/storm-mesos-0.9.3.tgz': HDFS copyToLocal failed: Failed
to execute 'hadoop fs -copyToLocal '/maprfs:///mesos/storm-mesos-0.9.3.tgz' '/mnt/data/mesos/slaves/530dda5a-481a-4117-8154-3aee637d3b38-S3/frameworks/530dda5a-481a-4117-8154-3aee637d3b38-0000/executors/word-count-1-1452129714/runs/4443d5ac-d034-49b3-bf12-08fb9b0d92d0/storm-mesos-0.9.3.tgz'';
the command was either not found or exited with a non-zero exit status: 255
> Failed to synchronize with slave (it's probably exited)
> {code}
> After a brief chat with [~jieyu], it was recommended to fix the current hdfs client code
because the new hadoop fetcher plugin is slated to use it.

This message was sent by Atlassian JIRA

View raw message