hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaeboo Jeong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6494) add mounting of HDFS Short-Circuit path for docker containers
Date Mon, 31 Jul 2017 15:24:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107424#comment-16107424

Jaeboo Jeong commented on YARN-6494:

Thank you for the feedbacks.

[~daniel@cloudera.com] Current patch was referenced another test cases in TestDockerContainerRuntime.
DockerLinuxContainerRuntime write the docker command to a temporary file, so I think it's
not bad to compare command strings with the contents of the temporary file. Of course, it
is true that it is difficult to memorize the order of commands and make sure that they are

[~shanekumpf@gmail.com] I agree with the white list for safe volume mounts.
However I thought this case is different from ordinary file mounts. I think that the path
for the short circuit is the configuration of the cluster, and the cluster user doesn't need
to consider about that path. I just think it's just part of the cluster setup.

> add mounting of HDFS Short-Circuit path for docker containers
> -------------------------------------------------------------
>                 Key: YARN-6494
>                 URL: https://issues.apache.org/jira/browse/YARN-6494
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Jaeboo Jeong
>            Assignee: Jaeboo Jeong
>         Attachments: YARN-6494.001.patch, YARN-6494.002.patch
> Currently there is a error message about HDFS short-circuit when docker container start.
> {code}
> WARN [main] org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory: error creating DomainSocket
> java.net.ConnectException: connect(2) error: No such file or directory when trying to
connect to ‘xxxxxxx’
>     at org.apache.hadoop.net.unix.DomainSocket.connect0(Native Method)
>     at org.apache.hadoop.net.unix.DomainSocket.connect(DomainSocket.java:250)
>     at org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory.createSocket(DomainSocketFactory.java:164)
>     at org.apache.hadoop.hdfs.BlockReaderFactory.nextDomainPeer(BlockReaderFactory.java:752)
> ...
> {code}
> if dfs.client.read.shortcircuit is true and dfs.domain.socket.path isn't equal “”,
we need to mount volume for short-circuit path.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message