hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14564) s3a test can hang in teardown with network problems
Date Wed, 21 Jun 2017 15:55:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057738#comment-16057738
] 

Steve Loughran commented on HADOOP-14564:
-----------------------------------------

You also need to cut back on retries, as socket timeouts, by default, just trigger new attempts.
Probably good for production, not for dev.
{code}
  <property>
    <name>fs.s3a.connection.establish.timeout</name>
    <value>5000</value>
  </property>
  <property>
    <name>fs.s3a.connection.socket.timeout</name>
    <value>${fs.s3a.connection.establish.timeout}</value>
  </property>
  <property>
    <name>fs.s3a.attempts.maximum</name>
    <value>2</value>
  </property>
{code}

> s3a test can hang in teardown with network problems
> ---------------------------------------------------
>
>                 Key: HADOOP-14564
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14564
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: 2.8.1
>            Reporter: Steve Loughran
>            Priority: Minor
>
> If you've a transient network test and things fail, then the directory cleanup in teardown
can block a test so that it times out entirely. 
> Proposed: shorten socket timeouts for s3 connections, assuming this is possible (stack
trace implies it was in a read(), not a connect()).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message