hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Deleted] (HADOOP-11488) Difference in default connection timeout for S3A FS
Date Mon, 19 Jan 2015 17:36:41 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-11488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Harsh J updated HADOOP-11488:
    Comment: was deleted

(was: Thanks for taking a look Steve.

For (1) the patch is just adding newline to EOF which was amiss in the document - I don't
think thats a bad thing, although yes it is unrelated. I'm willing to let that pass, as it
is not intrusive (like say, reordering of import statements).

For (2), I wasn't aware we didn't run them by default. Can you kindly also explain how to
run the tests manually so I may do so? I'll also file a separate JIRA to drop a README under
the hadoop-aws directory if it helps, as it is not clear how to otherwise.

Is your -1 just for these two points or do you also have a concern w.r.t. fixing the discrepancy
of timeouts in the code (Constants.java) vs. the preceding XML (core-default.xml)?)

> Difference in default connection timeout for S3A FS
> ---------------------------------------------------
>                 Key: HADOOP-11488
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11488
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 2.6.0
>            Reporter: Harsh J
>            Assignee: Daisuke Kobayashi
>            Priority: Minor
>         Attachments: HADOOP-11488.patch
> The core-default.xml defines fs.s3a.connection.timeout as 5000, and the code under hadoop-tools/hadoop-aws
defines it as 50000.
> We should update the former to 50s so it gets taken properly, as we're also noticing
that 5s is often too low, especially in cases such as large DistCp operations (which fail
with {{Read timed out}} errors from the S3 service).

This message was sent by Atlassian JIRA

View raw message