hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13599) s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown
Date Tue, 13 Sep 2016 19:50:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488213#comment-15488213
] 

Chris Nauroth commented on HADOOP-13599:
----------------------------------------

{code}
  private static final AtomicBoolean closed = new AtomicBoolean(false);
{code}

Please make this a member variable, so multiple {{S3AFileSystem}} instances created within
the same process can get closed independently.  I assume this is just a copy-paste error from
{{warnedOfCoreThreadDeprecation}}, where {{static}} is appropriate.

{code}
  @Test
  public void testCloseReentrant() throws Throwable {
    conf = new Configuration();
    fs = S3ATestUtils.createTestFileSystem(conf);
    fs.close();
    fs.close();
  }
{code}

I suggest changing the name of this test method, because it doesn't really cover reentrancy.
 It does cover idempotence though, so maybe {{testCloseIdempotent}}?

> s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-13599
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13599
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.3
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-13599-branch-2-001.patch, HADOOP-13599-branch-2-002.patch
>
>
> We've had a report of hive deadlocking on teardown, as a synchronous FS close was blocking
shutdown threads, similar to HADOOP-3139
> S3a close needs to be made non-synchronized. All we need is some code to prevent re-entrancy
at the start; easily done



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message