hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory
Date Thu, 15 Jun 2017 04:10:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049974#comment-16049974

Sean Mackrory commented on HADOOP-14488:

[~fabbri] Thanks - +1 to the changes as you describe them, but I don't see an updated attachment?

{quote}As an aside, I think we need to address the inconsistency tests being slow, due to
code like this:{quote}

I've been thinking about that too. I thought about doing it with a sort of "fast forward"
that increases a delta that the client adds on the current time when checking the age of it's
metadata. But then that gets out of sync with timestamps on new metadata that's created in
the metadata store - not sure if that would cause problems. I fear literally doing a flush()
of its metadata might cause problems too, as we're then immediately querying just the underlying
store without a delay we used to have? Anyway - will discuss further on the separate JIRA.

> s3guard w/ failure injection: listStatus fails after renaming file into directory
> ---------------------------------------------------------------------------------
>                 Key: HADOOP-14488
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14488
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Steve Loughran
>            Assignee: Sean Mackrory
>            Priority: Blocker
>         Attachments: HADOOP-14488-HADOOP-13345-001.patch, HADOOP-14488-HADOOP-13345-002.patch,
HADOOP-14488-HADOOP-13345-003.patch, HADOOP-14488-HADOOP-13345-004.patch, output.txt
> Running scala integration test with inconsistent s3 client & local DDB enabled
> {code}
> fs.rename("work/task-00/part-00", work)
> fs.listStatus(work)
> {code}
> The list status work fails with a message about the childStatus not being a child of
the parent. 
> Hypothesis: rename isn't updating the child path entry

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message