hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14215) DynamoDB client should waitForActive on existing tables
Date Wed, 05 Apr 2017 20:50:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957642#comment-15957642
] 

Mingliang Liu commented on HADOOP-14215:
----------------------------------------

During test, I found a corner case beyond v0 patch: when the table is being created by another
process, it waits it for active w/ v0 patch, and it will try to verify the table version compatibility
immediately. This can happen *before* the version marker is put by the other process that
is creating the table. This is a race condition. The exception is like:
{code}
java.io.IOException: S3Guard table lacks version marker. Table: mliu-s3guard
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.verifyVersionCompatibility(DynamoDBMetadataStore.java:764)
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:727)
	at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:239)
	at org.apache.hadoop.fs.s3a.s3guard.S3Guard.getMetadataStore(S3Guard.java:94)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:282)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3257)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:473)
{code}

So I think v0 patch is not enough to address this race condition.

p.s. I also remember that [~rajesh.balamohan] met the similar problem while he was testing
S3Guard with Hive. That is to say, w/o v0 patch, the problem is existing (though hidden mostly
by not active).

> DynamoDB client should waitForActive on existing tables
> -------------------------------------------------------
>
>                 Key: HADOOP-14215
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14215
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Sean Mackrory
>            Assignee: Sean Mackrory
>         Attachments: HADOOP-14215-HADOOP-13345.000.patch
>
>
> I saw a case where 2 separate applications tried to use the same non-pre-existing table
with table.create = true at about the same time. One failed with a ResourceInUse exception.
If a table does not exist, we attempt to create it and then wait for it to enter the active
state. If another jumps in in the middle of that, the table may exist, thus bypassing our
call to waitForActive(), and then try to use the table immediately.
> While we're at it, let's also make sure that the race condition where a table might get
created between checking if it exists and attempting to create it is handled gracefully.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message