hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-13908) S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore
Date Fri, 06 Jan 2017 23:10:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806095#comment-15806095
] 

Mingliang Liu edited comment on HADOOP-13908 at 1/6/17 11:10 PM:
-----------------------------------------------------------------

Chris and I have checked the test failure of v4 patch. We have not spotted the root cause.

Comparing with the v3 patch (now rebased from feature branch as v5), it firstly uses the getItem()
to probe; there may be some unknown internal state of table in client/sdk after that request
fails. That part we don't quite know yet. I tested the v5 patch multiple times (>10) and
no "requested resource not found" failure happened.

For this problem ie _initializing the table correctly_ vs. _using a lightweight approach to
probe the table status_. During my ~1mo PTO from next week, I'd stick with the v5 patch, and
address the latter later. However, if any help comes up, I will be very appreciated and help
debugging/reviewing/testing. Ping [~stevel@apache.org] and [~rajesh.balamohan].


was (Author: liuml07):
Chris and I have checked the test failure of v4 patch. We have not spotted the failure.

Comparing with the v3 patch (now rebased from feature branch as v5), it firstly uses the getItem()
to probe; there may be some unknown internal state of table in client/sdk after that request
fails. That part we don't quite know yet. I tested the v5 patch multiple times (>10) and
no failure happened.

For this problem ie _initializing the table correctly_ vs. _using a lightweight approach to
probe the table status_. During my ~1mo PTO from next week, I'd stick with the v5 patch, and
address the latter later. However, if any help comes up, I will be very appreciated and help
debugging/reviewing/testing. Ping [~stevel@apache.org] and [~rajesh.balamohan].

> S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-13908
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13908
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Mingliang Liu
>            Assignee: Mingliang Liu
>         Attachments: HADOOP-13908-HADOOP-13345.000.patch, HADOOP-13908-HADOOP-13345.001.patch,
HADOOP-13908-HADOOP-13345.002.patch, HADOOP-13908-HADOOP-13345.002.patch, HADOOP-13908-HADOOP-13345.003.patch,
HADOOP-13908-HADOOP-13345.004.patch, HADOOP-13908-HADOOP-13345.005.patch
>
>
> This was based on discussion in [HADOOP-13455]. Though we should not create table unless
the config {{fs.s3a.s3guard.ddb.table.create}} is set true, we still have to get the existing
table in {{DynamoDBMetadataStore#initialize()}} and wait for its becoming active, before any
table/item operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message