hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14335) Improve DynamoDB schema update story
Date Fri, 21 Apr 2017 12:19:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15978608#comment-15978608

Steve Loughran commented on HADOOP-14335:

I added the version in there so we can at least recognise that there are version mismatches;
without that we'd be stuck. At least now we can recognise the problem.

I didn't add any logic about forward/backward compat as I didn't know what to add. Same for
an update tool: didn't write one until is was needed

Maybe we could add a minor version marker and have logic of "guaranteed compatibility across
minor versions, but not major ones".

One aspect of table update is we may also want to add an offline marker to the DB too; when
doing a schema update or other maintenance, you should be able to mark the DB as offline;
clients would be expected to see this and for non-auth: downgrade, for auth: fail.

> Improve DynamoDB schema update story
> ------------------------------------
>                 Key: HADOOP-14335
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14335
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Sean Mackrory
>            Assignee: Sean Mackrory
> On HADOOP-13760 I'm realizing that changes to the DynamoDB schema aren't great to deal
with. Currently a build of Hadoop is hard-coded to a specific schema version. So if you upgrade
from one to the next you have to upgrade everything (and then update the version in the table
- which we don't have a tool or document for) before you can keep using S3Guard. We could
possibly also make the definition of compatibility a bit more flexible, but it's going to
be very tough to do that without knowing what kind of future schema changes we might want
ahead of time.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message