hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5947) Create DerbySchedulerConfigurationStore class using Derby as backing store
Date Wed, 07 Jun 2017 04:25:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040148#comment-16040148

Jonathan Hung commented on YARN-5947:

Attached YARN-5947-YARN-5734.001.patch for an initial pass at the LevelDB implementation.

It stores three things:
# confKey -> confVal
# "committedTxn" -> last good txn id stored in the db
# log.x -> LogMutation for txn x, for historical purposes / recovery

Currently it relies on the fact that MutableCSConfigurationProvider has a synchronized method
which does the logMutation and the confirmMutation, so that the logs cannot be logged/confirmed
out of order.

This patch also handles recovery of configuration, e.g. if a config is logged and the RM dies,
it will replay this log (and store it if the subsequent scheduler reinitialize succeeds).

Also there are no unit tests. Will add that.

> Create DerbySchedulerConfigurationStore class using Derby as backing store
> --------------------------------------------------------------------------
>                 Key: YARN-5947
>                 URL: https://issues.apache.org/jira/browse/YARN-5947
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Jonathan Hung
>            Assignee: Jonathan Hung
>         Attachments: YARN-5947.001.patch, YARN-5947-YARN-5734.001.patch
> DerbySchedulerConfigurationStore will extend YarnConfigurationStore for storing scheduler
configuration in a Derby embedded DB.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message