hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17292) Add observer notification before bulk loaded hfile is moved to region directory
Date Wed, 14 Dec 2016 19:20:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749199#comment-15749199
] 

Vladimir Rodionov commented on HBASE-17292:
-------------------------------------------

Ted, couple notes:

# Hooks in RegionObserver usually comes in pairs: pre- and post-. You introduce only one.
# Who else is going to use this hook in your opinion?  

{code}
   /**
   * Called before moving bulk loaded hfile to region directory.
   *
   * @param ctx
   * @param family column family
   * @param pairs List of pairs of { HFile location in staging dir, HFile path in region dir
}
   * Each pair are for the same hfile.
   * @throws IOException
   */
  void preCommitStoreFile(final ObserverContext<RegionCoprocessorEnvironment> ctx,
      final byte[] family, final List<Pair<String, Path>> pairs) throws IOException;
 
{code}

You have String and Path for file locations. Make them both Path.

> Add observer notification before bulk loaded hfile is moved to region directory
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-17292
>                 URL: https://issues.apache.org/jira/browse/HBASE-17292
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>         Attachments: 17292.v1.txt
>
>
> Currently the postBulkLoadHFile() hook notifies the locations of bulk loaded hfiles.
> However, if bulk load fails after hfile is moved to region directory but before postBulkLoadHFile()
hook is called, there is no way for pluggable components (replication, backup / restore) to
know which hfile(s) have been moved to region directory.
> This issue adds a preCommitStoreFile() hook which notifies path of to be committed hfile
before bulk loaded hfile is moved to region directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message