hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ioannis Konstantinou (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1861) Multi-Family support for bulk upload tools (HFileOutputFormat / loadtable.rb)
Date Sat, 09 Jan 2010 16:30:54 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798375#action_12798375

Ioannis Konstantinou commented on HBASE-1861:

Hi again. One thing I noticed during bulk upload (of a single column family) is a bug in the
following scenario (correct me if this is not the case): 
I have a mapper that reads input and emmits KeyValue objects to be fed in the KeyValueSortReducer.
The mapper emmits a number of KeyValue objects for each row. For the same rowid, the KeyValue
objects have different columnids. 
The problem is the following: when these KeyValue objects (that have the same rowid but different
colids in the same column family) reach the reducer, the TreeSet used to sort KeyValues, keeps
only the KeyValue that gets last (it replaces all entries with the last one that reaches the
reducer), as the KeyValue.COMPARATOR compares only the rowid !!!!!
Can I use a different Comparator??? KeyValue objects of the same rowid must be sorted before
writing them in the Hfile, or this does not matter???

> Multi-Family support for bulk upload tools (HFileOutputFormat / loadtable.rb)
> -----------------------------------------------------------------------------
>                 Key: HBASE-1861
>                 URL: https://issues.apache.org/jira/browse/HBASE-1861
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: mapreduce
>    Affects Versions: 0.20.0
>            Reporter: Jonathan Gray
>             Fix For: 0.21.0
> Add multi-family support to bulk upload tools from HBASE-48.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message