hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-10642) Add M/R over snapshots to 0.94
Date Mon, 10 Mar 2014 05:29:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13925455#comment-13925455
] 

Lars Hofhansl edited comment on HBASE-10642 at 3/10/14 5:28 AM:
----------------------------------------------------------------

V4 has the block distribution logic, and scan metrics, but keeps changes to existing HBase
classes to a minimum.

[~enis], could you have a look, especially at the block distribution stuff? (The existing
0.94 patch picked up the distribution from the table, not the snapshot, I am not sure the
HFileLinks influence this and whether even the trunk patch does the right thing - does it
follow HFileLinks? If not, how does it find the real file distribution?).
Also, in the trunk version I notice that we update the counters after each record, is that
by design? Seems CPU heavy.



was (Author: lhofhansl):
V4 has the block distribution logic, and scan metrics, but keeps changes to existing HBase
classes to a minimum.

[~enis], could have a look, especially at the block distribution stuff? (The existing 0.94
picked up the distribution from the table, not the snapshot, I am not sure the HFileLinks
influence this and whether even the trunk patch does the right thing).
Also, in the trunk version I notice that we update the counters after each record, is that
by design? Seems CPU heavy.


> Add M/R over snapshots to 0.94
> ------------------------------
>
>                 Key: HBASE-10642
>                 URL: https://issues.apache.org/jira/browse/HBASE-10642
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>             Fix For: 0.94.18
>
>         Attachments: 10642-0.94-v2.txt, 10642-0.94-v3.txt, 10642-0.94-v4.txt, 10642-0.94.txt,
SnapshotInputFormat.java
>
>
> I think we want drive towards all (or most) M/R over HBase to be against snapshots and
HDFS directly.
> Adopting a simple input format (even if just as a sample) as part of HBase will allow
us to direct users this way.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message