hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1057) Example MR jobs to simulate bulk importing
Date Fri, 12 Dec 2008 20:45:44 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Gray updated HBASE-1057:
---------------------------------

    Attachment: ImportTestMR_v1.java

MR job with tweakable parameters that bulk uploads rows with fixed-length and variable-length
columns and values.

Currently requires a single line HDFS file as input and an HBase table that exists as output,
though this uses the API directly and not TOF.  Someone better with MR might be able to remove
these two requirements.

There are some commented out lines in the case that you want to use write batching.

Can be easily modified to use TOF instead.

> Example MR jobs to simulate bulk importing
> ------------------------------------------
>
>                 Key: HBASE-1057
>                 URL: https://issues.apache.org/jira/browse/HBASE-1057
>             Project: Hadoop HBase
>          Issue Type: New Feature
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>            Priority: Trivial
>         Attachments: ImportTestMR_v1.java
>
>
> It's very useful to have standalone MR jobs that simulate production system load characteristics.
 Specifically bulk importing as this has been uncovering OOME and long-running compaction
issues.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message