hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/HBaseBulkLoad" by JohnSichi
Date Mon, 14 Feb 2011 21:48:13 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/HBaseBulkLoad" page has been changed by JohnSichi.
http://wiki.apache.org/hadoop/Hive/HBaseBulkLoad?action=diff&rev1=22&rev2=23

--------------------------------------------------

  In order to perform a parallel sort on the data, we need to range-partition it.  The idea
is to divide the space of row keys up into nearly equal-sized ranges, one per reducer which
will be used in the parallel sort.  The details will vary according to your source data, and
you may need to run a number of exploratory Hive queries in order to come up with a good enough
set of ranges.  Here's one example:
  
  {{{
- add jar lib/hive_contrib.jar;
+ add jar lib/hive-contrib-0.7.0.jar;
  set mapred.reduce.tasks=1;
  create temporary function row_sequence as 
  'org.apache.hadoop.hive.contrib.udf.UDFRowSequence';

Mime
View raw message