hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From manobal <mano...@gmail.com>
Subject HBase bulk load spawn high number of reducer tasks - any workaround
Date Mon, 14 Feb 2011 00:14:44 GMT

HBase bulk load (using configureIncrementalLoad helper method) configures the
job to create as many reducer task as the regions in the hbase table. So if
there are few hundred regions then the job would spawn few hundred reducer
tasks. This could get very slow on a small cluster..

Is there any workaround possible by using MultipleOutputFormat or something
else?

Thanks 
-- 
View this message in context: http://old.nabble.com/HBase-bulk-load-spawn-high-number-of-reducer-tasks---any-workaround-tp30917829p30917829.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message