hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bertrand Dechoux <decho...@gmail.com>
Subject Re: Query regarding HBase Mapreduce
Date Thu, 25 Oct 2012 16:04:55 GMT
Hi Amit,

You might want to add details to your question.

1) Lot of small files is a known 'problem' for Hadoop MapReduce. And you
will find information on it by searching.
I assume you have a more specific issue, what is it?

2) I am not sure what you mean by "HBase mapreduce on small files". If you
are using MapReduce with HBase as a source, you are not dealing with files
directly. If you are using HBase as a sink, then the lots of small files is
a problem which is orthogonal to the use of HBase. I don't think there is
such a thing as "HBase MapReduce". You might want to reformulate your use



On Thu, Oct 25, 2012 at 4:15 PM, amit bohra <bohra.a@gmail.com> wrote:

> Hi,
> We are working on processing of lot of small files. For processing them we
> are using HBase Mapreduce as of now. Currently we are working with files in
> the range for around few millions, but over the period of time it would
> grow to a larger extent.
> Did anyone faced any issues while working on HBase mapreduce on small
> files?
> Thanks and Regards,
> Amit Bohra

Bertrand Dechoux

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message