hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nanheng Wu <nanhen...@gmail.com>
Subject Re: Use loadtable.rb with compressed data?
Date Fri, 28 Jan 2011 05:08:50 GMT
Hi Stack, thanks for the answers! I am reasonably sure the
partitioning is OK because I just ran the same MR job with compression
turned off and everything works. I'd like to move to 0.90 but for the
short term I am stuck with 0.20. Is there anything I can do, maybe
copy some files from the 0.90 branch and tweak them to run on 0.20?
Please advice. thank you!

On Thu, Jan 27, 2011 at 9:04 PM, Stack <stack@duboce.net> wrote:
> loadtable.rb doesn't care about file content; it just moves files and
> updates .META.
> You sure you did the partitioning correctly?  Not seeing anything
> would come of incorrectly done partitioner.  There may also have been
> a bug in partitioner around this time.  Can you move to 0.90.0?  Bulk
> uploader is much improved there (It was rewritten between 0.20.6 and
> 0.90.0 and the new implementation has been given a much better
> airing).
> Yours,
> St.Ack
> On Thu, Jan 27, 2011 at 8:54 PM, Nanheng Wu <nanhengwu@gmail.com> wrote:
>> Hi,
>> I am using hbase 0.20.6.  Is it possible for the loadtable.rb script
>> to create the table from compressed output? I have a MR job where the
>> reducer outputs Gzip compressed HFiles. When I ran loadtable.rb it
>> didn't have any complaints and seemed to update the meta data table
>> correctly. But when I tried to query against the table no data would
>> come back (scan show 0 zero etc). Does anyone know if it's possible?
>> or If I must create tables from compressed HFIles directly, what other
>> options do I have besides the script? Thanks!

View raw message