hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Use loadtable.rb with compressed data?
Date Fri, 28 Jan 2011 05:24:14 GMT
On Thu, Jan 27, 2011 at 9:08 PM, Nanheng Wu <nanhengwu@gmail.com> wrote:
> Hi Stack, thanks for the answers! I am reasonably sure the
> partitioning is OK because I just ran the same MR job with compression
> turned off and everything works. I'd like to move to 0.90 but for the
> short term I am stuck with 0.20. Is there anything I can do, maybe
> copy some files from the 0.90 branch and tweak them to run on 0.20?
> Please advice. thank you!
>

Don't try backporting.  You'll end up really hating us if you try to do that.

I was off in my first answer.  We read metadata from the files.  Maybe
when stuff is compressed we are doing something dumb in loadtable.rb
though we're reading metadata, not keyvalues.   Do the regions look
right?  The ones in .META.?    Do endkey and startkeys match up as you
move from one region to the next?  How many regions are there?  If
same data and it worked previously -- did the previous run have same
amount of data?  It didn't all fit into one region when you ran it
uncompressed? -- then it would seem to point at a issue w/ our loading
compressed files.

St.Ack

Mime
View raw message