Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of saint.ack@gmail.com
 designates 74.125.82.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type;
        b=QzhKcqzNk0Q6f0tNO/2WONYBxqODLM44zv4RC8hxKeNFjWMJXzZV9l7LT5V3+jltTS
         NrkDhJMZc96zhULPQAkOLgx7f08hLw90s1i64qewXyzwkUGIdmoG0IU9VQNVVpUmbI9u
         BDQLQq094z9AsjA032o2RTRkgWUUniOiIoAtQ=
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: <AANLkTimEWNbZ6=DL7i=4myvXytrw6sA0Se8HmUBZSsL3@mail.gmail.com>
References: <AANLkTikUy-g_=rEbySY7BTuJfkch8NByTahnHpHCjMQT@mail.gmail.com>
	<AANLkTikjLiWsQQmZq=4ETz5yA9swf8DD+aQmtoGHU2A-@mail.gmail.com>
	<AANLkTimEWNbZ6=DL7i=4myvXytrw6sA0Se8HmUBZSsL3@mail.gmail.com>
Date: Thu, 27 Jan 2011 21:24:14 -0800
Message-ID: <AANLkTik-hiGiSig+KpAP5e-EROR3LoMFLE50qAFahcS8@mail.gmail.com>
Subject: Re: Use loadtable.rb with compressed data?
From: Stack <stack@duboce.net>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Jan 27, 2011 at 9:08 PM, Nanheng Wu <nanhengwu@gmail.com> wrote:
> Hi Stack, thanks for the answers! I am reasonably sure the
> partitioning is OK because I just ran the same MR job with compression
> turned off and everything works. I'd like to move to 0.90 but for the
> short term I am stuck with 0.20. Is there anything I can do, maybe
> copy some files from the 0.90 branch and tweak them to run on 0.20?
> Please advice. thank you!
>

Don't try backporting.  You'll end up really hating us if you try to do that.

I was off in my first answer.  We read metadata from the files.  Maybe
when stuff is compressed we are doing something dumb in loadtable.rb
though we're reading metadata, not keyvalues.   Do the regions look
right?  The ones in .META.?    Do endkey and startkeys match up as you
move from one region to the next?  How many regions are there?  If
same data and it worked previously -- did the previous run have same
amount of data?  It didn't all fit into one region when you ran it
uncompressed? -- then it would seem to point at a issue w/ our loading
compressed files.

St.Ack