Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of wtan@us.ibm.com designates
 32.97.182.139 as permitted sender)
In-Reply-To: <1391028493.66723.YahooMailNeo@web140606.mail.bf1.yahoo.com>
References: 
 <OF29595329.9553A1C2-ON85257C6F.0070EFCA-85257C6F.007120FA@us.ibm.com>
 <1391028493.66723.YahooMailNeo@web140606.mail.bf1.yahoo.com>
To: user@hbase.apache.org
MIME-Version: 1.0
Subject: Re: larger HFile block size for very wide row?
Message-ID: 
 <OF106B9F34.D1BDBCCF-ON85257C6F.00727BDE-85257C6F.0072B084@us.ibm.com>
From: Wei Tan <wtan@us.ibm.com>
Date: Wed, 29 Jan 2014 15:52:43 -0500
Content-Type: multipart/alternative;
 boundary="=_alternative 0072B08185257C6F_="

--=_alternative 0072B08185257C6F_=
Content-Type: text/plain; charset="US-ASCII"

Sorry, 1000 columns, each 2K, so each row is 2M. I guess HBase will keep a 
single KV (i.e., a column rather than a row) in a block, so a row will 
span multiple blocks?

My scan pattern is: I will do range scan, find the matching row keys, and 
fetch the whole row for each row that matches my criteria.

Best regards,
Wei

---------------------------------
Wei Tan, PhD
Research Staff Member
IBM T. J. Watson Research Center
http://researcher.ibm.com/person/us-wtan


From:   lars hofhansl <larsh@apache.org>
To:     "user@hbase.apache.org" <user@hbase.apache.org>, 
Date:   01/29/2014 03:49 PM
Subject:        Re: larger HFile block size for very wide row?


You 1000 columns? Not 1000k = 1m column, I assume.
So you'll have 2MB KVs. That's a bit on the large side.

HBase will "grow" the block to fit the KV into it. It means you have 
basically one block per KV.
I guess you address these rows via point gets (GET), and do not typically 
scan through them, right?

Do you see any performance issues?

-- Lars


________________________________
 From: Wei Tan <wtan@us.ibm.com>
To: user@hbase.apache.org 
Sent: Wednesday, January 29, 2014 12:35 PM
Subject: larger HFile block size for very wide row?
 

Hi, I have a HBase table where each row has ~1000k columns, ~2K each. My 
table scan pattern is to use a row key filter but I need to fetch the 
whole row (~1000 k) columns back.

Shall I set HFile block size to be larger than the default 64K?
Thanks,
Wei

---------------------------------
Wei Tan, PhD
Research Staff Member
IBM T. J. Watson Research Center
http://researcher.ibm.com/person/us-wtan

--=_alternative 0072B08185257C6F_=--