hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: direct Hfile Read and Writes
Date Thu, 28 Jun 2012 22:39:11 GMT
On Thu, Jun 28, 2012 at 1:40 AM, samar kumar <samar.opensource@gmail.com> wrote:
> Thanks for the replies .  I am aware of the apis but can anyone give me
> little bit more insight on the details. After creating the HFiles and
> calling IncrementalLoadHFile how does it internally change the RS, Catalog
> tables etc?
> Can anyone explain the flow.

It does not change catalog tables; catalog table only has region
layout, not what files make up a region.  The loading of an hfile into
a region just atomically updates the set of hfiles the currently open
region is using swapping in the new one.

Study the bulk load tool though.  Notice how it will make the
proffered set of hfiles fit the current region layout splitting hfiles
if necessary so they fit w/i the current region confines.  See how it
also threads the updating because update is slow done serially.  Back
up more to see how the files are created using total order
partitioner.  Leverage what bulk loader does if you want to avoid long
solved issues.


View raw message