hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Vissapragada <bhara...@cloudera.com>
Subject Re: Is hbase call delete request for old hfile(storeFile) file if minor/major compaction happen for them
Date Tue, 25 Feb 2014 14:47:11 GMT
Hi,

I believe CatalogJanitor is for a different purpose (cleaning meta post
splits). He should be looking at o.a.h.h.master.cleaner.* ! Anway files are
deleted.

- Bharath


On Tue, Feb 25, 2014 at 5:43 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi Upendra,
>
> Olds files are moved by HBase later.  Take a look at
> org.apache.hadoop.hbase.master.CatalogJanitor
>
> JM
>
>
> 2014-02-25 6:25 GMT-05:00 Upendra Yadav <upendra1024@gmail.com>:
>
> > Thanks...
> >
> >
> > On Tue, Feb 25, 2014 at 2:18 PM, Bharath Vissapragada <
> > bharathv@cloudera.com
> > > wrote:
> >
> > > In short, it needn't "download" files, that metadata is stored and
> > exposed
> > > in the API of a StoreFile and regionserver just calls that method!
> > >
> > >
> > > On Tue, Feb 25, 2014 at 2:16 PM, Bharath Vissapragada <
> > > bharathv@cloudera.com
> > > > wrote:
> > >
> > > > Not sure what you are asking but if you want to know the size of a
> > store
> > > > file, you can use this method,
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java#L1440
> > > >
> > > > One such usage is here,
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java#L292
> > > >
> > > > It gets the sizes of all the store files, I believe thats what you
> are
> > > > asking! Please correct me if I'm wrong,
> > > >
> > > >
> > > > On Tue, Feb 25, 2014 at 12:37 PM, Upendra Yadav <
> upendra1024@gmail.com
> > > >wrote:
> > > >
> > > >> Is hbase call delete request for old hfile(storeFile) file if
> > > minor/major
> > > >> compaction happen for them and generate new hfile and store it in
> > hdfs.
> > > >>
> > > >> for minor compaction how it will get size of every hfile. lets we
> have
> > > 20
> > > >> no. of hfiles for a CF and minor compaction wants to perform. Will
> it
> > > >> download all hfile and decide compaction have to do or not. Or it
> have
> > > >> in-memory hfile's size information for all hfile.
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Bharath Vissapragada
> > > > <http://www.cloudera.com>
> > > >
> > >
> > >
> > >
> > > --
> > > Bharath Vissapragada
> > > <http://www.cloudera.com>
> > >
> >
>



-- 
Bharath Vissapragada
<http://www.cloudera.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message