hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikram Singh Chandel <vikramsinghchan...@gmail.com>
Subject Re: How to get Last access time of a record
Date Mon, 24 Feb 2014 21:38:08 GMT
Hi Vladimir
We are planing to have around 40Gb for L1 and 150Gb for L2 and when this
size is breached then we have start cleaning L1 and L2.
now this cleaning (deletion of records) i needed that LRU info at record
level, i.e. delete all records which are not been used past 15 days or
later.
We will save save this LRU info in a Metric column family.

What we thought of using a Post Get Observer to write the value to Last
Read column of Metric column family.
this info we will later use for deletion of records.

Is there any other simpler way. As you said block cache is at table level(
if i am correct) but we info at record level

Thanks


On Tue, Feb 25, 2014 at 1:42 AM, Vladimir Rodionov
<vrodionov@carrieriq.com>wrote:

> I recommend you work a little bit more on design.
> NoSQL in general and HBase in particular are not very good at joining
> tables, but very good at point and range queries.
>
> Sure, you can do some optimizations in your current approach: create CACHE
> table as IN_MEMORY, set TTL for say 1day (or less, depends
> on the data volume your are able to store ) and utilize HBase internal
> block cache (which is LRU) for that table.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
>
> ________________________________________
> From: Vikram Singh Chandel [vikramsinghchandel@gmail.com]
> Sent: Monday, February 24, 2014 11:38 AM
> To: user@hbase.apache.org
> Subject: Re: How to get Last access time of a record
>
> Hi Vladimir,
>
> We are going to implement cache in HBase, let me give you a example
>
> We have two tables
> Table1: 80 m records say Author
> Table2 : 5k records say Category
> query : Get details of all publications by Author XYZ broken down by
> Category
>
> We fire a get on Table 1 to get a list of publications ids(hashed)
> Then we do a scan on  Table 2 to get list of publications for each category
> and then we do Intersection
> of both list and in the end get the details from publication table.
>
> Now suppose same query comes again instead of doing all this computation
> again we are going to save the intersected results
> in a table we are calling L2 Cache (there's a L1 also)
>
> Hope you would have got idea of what we are trying to achieve.
> Now if you can help please
>
>
>
>
>
> On Tue, Feb 25, 2014 at 12:20 AM, Vladimir Rodionov <
> vrodionov@carrieriq.com
> > wrote:
>
> > Interesting. You want to use HBase as a cache. What data are going to
> > cache? Is it some kind of a cold storage
> > on tapes or Blu-Ray disks? Just curious.
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: vrodionov@carrieriq.com
> >
> > ________________________________________
> > From: Vikram Singh Chandel [vikramsinghchandel@gmail.com]
> > Sent: Monday, February 24, 2014 4:25 AM
> > To: user@hbase.apache.org
> > Subject: Re: How to get Last access time of a record
> >
> > Hi
> > Hbase provides cache on non processed data, we are implementing a second
> > level of caching on processed data,
> > for eg on intersected data between two tables, or on post processed data.
> >
> >
> > On Mon, Feb 24, 2014 at 5:02 PM, haosdent <haosdent@gmail.com> wrote:
> >
> > > HBase have already maintained a cache.
> > >
> > > >we can get last accessed time for a record
> > >
> > > I think you could get this from your application level.
> > >
> > >
> > > On Mon, Feb 24, 2014 at 7:21 PM, Vikram Singh Chandel <
> > > vikramsinghchandel@gmail.com> wrote:
> > >
> > > > Hi
> > > >
> > > > We are planning to implement caching mechanism for our Hbase data
> model
> > > for
> > > > that we have to remove the *LRU (least recently used)  records* from
> > the
> > > > cached table.
> > > >
> > > > Is there any way by which we can get last accessed time for a record,
> > > > primarily the access will be
> > > > using *Range Scan and Get *
> > > >
> > > > --
> > > > *Regards*
> > > >
> > > > *VIKRAM SINGH CHANDEL*
> > > >
> > > > Please do not print this email unless it is absolutely
> > necessary,Reduce.
> > > > Reuse. Recycle. Save our planet.
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards,
> > > Haosdent Huang
> > >
> >
> >
> >
> > --
> > *Regards*
> >
> > *VIKRAM SINGH CHANDEL*
> >
> > Please do not print this email unless it is absolutely necessary,Reduce.
> > Reuse. Recycle. Save our planet.
> >
> > Confidentiality Notice:  The information contained in this message,
> > including any attachments hereto, may be confidential and is intended to
> be
> > read only by the individual or entity to whom this message is addressed.
> If
> > the reader of this message is not the intended recipient or an agent or
> > designee of the intended recipient, please note that any review, use,
> > disclosure or distribution of this message or its attachments, in any
> form,
> > is strictly prohibited.  If you have received this message in error,
> please
> > immediately notify the sender and/or Notifications@carrieriq.com and
> > delete or destroy any copy of this message and its attachments.
> >
>
>
>
> --
> *Regards*
>
> *VIKRAM SINGH CHANDEL*
>
> Please do not print this email unless it is absolutely necessary,Reduce.
> Reuse. Recycle. Save our planet.
>
> Confidentiality Notice:  The information contained in this message,
> including any attachments hereto, may be confidential and is intended to be
> read only by the individual or entity to whom this message is addressed. If
> the reader of this message is not the intended recipient or an agent or
> designee of the intended recipient, please note that any review, use,
> disclosure or distribution of this message or its attachments, in any form,
> is strictly prohibited.  If you have received this message in error, please
> immediately notify the sender and/or Notifications@carrieriq.com and
> delete or destroy any copy of this message and its attachments.
>



-- 
*Regards*

*VIKRAM SINGH CHANDEL*

Please do not print this email unless it is absolutely necessary,Reduce.
Reuse. Recycle. Save our planet.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message