hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin O'dell" <kevin.od...@cloudera.com>
Subject Re: delete rows without writing HLog may be appear in the future?
Date Wed, 21 Nov 2012 15:43:14 GMT
I would recommend delete with HLog put, but lets say your writes are
minimal you should only have 32 hours(default) of WAL around at a time
before the they are all flushed from too many HLogs.  So you should not
have week old deletes coming back.  One thought would be to raise your WAL
size but lower the total number WALs kept.  This will retain the same
amount of data but should flush fully quicker.

Can someone thought check the above logic?

On Wed, Nov 21, 2012 at 10:37 AM, Bing Jiang <jiangbinglover@gmail.com>wrote:

> In our apps, deletes will be frequent, and it occurs to each records every
> time, if write hlog, the performance and response will be low. In fact,we
> can bear with some records with delete fail, but recently I have found more
> records delete some time ago, for example, one week , they reappear
> again.Then, that makes me curious about what should do next., delete with
> writing hlog, or put without hlog....
> On Nov 21, 2012 11:19 PM, "Kevin O'dell" <kevin.odell@cloudera.com> wrote:
>
> > Bing,
> >
> >   I am curious to hear more about Mike's question.  Why are you not using
> > the WAL for your deletes?
> >
> > On Wed, Nov 21, 2012 at 10:17 AM, Bing Jiang <jiangbinglover@gmail.com
> > >wrote:
> >
> > > yes,hbase has made a compaction between batch-put and deletes. any
> ideas?
> > >
> > > On Nov 21, 2012 11:10 PM, "Michael Segel" <michael_segel@hotmail.com>
> > > wrote:
> > > >
> > > > Some time later?
> > > >
> > > > Time of course is relative, so I have to ask what occurred between
> the
> > > write and the delete?
> > > > How much time? Did you have any compactions in between the write and
> > the
> > > delete?
> > > >
> > > > Why are you not consistent in your use of the WAL ?
> > > >
> > > >
> > > > On Nov 21, 2012, at 6:37 AM, Bing Jiang <jiangbinglover@gmail.com>
> > > wrote:
> > > >
> > > > > hiļ¼Œall.
> > > > > I want to describe a phenomenon that happens to our hbase cluster.
> > > > > I use puts(List<Put>) to insert many records with writing hlog
> > enable,
> > > > > and some time later I delete all of these records with writing hlog
> > > disable.
> > > > > When one week later, i scan the table, I found some records I have
> > > delete
> > > > > reappear again.
> > > > > It is an interesting case. In my opinion, if we delete data without
> > > enable
> > > > > writing hlog, when regionserver fails, the log will replay in
> another
> > > > > regionserver.
> > > > > Can anyone tell me if I persist on deleting records without enable
> > > writing
> > > > > hlog, is there a way to prevent these records from reappearing
> again
> > > some
> > > > > time later?
> > > > >
> > > > > Cheers!
> > > > > --
> > > > > Bing Jiang
> > > > > weibo: http://weibo.com/jiangbinglover
> > > > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > > > BLOG: http://www.binospace.com
> > > > > National Research Center for Intelligent Computing Systems
> > > > > Institute of Computing technology
> > > > > Graduate University of Chinese Academy of Science
> > > >
> > >
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
> >
>



-- 
Kevin O'Dell
Customer Operations Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message