hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianying Chang <tych...@gmail.com>
Subject Re: Major compaction cannot remove deleted rows until the region is split. Strange!
Date Fri, 27 May 2016 22:17:51 GMT
Yes, it is 94.26.  By a quick glance, I didn't  see any put that is older
than the delete marker's TS, which could go as far as about couple weeks
ago since major compaction on it for long time seems.

Also it is really strange that if the region is split, then seems
everything is working as expected. Also we noticed, the same region
replicated at the slave side is totally normal, i.e. at 20+G....

On Fri, May 27, 2016 at 3:13 PM, Stack <stack@duboce.net> wrote:

> On Fri, May 27, 2016 at 2:32 PM, Tianying Chang <tychang@gmail.com> wrote:
>
> > Hi,
> >
> > We saw a very strange case in one of our production cluster. A couple
> > regions cannot get their deleted rows or delete marker removed even after
> > major compaction. However when the region triggered split (we set 100G
> for
> > auto split), the deletion worked. The 100G region becomes two 10G
> daughter
> > regions, and all the delete marker are gone.
> >
> > Also, the same region in the slave cluster (through replication) have
> > normal size at about 20+G.
> >
> > BTW, the delete marker in the regions are mostly deleteFamily if it
> > matters.
> >
> > This is really weird. Anyone has any clue for this strange behavior?
> >
> > Thanks
> > Tian-Ying
> >
> > These 0.94 Tian-Ying?
>
> It looks like the DeleteFamily is retained only; do you see incidence where
> there may have been versions older than the DeleteFamily that are also
> retained post-major-compaction?
>
> St.Ack
>
>
>
> > A snippet of the HFile generated by the major compaction:
> >
> > : \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459808114380/DeleteFamily/vlen=0/ts=2292870047
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459808114011/DeleteFamily/vlen=0/ts=2292869794
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459805381104/DeleteFamily/vlen=0/ts=2291072240
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459805380673/DeleteFamily/vlen=0/ts=2291071997
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459802643449/DeleteFamily/vlen=0/ts=2290248886
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459802643246/DeleteFamily/vlen=0/ts=2290248786
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459799913003/DeleteFamily/vlen=0/ts=2289446916
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459797181831/DeleteFamily/vlen=0/ts=2288670451
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459794447388/DeleteFamily/vlen=0/ts=2287911443
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459791708803/DeleteFamily/vlen=0/ts=2287213792
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459788978387/DeleteFamily/vlen=0/ts=2286488738
> > V:
> > K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@
> > \x10\x00?PF/d:/1459786243642/DeleteFamily/vlen=0/ts=2285778927
> > V:
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message