hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Luo <j...@merkleinc.com>
Subject RE: Major compaction cannot remove deleted rows until the region is split. Strange!
Date Fri, 27 May 2016 21:54:54 GMT
What if you manually trigger major-compact on that particular region? Does it run and the delete
markers removed?

-----Original Message-----
From: Tianying Chang [mailto:tychang@gmail.com]
Sent: Friday, May 27, 2016 4:33 PM
To: user@hbase.apache.org
Subject: Major compaction cannot remove deleted rows until the region is split. Strange!


We saw a very strange case in one of our production cluster. A couple regions cannot get their
deleted rows or delete marker removed even after major compaction. However when the region
triggered split (we set 100G for auto split), the deletion worked. The 100G region becomes
two 10G daughter regions, and all the delete marker are gone.

Also, the same region in the slave cluster (through replication) have normal size at about

BTW, the delete marker in the regions are mostly deleteFamily if it matters.

This is really weird. Anyone has any clue for this strange behavior?


A snippet of the HFile generated by the major compaction:

: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459808114380/DeleteFamily/vlen=0/ts=2292870047
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459808114011/DeleteFamily/vlen=0/ts=2292869794
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459805381104/DeleteFamily/vlen=0/ts=2291072240
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459805380673/DeleteFamily/vlen=0/ts=2291071997
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459802643449/DeleteFamily/vlen=0/ts=2290248886
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459802643246/DeleteFamily/vlen=0/ts=2290248786
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459799913003/DeleteFamily/vlen=0/ts=2289446916
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459797181831/DeleteFamily/vlen=0/ts=2288670451
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459794447388/DeleteFamily/vlen=0/ts=2287911443
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459791708803/DeleteFamily/vlen=0/ts=2287213792
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459788978387/DeleteFamily/vlen=0/ts=2286488738
K: \xA0\x00\x00L\x1A@\x1CBe\x00\x00\x08m\x03\x1A@\x10\x00?PF/d:/1459786243642/DeleteFamily/vlen=0/ts=2285778927
Join us at Merkle’s 2016 annual Performance Marketer Executive Summit – June 7 – 9 in
Memphis, TN

Download the latest installment of our annual Marketing Imperatives, “Winning with People-Based
This email and any attachments transmitted with it are intended for use by the intended recipient(s)
only. If you have received this email in error, please notify the sender immediately and then
delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or
distribute this email without the author’s prior permission. We take precautions to minimize
the risk of transmitting software viruses, but we advise you to perform your own virus checks
on any attachment to this message. We cannot accept liability for any loss or damage caused
by software viruses. The information contained in this communication may be confidential and
may be subject to the attorney-client privilege.
View raw message