hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jingcheng Du <jingcheng...@intel.com>
Subject Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
Date Thu, 28 May 2015 06:24:06 GMT
Andrew Purtell wrote
> What coverage do we have for verifying the integrity of MOB references? 
> Will the sweep tool detect, alert on, and optionally repair dangling 
> references? (I could answer this for myself by looking at MOB branch, but 
> hopefully someone here has an answer at the ready.) I assume we calculate 
> and store checksums for MOB data itself so we know if values are corrupt. 
> Does the sweep tool detect MOB value corruption? Can it be repaired? Do we 
> have a good ops story for why HBCK is no longer sufficient on its own, 
> there's a separate tool with a whole new set of options - and a
> requirement 
> for a MR runtime! - for checking MOB data? That last one is a rhetorical 
> question (smile), the ops story is... unsatisfying. It's like we've taken
> a 
> self sufficient HBase and bolted in parts of Hive, so now we need MR. 

Now we have two mechanism to compact mob file (Merge the small files and
drop the deleted cells), one is the sweep tool (with MR), the other is
MobCompactor (without MR).
Both of them do not drop the dangling reference cells (if we have although I
think it hardly happens).
I think the code to detect the missing and corrupt can be added in HBCK and
HFile.main.



--
View this message in context: http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071912.html
Sent from the HBase Developer mailing list archive at Nabble.com.

Mime
View raw message