hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jingcheng Du <jingcheng...@intel.com>
Subject Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
Date Thu, 28 May 2015 05:43:50 GMT
Andrew Purtell wrote
> HBCK can check and sideline dangling reference files. I think of MOB files 
> as "core enough" auxiliary files that need some support. I suppose unlike 
> reference files their presence or absence won't produce a region open 
> failure, we would see dangling pointers later when tying to service 
> queries. (Yes?) Will that abort the RS? Pardon the ignorant question, 
> normally I could check the code but I'm at the airport on a phone. 

Thanks for comments!
In mob, usually the reference cells are committed after the mob files are
done. I think it hardly happens that a reference cell cannot find its mob
Even if there's a dangling reference cell, the RS won't be aborted, a empty
cell is returned instead.

Andrew Purtell wrote
> On that subject, I should file follow up issues for more check and repair 
> options for HFiles. We should be able to detect missing or corrupt files
> of 
> all variety: HFile, reference, MOB. This may require an expensive scan
> over 
> lots of files, but this is like fsck full disk surface scans and those
> have 
> similar costs. Providing MR based tools is fine but we should have 
> multithreaded tools that can stand in if a MR runtime is not available. 
> Import, Export, VerifyReplication...all of these tools are in a different, 
> lesser, class than integrity and repair tools, in my opinion. Since MOB 
> will likely be merged into trunk by then I'll be sure to include it. I 
> agree it's not fair to ask more of MOB then what we have now for HFile.

To detect the corrupt files, some code are needed in the file checker to
check mob files after knowing it's a mob-enabled column.
To detect the missing or dangling reference cells, I think we have to do a
full-table scan, like what is done now in HFile.main.
We can do that.

View this message in context: http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html
Sent from the HBase Developer mailing list archive at Nabble.com.

View raw message