hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
Date Wed, 08 Jul 2015 03:37:04 GMT
Looks like all MOB-related JIRAs have been resolved.

Should the voting process be resumed ?

Cheers

On Thu, May 28, 2015 at 5:48 PM, Anoop John <anoop.hbase@gmail.com> wrote:

> Yes Andy. The sweep tool is completely optional now. We have a chore doing
> the compaction, like we trigger auto major compaction.  We can configure
> the interval. Auto can be turned off and user can explicitly call also. We
> have shell and API support.
>
> Anoop
>
> On Friday, May 29, 2015, Andrew Purtell <apurtell@apache.org> wrote:
> > MOB references in cells won't find their value if the MOB hfile has been
> > corrupted. Dealing with that would be like any other corrupted HFile,
> > understood. The dangling references make thinking about (partial)
> recovery
> > and repair interesting.
> >
> >
> > On Thursday, May 28, 2015, Jingcheng Du <jingcheng.du@intel.com> wrote:
> >
> >> Andrew Purtell wrote
> >> > HBCK can check and sideline dangling reference files. I think of MOB
> >> files
> >> > as "core enough" auxiliary files that need some support. I suppose
> unlike
> >> > reference files their presence or absence won't produce a region open
> >> > failure, we would see dangling pointers later when tying to service
> >> > queries. (Yes?) Will that abort the RS? Pardon the ignorant question,
> >> > normally I could check the code but I'm at the airport on a phone.
> >>
> >> Thanks for comments!
> >> In mob, usually the reference cells are committed after the mob files
> are
> >> done. I think it hardly happens that a reference cell cannot find its
> mob
> >> file.
> >> Even if there's a dangling reference cell, the RS won't be aborted, a
> empty
> >> cell is returned instead.
> >>
> >>
> >> Andrew Purtell wrote
> >> > On that subject, I should file follow up issues for more check and
> repair
> >> > options for HFiles. We should be able to detect missing or corrupt
> files
> >> > of
> >> > all variety: HFile, reference, MOB. This may require an expensive scan
> >> > over
> >> > lots of files, but this is like fsck full disk surface scans and those
> >> > have
> >> > similar costs. Providing MR based tools is fine but we should have
> >> > multithreaded tools that can stand in if a MR runtime is not
> available.
> >> > Import, Export, VerifyReplication...all of these tools are in a
> >> different,
> >> > lesser, class than integrity and repair tools, in my opinion. Since
> MOB
> >> > will likely be merged into trunk by then I'll be sure to include it. I
> >> > agree it's not fair to ask more of MOB then what we have now for
> HFile.
> >>
> >> To detect the corrupt files, some code are needed in the file checker to
> >> check mob files after knowing it's a mob-enabled column.
> >> To detect the missing or dangling reference cells, I think we have to do
> a
> >> full-table scan, like what is done now in HFile.main.
> >> We can do that.
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
>
> http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html
> >> Sent from the HBase Developer mailing list archive at Nabble.com.
> >>
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message