hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: [DISCUSSION] Merge of the hbase-11339 mob branch into master.
Date Wed, 08 Jul 2015 04:16:13 GMT
+1 to start the voting again.

On Wed, Jul 8, 2015 at 9:07 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Looks like all MOB-related JIRAs have been resolved.
>
> Should the voting process be resumed ?
>
> Cheers
>
> On Thu, May 28, 2015 at 5:48 PM, Anoop John <anoop.hbase@gmail.com> wrote:
>
> > Yes Andy. The sweep tool is completely optional now. We have a chore
> doing
> > the compaction, like we trigger auto major compaction.  We can configure
> > the interval. Auto can be turned off and user can explicitly call also.
> We
> > have shell and API support.
> >
> > Anoop
> >
> > On Friday, May 29, 2015, Andrew Purtell <apurtell@apache.org> wrote:
> > > MOB references in cells won't find their value if the MOB hfile has
> been
> > > corrupted. Dealing with that would be like any other corrupted HFile,
> > > understood. The dangling references make thinking about (partial)
> > recovery
> > > and repair interesting.
> > >
> > >
> > > On Thursday, May 28, 2015, Jingcheng Du <jingcheng.du@intel.com>
> wrote:
> > >
> > >> Andrew Purtell wrote
> > >> > HBCK can check and sideline dangling reference files. I think of MOB
> > >> files
> > >> > as "core enough" auxiliary files that need some support. I suppose
> > unlike
> > >> > reference files their presence or absence won't produce a region
> open
> > >> > failure, we would see dangling pointers later when tying to service
> > >> > queries. (Yes?) Will that abort the RS? Pardon the ignorant
> question,
> > >> > normally I could check the code but I'm at the airport on a phone.
> > >>
> > >> Thanks for comments!
> > >> In mob, usually the reference cells are committed after the mob files
> > are
> > >> done. I think it hardly happens that a reference cell cannot find its
> > mob
> > >> file.
> > >> Even if there's a dangling reference cell, the RS won't be aborted, a
> > empty
> > >> cell is returned instead.
> > >>
> > >>
> > >> Andrew Purtell wrote
> > >> > On that subject, I should file follow up issues for more check and
> > repair
> > >> > options for HFiles. We should be able to detect missing or corrupt
> > files
> > >> > of
> > >> > all variety: HFile, reference, MOB. This may require an expensive
> scan
> > >> > over
> > >> > lots of files, but this is like fsck full disk surface scans and
> those
> > >> > have
> > >> > similar costs. Providing MR based tools is fine but we should have
> > >> > multithreaded tools that can stand in if a MR runtime is not
> > available.
> > >> > Import, Export, VerifyReplication...all of these tools are in a
> > >> different,
> > >> > lesser, class than integrity and repair tools, in my opinion. Since
> > MOB
> > >> > will likely be merged into trunk by then I'll be sure to include
> it. I
> > >> > agree it's not fair to ask more of MOB then what we have now for
> > HFile.
> > >>
> > >> To detect the corrupt files, some code are needed in the file checker
> to
> > >> check mob files after knowing it's a mob-enabled column.
> > >> To detect the missing or dangling reference cells, I think we have to
> do
> > a
> > >> full-table scan, like what is done now in HFile.main.
> > >> We can do that.
> > >>
> > >>
> > >>
> > >> --
> > >> View this message in context:
> > >>
> >
> >
> http://apache-hbase.679495.n3.nabble.com/DISCUSSION-Merge-of-the-hbase-11339-mob-branch-into-master-tp4071644p4071911.html
> > >> Sent from the HBase Developer mailing list archive at Nabble.com.
> > >>
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message