hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: HFileLink backreferences
Date Wed, 23 Jul 2014 22:03:08 GMT
It might be possible to do it so that hfilelink does not create the
backreferences in this case only read access is required.

Exporting the snapshot first into a different dir, then restoring and
running the snapshots on top is also another solution.


On Wed, Jul 23, 2014 at 12:44 PM, D vd Reddy <dvdeepankar.reddy@gmail.com>
wrote:

> Hi,
>
> This was the same issue, we were seeing when we wanted to use this. And
> also this was also difficult when we wanted to use this for HBase - Hive
> integration over snapshot (see HBASE-11484). I think what Matteo Bertozzi
> <https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mbertozzi>
> suggested
> in the the jira HBASE-11484 can be a nice solution for this, where we
> provide a way to run the MR without restoring the snapshot
>
>
> On Wed, Jul 23, 2014 at 11:56 AM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > Heya,
> >
> > I see that we maintain backreferences for hfilelinks. This appears to be
> > used by HFileLinkCleaner to determine when an HFile has no HFileLinks and
> > thus whether it can be deleted without orphaning those links.
> >
> > This is problematic for using the mapreduce over snapshot files feature.
> > However, restoring a snapshot can create HFileLinks to existing files in
> > the restore directory. Those links then create back-references to the
> root
> > path. Thus we have a situation where the user running the MR job requires
> > write access to the hbase root path.
> >
> > Was this already discussed in the original ticket (HBASE-8369)? We
> mention
> > the requirement of read permissions (and bypassing security) in the
> release
> > note, but I didn't note any comments for write. Requiring write
> permission
> > effectively means you can only MR as the hbase user, which is pretty
> much a
> > non-starter for any interesting integration of feature.
> >
> > Thoughts?
> >
> > Thanks,
> > Nick
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message