Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.213.54 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <A0D479D9-4EC9-4C21-88EF-F6D370891E7F@digitalenvoy.net>
References: <0856057E-7A16-4A3D-883D-5995AE74419E@digitalenvoy.net>
	<CALte62yEeAi2TbkY-teJfyAxo5hbYn8CX4KnCAG5cztNmDp4ag@mail.gmail.com>
	<8BDC1CCA-3DD7-47D6-95D2-108E5A173D8D@digitalenvoy.net>
	<C8CA2024-6B21-4005-9E1F-DE7960CCEAD9@gmail.com>
	<A0D479D9-4EC9-4C21-88EF-F6D370891E7F@digitalenvoy.net>
Date: Sun, 7 Sep 2014 07:46:04 -0700
Message-ID: 
 <CALte62wbH9FwSCGzpoK9hxYwLbgq4gx-_ndmC_J4JsPmtS1eFQ@mail.gmail.com>
Subject: Re: directory usage question
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=089e0163544c05439305027ac3c9

--089e0163544c05439305027ac3c9
Content-Type: text/plain; charset=UTF-8

Eclipse doesn't show that RestoreSnapshotHelper.restoreHdfsRegions() is
called by initTableSnapshotMapperJob (in master branch)

Looking at TableMapReduceUtil.java in 0.98, I don't see direct relation
between the two.

Do you have stack trace or something else showing the relationship ?

Cheers


On Sun, Sep 7, 2014 at 5:48 AM, Brian Jeltema <
brian.jeltema@digitalenvoy.net> wrote:

> initTableSnapshotMapperJob writes into this directory (indirectly) via
> RestoreSnapshotHelper.restoreHdfsRegions
>
> Is this expected? I would have expected writes to be limited to the temp
> directory passed in the init call
>
> Brian
>
> On Sep 7, 2014, at 8:17 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > The files under archive directory are referenced by snapshots.
> > Please don't delete them manually.
> >
> > You can delete unused snapshots.
> >
> > Cheers
> >
> > On Sep 7, 2014, at 4:08 AM, Brian Jeltema <
> brian.jeltema@digitalenvoy.net> wrote:
> >
> >>
> >> On Sep 6, 2014, at 9:32 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>
> >>> Can you post your hbase-site.xml ?
> >>>
> >>> /apps/hbase/data/archive/data/default is where HFiles are archived
> (e.g.
> >>> when a column family is deleted, HFiles for this column family are
> stored
> >>> here).
> >>> /apps/hbase/data/data/default seems to be your hbase.rootdir
> >>
> >> hbase.rootdir is defined to be hdfs://foo:8020/apps/hbase/data. I think
> that's the default that Ambari creates.
> >>
> >> So the HFiles in the archive subdirectory have been discarded and can
> be deleted safely?
> >>
> >>> bq. a problem I'm having running map/reduce jobs against snapshots
> >>>
> >>> Can you describe the problem in a bit more detail ?
> >>
> >> I don't understand what I'm seeing well enough to ask an intelligent
> question yet.
> >> I appear to be scanning duplicate rows when using
> initTableSnapshotMapperJob,
> >> but I'm trying to get a better understanding of how this works, since
> It's probably just
> >> something I'm doing wrong.
> >>
> >> Brian
> >>
> >>> Cheers
> >>>
> >>>
> >>> On Sat, Sep 6, 2014 at 6:09 AM, Brian Jeltema <
> >>> brian.jeltema@digitalenvoy.net> wrote:
> >>>
> >>>> I'm trying to track down a problem I'm having running map/reduce jobs
> >>>> against snapshots.
> >>>> Can someone explain the difference between files stored in:
> >>>>
> >>>>  /apps/hbase/data/archive/data/default
> >>>>
> >>>> and files stored in
> >>>>
> >>>>  /apps/hbase/data/data/default
> >>>>
> >>>> (Hadoop 2.4, HBase 0.98)
> >>>>
> >>>> Thanks
> >>
> >
>
>

--089e0163544c05439305027ac3c9--