hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
Date Thu, 18 Apr 2013 06:16:16 GMT
Thanks Enis and Andrew; I think I missed the key point of conformance
with other FSes' behavior.

On Wed, Apr 17, 2013 at 11:29 PM, Enis Söztutar <enis.soz@gmail.com> wrote:
> Harsh, the discussion above includes the reasoning behind forcing the
> change in hbase rather than hdfs. Although HBase has shipped with this,
> HDFS's snapshots are user visible, meaning that you can do:
>
> hadoop fs -ls /user/foo/.snapshot/
>
> Plus, it is a convention for file systems (netapp, etc) to expose snapshots
> this way, having a name ".snapshot". HBase's snapshot directories are not
> user visible, and not widely used yet.
>
> Enis
>
>
> On Wed, Apr 17, 2013 at 9:14 AM, Andrew Purtell <apurtell@apache.org> wrote:
>
>> Thanks for the consideration but we've just committed a change to address
>> this as HBASE-8352
>>
>>
>> On Wednesday, April 17, 2013, Harsh J wrote:
>>
>> > Pardon my late inquisition here but since HBase already shipped out
>> > with a name .snapshots/, why do we force them to change it, and not
>> > rename HDFS' snapshots to use .hdfs-snapshots, given that HDFS
>> > Snapshots has not been released for any users yet. The way I see it,
>> > that'd be much more easier to do than making a workaround for a done
>> > deal on HBase, which already has its snapshot's users.
>> >
>> > @Tsz-Wo - If the snapshots in HDFS aren't a 'generic' feature
>> > applicable to other FileSystem interface implementations as well, then
>> > .hdfs-snapshots should be fine for it - no?
>> >
>> > On Wed, Apr 17, 2013 at 2:32 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > > Hi,
>> > > Please take a look at patch v5 attached to HBASE-8352.
>> > >
>> > > It would be nice to resolve this blocker today so that 0.94.7 RC can be
>> > cut.
>> > >
>> > > Thanks
>> > >
>> > > On Tue, Apr 16, 2013 at 10:12 AM, lars hofhansl <larsh@apache.org>
>> > wrote:
>> > >
>> > >> Please see my last comment on the jira. We can make this work without
>> > >> breaking users who are using HDFS snapshots.
>> > >>
>> > >>   ------------------------------
>> > >>  *From:* Ted Yu <yuzhihong@gmail.com>
>> > >> *To:* dev@hbase.apache.org
>> > >> *Cc:* hdfs-dev@hadoop.apache.org; lars hofhansl <larsh@apache.org>
>> > >> *Sent:* Tuesday, April 16, 2013 10:00 AM
>> > >> *Subject:* Re: collision in the naming of '.snapshot' directory
>> between
>> > >> hdfs snapshot and hbase snapshot
>> > >>
>> > >> Let's get proper release notes for HBASE-8352 .
>> > >>
>> > >> Either Lars or I can send out notification to user mailing list so
>> that
>> > >> there is enough preparation for this change.
>> > >>
>> > >> Cheers
>> > >>
>> > >> On Tue, Apr 16, 2013 at 8:46 AM, Jonathan Hsieh <jon@cloudera.com>
>> > wrote:
>> > >>
>> > >> I was away from keyboard when I asserted that hdfs snapshot was a
>> hadoop
>> > >> 2.1 or 3.0 feature.  Apparently it is targeted as a hadoop 2.0.5
>> > feature.
>> > >>  (I'm a little surprised -- expected this to be a hadoop2 compat
>> > breaking
>> > >> feature) -- so I agree that this is a bit more urgent.
>> > >>
>> > >> Anyway, I agree that the fs .snapshot naming convention is long
>> standing
>> > >> and should win.
>> > >>
>> > >> My concern is with breaking compatibility in 0.94 again -- if we don't
>> > go
>> > >> down the conf variable route,  I consider having docs to properly
>> > document
>> > >> how to do the upgrade and caveats of doing the upgrade in the
>> > docs/release
>> > >> notes blocker to hbase 0.94.7.  (specifically mentioning from 0.94.6
>> to
>> > >> 0.94.7, and to possibly to 0.95).
>> > >>
>> > >> Jon.
>> > >>
>> > >> On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
>> > >>
>> > >> > bq. Alternatively, we can detect the underlying Hadoop version,
and
>> > use
>> > >> > either .snapshot or .hbase_snapshot in 0.94 depending on h1 &
h2.
>> > >> >
>> > >> > I think this would introduce more confusion, especially for
>> > operations.
>> > >> >
>> > >> > Cheers
>> > >> >
>> > >> > On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <enis.soz@gmail.com>
>> > >> wrote:
>> > >> >
>> > >> > > Because HDFS exposes the snapshots so that the normal file
system
>> > >> > > operations are mapped inside snapshot dirs, I think HDFS
reserving
>> > the
>> > >> > > .snapshot name makes sense. OTOH, nothing is specific about
the
>> dir
>> > >> name
>> > >> > > that is chosen by HBase.
>> > >> > >
>> > >> > > I would prefer to change the dir name in 0.94 as well, since
0.94
>> is
>> > >> also
>> > >> > > being run on top of hadoop 2. Alternatively, we can detect
the
>> > >> underlying
>> > >> > > Hadoop version, and use either .snapshot or .hbase_snapshot
in
>> 0.94
>> > >> > > depending on h1 & h2.
>> > >> > >
>> > >> > > Enis
>> > >> > >
>> > >> > >
>> > >> > > On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <yuzhihong@gmail.com>
>> > wrote:
>> > >> > >
>> > >> > > > bq. let's make the hbase snapshot for a conf variable.
>> > >> > > >
>> > >> > > > Once we decide on the new name of snapshot directory,
we should
>> > still
>> > >> > use
>> > >> > > > hardcoded value. This aligns with current code base:
>> > >> > > > See this snippet from HConstants:
>> > >--
>> > Harsh J
>> >
>>
>>
>> --
>> Best regards,
>>
>>    - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>



-- 
Harsh J

Mime
View raw message