Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CA+qbEUNfZ6m7=AydheB62v7m+kXHwYJaiBQC4Fyb9GMTJLee1A@mail.gmail.com>
References: 
 <CAO_q5xopCgOP8eZmy9DSVr37ZrgYcsjs8SYaf_OxYBD-OkWdWg@mail.gmail.com>
	<CA+qbEUNfZ6m7=AydheB62v7m+kXHwYJaiBQC4Fyb9GMTJLee1A@mail.gmail.com>
Date: Mon, 15 Jun 2015 13:30:13 -0700
Message-ID: 
 <CA+01ahh9gUwtVTZnPorL15+ydv=1FfPAq8kbfAJrHnuGc4wckg@mail.gmail.com>
Subject: Re: How to Tag a request using optional field
From: lohit <lohit.vijayarenu@gmail.com>
To: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a11c2c5882a5771051894534b

--001a11c2c5882a5771051894534b
Content-Type: text/plain; charset=UTF-8

It sounds like you would need way to store UUID somewhere so that on
INotify about file you can retrieve it. How about using HDFS extended
attributes to save UUID for inode and on notification, fetch extended
attributes?

2015-06-15 13:11 GMT-07:00 Colin P. McCabe <cmccabe@apache.org>:

> On Thu, Jun 4, 2015 at 2:46 PM, Rahul Shrivastava <rhshriva@gmail.com>
> wrote:
> > Hi,
> >
> >
> > Suppose I write a Java client to create a directory on HDFS. Is there a
> way
> > to tag this request and get the tagged information on NameNode via
> > DFSInotifyEventInputStream or otherwise ?
> >
> > In short, is there a way to give optional information attached to every
> > request ?
> >
> > I do not want to save this information in File System but just to use
> this
> > information via DFSInotifyEventInputStream to relate it back to the
> client.
> >
> > The requirement is attach a UUID with every request on the client and
> > process this UUID from NameNode.
> >
> > thanks
> > Rahul
>
> Hi Rahul,
>
> By definition, inotify reports on things which are "saved" i.e.
> persisted somehow on the NameNode.  If the information were not
> persisted, we'd run into trouble using inotify in scenarios like
> shutting down the NameNode and bringing it back up, or failing over
> between namenodes.
>
> Requests do have UUIDs in the form of RPC call IDs.  It would be
> rather difficult to expose this information through the Java API,
> though.  I'm not sure how we would allow the calling code for mkdirs
> (for example) to get the call ID of the RPC it made.  Certainly we
> don't want to change the return signature of mkdirs, and using
> thread-local data seems kind of ugly.  I'm also not sure if we want to
> make the call ID part of the inotify API since it's an implementation
> detail.
>
> We had a discussion at Hadoop Summit about ways to use inotify
> effectively.  One thing that came up is that we should add support for
> determining that an HDFS snapshot had occurred to inotify.  This would
> allow users to do things like:
>
> 1. start listening for events
> 2. take a snapshot named Foo_2015-06-15_13:09
> 3. get a SnapshotCreateEvent for Foo_2015-06-15_13:09
> 4. do a full scan on the snapshot and buffer that information somewhere
> 5. ask for all events that happened since the SnapshotCreateEvent and
> apply them to the buffered information
>
> At the end of #5, you have a fully consistent view of the filesystem.
> I'm not sure if this is your exact usecase, but perhaps it's helpful.
>
> best,
> Colin
>


-- 
Have a Nice Day!
Lohit

--001a11c2c5882a5771051894534b--