hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmad Humayun" <ahmad.hu...@gmail.com>
Subject Re: deleting files from the DFS
Date Fri, 14 Mar 2008 04:04:44 GMT
Yes!! So is there any locking mechanisms? Plus what dictates this delay in
file deletion and actual deletion of the file blocks over the cluster?


regards,

On Fri, Mar 14, 2008 at 6:22 AM, edward yoon <edward@udanax.org> wrote:

> Can zookeeper locking services be used for these cases?
>
> Thanks,
> Edward.
>
> On 3/13/08, Sanjay Radia <sradia@yahoo-inc.com> wrote:
> > Ahmad Humayun wrote:
> > > So does that mean nodes can possibly read files that have been
> "deleted"
> > >
> > If the name node entry has been deleted, new opens will fail.
> > Already opened files may have their reads succeed if they have cached
> > the block locations and also if the
> > blocks have not been deleted.
> >
> > Hence, if you open a large file and someone deletes the file after you
> > open it, at some point the reads may fail if
> > the client needs to go to the namenode to get block locations.
> > This not posix semantics but keeping the namenode stateless has
> > advantages in scaling.
> > > thanks :)
> > >
> > >
> > > regards,
> > >
> > > On Wed, Mar 12, 2008 at 10:57 PM, Hairong Kuang <hairong@yahoo-inc.com
> >
> > > wrote:
> > >
> > >
> > >>> Case 1: Lets say there is a node which opens a file on the DFS to
> read.
> > >>>
> > >> From
> > >>
> > >>> my understanding, the open request will go to the namenode, which
> will
> > >>>
> > >> tell
> > >>
> > >>> it where the blocks of the file are. What if another node comes up
> and
> > >>>
> > >> asks
> > >>
> > >>> to delete that file. Of course this request will go to the namenode.
> How
> > >>> will the namenode respond in such a situation?
> > >>>
> > >>>
> > >> The namenode is stateless. So the delete request will go through. The
> > >> first
> > >> request may fail.
> > >>
> > >>
> > >>> Case 2: Whenever a node wants to delete a file, I think the request
> > >>>
> > >> first
> > >>
> > >>> lands with the namenode? What happens to the replicas of the blocks
> of
> > >>>
> > >> that
> > >>
> > >>> file, as soon as that file gets deleted?
> > >>>
> > >>>
> > >> The replicas are scheduled to be deleted by the namenode. But there
> may be
> > >> some delay before they are actually deleted on the datanodes.
> > >>
> > >> Hairong
> > >>
> > >>
> > >>
> > >
> > >
> > >
> >
> >
>
>
> --
> B. Regards,
> Edward yoon @ NHN, corp.
>



-- 
Ahmad Humayun
Research Assistant
Computer Science Dpt., LUMS
+92 321 4457315

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message