hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmad Humayun" <ahmad.hu...@gmail.com>
Subject Re: hdfsLock
Date Mon, 03 Mar 2008 08:00:26 GMT
Is there any way to append a record at the end of a file....something like
what GFS has?


regards,

On Mon, Mar 3, 2008 at 12:23 PM, dhruba Borthakur <dhruba@yahoo-inc.com>
wrote:

> Hi  Ahmad,
>
> In the current implementation of HDFS, files are "write-once". You
> cannot change/modify the contents of a file once it is written.
>
> Thanks,
> dhruba
>
> -----Original Message-----
> From: Ahmad Humayun [mailto:ahmad.humyn@gmail.com]
> Sent: Saturday, March 01, 2008 1:27 AM
> To: core-dev@hadoop.apache.org
> Subject: Re: hdfsLock
>
> Well this is what I want to do. Lets say I have two nodes A & B each
> working
> with the HDFS. I have a file named, say, temp. In the file temp I just
> store
> a single integer (1 or 0). Now both these nodes access the file temp.
> Everything in concept should work fine as long as both the nodes are
> just
> reading the file, but what if a Node has to change the integer inside
> temp.
> I think the normal procedure will be to read the temp file's integer,
> delete
> the file and write a new file temp with the new required value. Lets say
> Node B wanted to go through this procedure of changing the value inside
> temp. While Node B is "changing/writing" to temp, I don't want Node A to
> be
> able to either read or write to temp. Its something like 'flock()'
> through
> the same fd (file descriptor)
>
> In short I want exclusive access for the Node changing a file. Its the
> Readers Writers problem :)
>
>
> Thanks for the wonderful support on this mailing list :)
>
> regards,
>
>
> On Fri, Feb 29, 2008 at 12:47 AM, Raghu Angadi <rangadi@yahoo-inc.com>
> wrote:
>
> >
> > What do you want to use these locks for? From your question, you seem
> to
> > be concerned about multiple threads accessing a stream. FileLocks have
> > nothing to do with that. They were meant to be something like
> 'flock()'
> > system call.
> >
> > Raghu.
> >
> > Ahmad Humayun wrote:
> > > Do you any reason why? Is it because only one thread can write to a
> > specific
> > > file in the hdfs at a single time....and multiple simultaneous
> readers
> > > doesn't cause any problem...hence locks make no sense in this
> > situation???
> > >
> > > thanks for the help.
> > >
> > >
> > > regards,
> > >
> > > On Thu, Feb 28, 2008 at 10:28 PM, Raghu Angadi
> <rangadi@yahoo-inc.com>
> > > wrote:
> > >
> > >> File locking is not supported in HDFS. Not sure if it ever was
> > supported
> > >> properly. This interface was deprecated last year.
> > >>
> > >> Raghu.
> > >>
> > >> Ahmad Humayun wrote:
> > >>> Hello everyone,
> > >>>
> > >>> Does anybody have an idea why hdfsLock and hdfsReleaseLock been
> taken
> > >> out of
> > >>> libhdfs? How do I lock a file now using libhdfs? Can somebody
> point me
> > >> to
> > >>> the changelog or smth?
> > >>>
> > >>>
> > >>> regards,
> > >>
> > >
> > >
> >
> >
>
>
> --
> Ahmad Humayun
> Research Assistant
> Computer Science Dpt., LUMS
> +92 321 4457315
>



-- 
Ahmad Humayun
Research Assistant
Computer Science Dpt., LUMS
+92 321 4457315

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message