hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpit Agarwal <aagar...@hortonworks.com>
Subject Re: Is FileSystem thread-safe?
Date Mon, 01 Apr 2013 22:17:39 GMT
Hi John,

DistributedFileSystem is intended to be thread-safe, true to its name.

Metadata operations are handled by the NameNode server which synchronizes
concurrent client requests via locks (you can look at the FSNameSystem

Some discussion on the thread-safety aspects of HDFS:


On Sun, Mar 31, 2013 at 11:52 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> If you look at DistributedFileSystem source code, you would see that it
> calls the DFSClient field member for most of the actions.
> Requests to Namenode are then made through ClientProtocol.
> An hdfs committer would be able to give you affirmative answer.
> On Sun, Mar 31, 2013 at 11:27 AM, John Lilley <john.lilley@redpoint.net>wrote:
>>  *From:* Ted Yu [mailto:yuzhihong@gmail.com]
>> *Subject:* Re: Is FileSystem thread-safe?****
>> >>FileSystem is an abstract class, what concrete class are you using
>> (DistributedFileSystem, etc) ? ****
>> Good point.  I am calling FileSystem.get(URI uri, Configuration conf)
>> with an URI like “hdfs://server:port/…” on a remote server, so I assume it
>> is creating a DistributedFileSystem.  However I am not finding any
>> documentation discussing its thread-safety (or lack thereof), perhaps you
>> can point me to it?****
>> Thanks, john****

View raw message