hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Goel <deic...@gmail.com>
Subject Re: Hadoop/HDFS/POSIX
Date Sat, 04 Jun 2016 05:26:34 GMT
Thank You. Would you suggest profiling the application (if it is ethical)
and finding out what are the differences? Or does it seem a wasteful
exercise?

Hey

Namaskara~Nalama~Guten Tag~Bonjour


   --
Keigu

Deepak
73500 12833
www.simtree.net, deepak@simtree.net
deicool@gmail.com

LinkedIn: www.linkedin.com/in/deicool
Skype: thumsupdeicool
Google talk: deicool
Blog: http://loveandfearless.wordpress.com
Facebook: http://www.facebook.com/deicool

"Contribute to the world, environment and more : http://www.gridrepublic.org
"

On Sat, Jun 4, 2016 at 1:32 AM, Chris Nauroth <cnauroth@hortonworks.com>
wrote:

> Unfortunately, I'm not aware of any comprehensive documentation on how
> HDFS adheres to POSIX and how it differs.  To some degree, it's possible to
> look at the FileSystem specification and corresponding contract tests as a
> source of information, but it's not structured to answer that question
> directly.  I filed HADOOP-13104 to track improving that, but I haven't been
> able to work on it yet.
>
>
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/filesystem/index.html
>
> https://issues.apache.org/jira/browse/HADOOP-13104
>
> There is also some discussion of POSIX specifically related to permission
> handling in the HDFS permissions guide.
>
>
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html
>
> --Chris Nauroth
>
> From: Gagan Brahmi <gaganbrahmi@gmail.com>
> Date: Friday, June 3, 2016 at 9:18 AM
> To: Deepak Goel <deicool@gmail.com>
> Cc: user <user@hadoop.apache.org>
> Subject: Re: Hadoop/HDFS/POSIX
>
> AFAIK the major difference include the simplified Coherency model
> <https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#Simple_Coherency_Model>
> in HDFS.
>
> The other one is the file permissions. The read (r), write (w) & execute
> (x) works the same way for directories in HDFS like a POSIX file system.
> But for the  files, execute permission is ignored in HDFS since you can't
> really execute a file in HDFS.
>
> HTH
>
>
> Regards,
> Gagan Brahmi
>
> On Fri, Jun 3, 2016 at 1:30 AM, Deepak Goel <deicool@gmail.com> wrote:
>
>>
>> Hey
>>
>> Namaskara~Nalama~Guten Tag~Bonjour
>>
>> The Hadoop document gives a pointer that HDFS relaxes some POSIX
>> requirements to maintain high throughput rate.
>>
>> https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html
>>
>> But it does not give exactly what are the things which it relaxes? Any
>> idea which programs should I look into to find the things which HDFS
>> relaxes?
>>
>> Thanks
>> Deepak
>>    --
>> Keigu
>>
>> Deepak
>> 73500 12833
>> www.simtree.net, deepak@simtree.net
>> deicool@gmail.com
>>
>> LinkedIn: www.linkedin.com/in/deicool
>> Skype: thumsupdeicool
>> Google talk: deicool
>> Blog: http://loveandfearless.wordpress.com
>> Facebook: http://www.facebook.com/deicool
>>
>> "Contribute to the world, environment and more :
>> http://www.gridrepublic.org
>> "
>>
>
>

Mime
View raw message