hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (Naga)" <garlanaganarasi...@huawei.com>
Subject [No Subject]
Date Fri, 26 Sep 2014 14:10:34 GMT
Hi All,

I have following doubts on pluggable FileSystem and YARN
1. If all the implementations should extend FileSystem then why there is a parallel class
AbstractFileSystem. which ViewFS extends ?
2. Is YARN supposed to run on any of the pluggable org.apache.hadoop.fs.FileSystem like s3
?
if its suppose to run then when submitting a job in the client side  YARNRunner is calling
FileContext.getFileContext(this.conf);
which is further calling FileContext.getAbstractFileSystem() which throws exception for S3.
So i am not able to run YARN job with ViewFS with S3 as mount. And based on the code even
if i configure only S3 then also its going to fail.
3. HDFS does not support "fs.AbstractFileSystem.s3.impl" with some default class similar to
org.apache.hadoop.fs.s3.S3FileSystem ?


Regards,

Naga



Huawei Technologies Co., Ltd.
Phone:
Fax:
Mobile:  +91 9980040283
Email: naganarasimhagr@huawei.com<mailto:naganarasimhagr@huawei.com>
Huawei Technologies Co., Ltd.
http://www.huawei.com


Mime
View raw message