ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yaoqin <yao...@huawei.com>
Subject 答复: How to access data stored in IGFS or IGFS as Hadoop cache
Date Mon, 07 Sep 2015 11:57:26 GMT
1.       If the way accessing IGFS is same as HDFS, how can I ensure where the data is stored
, in IGFS or HDFS? (consider IGFS as HDFS cache mode)
And if I want to compare the performance of these two file system, what shall I do?


2.       I tried  to set fs.defaultFS as “igfs:///”, but hadoop could not start for no
namenode detected


发件人: Vladimir Ozerov [mailto:vozerov@gridgain.com]
发送时间: 2015年9月7日 15:13
收件人: user@ignite.apache.org
抄送: Hejun (Jun He)
主题: Re: How to access data stored in IGFS or IGFS as Hadoop cache


>> 1.  How to access(set/get) data stored in IGFS or IGFS as Hadoop cache?

This depeds on how do you want to access it. If you are accessing data form Hadoop, then it
is not different from any other Hadoop file systems: you can use FileSystem API, "dfs" command
line tool, etc.. If you want to access data through Ignite API, use IgniteFileSystem API:
https://apacheignite.readme.io/docs/igfs

>> 2.  In IGFS as Hadoop cache mode, is core-site.xml need to be configured? (hadoop
v2)

Yes, it needs to be configured.

>> - what shall fs.defaultFS be?

"fs.defaultFS" parameter should be set to the same value as "fs.default.name<http://fs.default.name>"
in v1. It should be URI to running IGFS instance. Please see "Configure Hadoop" and "File
System URI" paragraphs here https://apacheignite.readme.io/docs/file-system for examples.

Let us know if you still have outstanding questions.

>> - where shall this core-site.xml file put, in hadoop path or ignite path or both?

core-site.xml is only needed by Hadoop. Please put it into Hadoop path as usual.

On Sun, Sep 6, 2015 at 7:14 AM, yaoqin <yaoqin@huawei.com<mailto:yaoqin@huawei.com>>
wrote:

1.  How to access(set/get) data stored in IGFS or IGFS as Hadoop cache?

2.  In IGFS as Hadoop cache mode, is core-site.xml need to be configured? (hadoop v2)

- what shall fs.defaultFS be?

- where shall this core-site.xml file put, in hadoop path or ignite path or both?





Mime
View raw message