ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Veselovsky (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-179) Secondary Hadoop FS URI is not supported
Date Tue, 24 Feb 2015 16:14:04 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325857#comment-14325857
] 

Ivan Veselovsky edited comment on IGNITE-179 at 2/24/15 4:13 PM:
-----------------------------------------------------------------

The documentation at http://hadoop.gridgain.org/latest/Configuration is relevant to the latest
GridGain release (6.5.1), but not quite relevant to the latest Apache Ignite.

The up-to-date configuration of the secondary filesystem for Ignite (<IGNITE_HOME>/config/default-config.xml)
looks like the following:
....
        <property name="igfsConfiguration">
            <list>
                <bean class="org.apache.ignite.configuration.IgfsConfiguration" parent="igfsCfgBase">
                    <property name="name" value="igfs"/>

                    <!-- Caches with these names must be configured. -->
                    <property name="metaCacheName" value="igfs-meta"/>
                    <property name="dataCacheName" value="igfs-data"/>

                    <!-- Configure TCP endpoint for communication with the file system
instance. -->
                    <property name="ipcEndpointConfiguration">
                        <map>
                            <entry key="type" value="tcp"/>
                            <entry key="host" value="0.0.0.0"/>
                            <entry key="port" value="9999"/>
                        </map>
                    </property>

                    <property name="defaultMode" value="PROXY"/>

                    <!-- Secondary Hadoop FS URI. -->
                    <property name="secondaryFileSystem">
                        <bean class="org.apache.ignite.internal.igfs.hadoop.IgfsHadoopFileSystemWrapper">
                            <constructor-arg name="uri"     value="hdfs://localhost:9000"/>
                            <constructor-arg name="cfgPath" value="<HADOOP_SERVER>/etc/hadoop/core-site.xml"/>
                        </bean>
                    </property>
                </bean>
            </list>
        </property>
....

Where
 hdfs://localhost:9000 is the URI of Hadoop HDFS filesystem configured on the Hadoop server,
 <HADOOP_SERVER>/etc/hadoop/core-site.xml is the Hadoop server configuration file, and
 0.0.0.0:9999 corresponds to the port specified for igfs filesystem in Hadoop *client*  configuration,
<HADOOP_CLIENT>/etc/hadoop/core-site.xml (this is the config initially substituted by
setup-hadoop.sh), like this:
<configuration>
    <property>
        <name>fs.default.name</name>
        <value>igfs://igfs@localhost:9999</value>
    </property>

    <property>
        <name>fs.igfs.impl</name>
        <value>org.apache.ignite.igfs.hadoop.v1.IgfsHadoopFileSystem</value>
    </property>

    <property>
        <name>fs.AbstractFileSystem.igfs.impl</name>
        <value>org.apache.ignite.igfs.hadoop.v2.IgfsHadoopFileSystem</value>
    </property>

    <property>
        <name>dfs.client.block.write.replace-datanode-on-failure.policy</name>
        <value>NEVER</value>
    </property>
</configuration>


was (Author: iveselovskiy):
It looks like now this method is called
org.apache.ignite.configuration.IgfsConfiguration#setSecondaryFileSystem(Igfs)

> Secondary Hadoop FS URI is not supported
> ----------------------------------------
>
>                 Key: IGNITE-179
>                 URL: https://issues.apache.org/jira/browse/IGNITE-179
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Yakov Zhdanov
>            Assignee: Ivan Veselovsky
>             Fix For: sprint-2
>
>
> asked at https://github.com/gridgain/gridgain/issues/94
> {noformat}
> Hi, all
> Under the guidance of http://hadoop.gridgain.org/latest/Configuration, I add the following
configuration items into default-config.xml to set DUAL_ASYNC mode and use Apache HDFS as
my secondary file system:
>     <!-- Set DUAL_ASYNC mode as default. -->
>     <property name="defaultMode" value="DUAL_ASYNC"/>
>     <!-- Secondary Hadoop FS URI. -->
>     <property name="secondaryHadoopFileSystemUri" value="hdfs://9.91.11.163:9000"/>
>     <!-- Secondary Hadoop FS configuration folder path. -->
>     <property name="secondaryHadoopFileSystemConfigPath" value="/home/qxw/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml"/>
> But there is following error when I run bin/ggstart.sh:
> class org.gridgain.grid.GridException: Failed to instantiate Spring XML application context
[springUrl=file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml, err=Error
creating bean with name 'grid.cfg' defined in URL [file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml]:
Cannot create inner bean 'org.gridgain.grid.ggfs.GridGgfsConfiguration#1602c861' of type [org.gridgain.grid.ggfs.GridGgfsConfiguration]
while setting bean property 'ggfsConfiguration' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'org.gridgain.grid.ggfs.GridGgfsConfiguration#1602c861' defined
in URL [file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml]: Error
setting property values; nested exception is org.springframework.beans.NotWritablePropertyException:
Invalid property 'secondaryHadoopFileSystemUri' of bean class [org.gridgain.grid.ggfs.GridGgfsConfiguration]:
Bean pr operty 'secondaryHadoopFileSystemUri' is not writable or has an invalid setter method.
Does the parameter type of the setter match the return type of the getter?]
> For more information see:
> Troubleshooting: http://bit.ly/GridGain-Troubleshooting
> Documentation Center: http://bit.ly/GridGain-Documentation
>     at org.gridgain.grid.kernal.processors.spring.GridSpringProcessorImpl.loadConfigurations(GridSpringProcessorImpl.java:98)
>     at org.gridgain.grid.kernal.GridGainEx.loadConfigurations(GridGainEx.java:560)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:742)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:709)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:663)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:523)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:493)
>     at org.gridgain.grid.GridGain.start(GridGain.java:314)
>     at org.gridgain.grid.startup.cmdline.GridCommandLineStartup.main(GridCommandLineStartup.java:293)
> Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'grid.cfg' defined in URL [file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml]
> The execption shows Bean property 'secondaryHadoopFileSystemUri' is not writable or has
an invalid setter method, and I don't find "secondaryHadoopFileSystemUri" field and relevant
method in GridGgfsConfiguration.java.
> The gridgain version I used is gridgain-hadoop-os-6.6.2.
> So, does "secondaryHadoopFileSystemUri" have been renamed? or, could it be said that
the opensource version don't support secondary file system, only support ggfs PRIMARY mode?
> Thanks.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message