ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Veselovsky (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-179) Secondary Hadoop FS URI is not supported
Date Wed, 25 Feb 2015 08:20:04 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325857#comment-14325857
] 

Ivan Veselovsky edited comment on IGNITE-179 at 2/25/15 8:19 AM:
-----------------------------------------------------------------

Problem is that the documentation at http://hadoop.gridgain.org/latest/Configuration is relevant
to GridGain release 6.5.1, but not quite relevant to release 6.6.2.

A valid configuration of the secondary filesystem for GridGain 6.6.2 (<GRIDGAIN_HOME>/config/default-config.xml)
may look like the following:
....
        <property name="ggfsConfiguration">
            <list>
                <bean class="org.gridgain.grid.ggfs.GridGgfsConfiguration" parent="ggfsCfgBase">
                    <property name="name" value="ggfs"/>
                    ....

                    <property name="ipcEndpointConfiguration">
                        <map>
                            <entry key="type" value="tcp"/>
                            <entry key="host" value="0.0.0.0"/>
                            <entry key="port" value="9999"/>
                        </map>
                    </property>

                    <property name="defaultMode" value="PROXY"/>

                     <!-- Secondary Hadoop FS URI. -->
                     <property name="secondaryFileSystem">
                         <bean class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
                             <constructor-arg name="uri"     value="hdfs://localhost:9000"/>
                             <constructor-arg name="cfgPath" value="<HADOOP_SERVER>/etc/hadoop/core-site.xml"/>
                         </bean>
                     </property>

                </bean>
            </list>
        </property>
....

Where
 hdfs://localhost:9000 is the URI of Hadoop HDFS filesystem configured on the Hadoop server,
 <HADOOP_SERVER>/etc/hadoop/core-site.xml is the Hadoop server configuration file, and
 0.0.0.0:9999 corresponds to the port specified for ggfs filesystem in Hadoop *client*  configuration,
<HADOOP_CLIENT>/etc/hadoop/core-site.xml (this is the config initially substituted by
script "setup-hadoop.sh"), like this:

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>ggfs://ggfs@localhost:9999</value>
    </property>

    <property>
        <name>fs.ggfs.impl</name>
        <value>org.gridgain.grid.ggfs.hadoop.v1.GridGgfsHadoopFileSystem</value>
    </property>
....


was (Author: iveselovskiy):
.

> Secondary Hadoop FS URI is not supported
> ----------------------------------------
>
>                 Key: IGNITE-179
>                 URL: https://issues.apache.org/jira/browse/IGNITE-179
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Yakov Zhdanov
>            Assignee: Ivan Veselovsky
>             Fix For: sprint-2
>
>
> asked at https://github.com/gridgain/gridgain/issues/94
> {noformat}
> Hi, all
> Under the guidance of http://hadoop.gridgain.org/latest/Configuration, I add the following
configuration items into default-config.xml to set DUAL_ASYNC mode and use Apache HDFS as
my secondary file system:
>     <!-- Set DUAL_ASYNC mode as default. -->
>     <property name="defaultMode" value="DUAL_ASYNC"/>
>     <!-- Secondary Hadoop FS URI. -->
>     <property name="secondaryHadoopFileSystemUri" value="hdfs://9.91.11.163:9000"/>
>     <!-- Secondary Hadoop FS configuration folder path. -->
>     <property name="secondaryHadoopFileSystemConfigPath" value="/home/qxw/hadoop/hadoop-2.6.0/etc/hadoop/core-site.xml"/>
> But there is following error when I run bin/ggstart.sh:
> class org.gridgain.grid.GridException: Failed to instantiate Spring XML application context
[springUrl=file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml, err=Error
creating bean with name 'grid.cfg' defined in URL [file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml]:
Cannot create inner bean 'org.gridgain.grid.ggfs.GridGgfsConfiguration#1602c861' of type [org.gridgain.grid.ggfs.GridGgfsConfiguration]
while setting bean property 'ggfsConfiguration' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'org.gridgain.grid.ggfs.GridGgfsConfiguration#1602c861' defined
in URL [file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml]: Error
setting property values; nested exception is org.springframework.beans.NotWritablePropertyException:
Invalid property 'secondaryHadoopFileSystemUri' of bean class [org.gridgain.grid.ggfs.GridGgfsConfiguration]:
Bean pr operty 'secondaryHadoopFileSystemUri' is not writable or has an invalid setter method.
Does the parameter type of the setter match the return type of the getter?]
> For more information see:
> Troubleshooting: http://bit.ly/GridGain-Troubleshooting
> Documentation Center: http://bit.ly/GridGain-Documentation
>     at org.gridgain.grid.kernal.processors.spring.GridSpringProcessorImpl.loadConfigurations(GridSpringProcessorImpl.java:98)
>     at org.gridgain.grid.kernal.GridGainEx.loadConfigurations(GridGainEx.java:560)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:742)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:709)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:663)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:523)
>     at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:493)
>     at org.gridgain.grid.GridGain.start(GridGain.java:314)
>     at org.gridgain.grid.startup.cmdline.GridCommandLineStartup.main(GridCommandLineStartup.java:293)
> Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'grid.cfg' defined in URL [file:/home/qxw/gridgain/gridgain-hadoop-os-6.6.2/config/default-config.xml]
> The execption shows Bean property 'secondaryHadoopFileSystemUri' is not writable or has
an invalid setter method, and I don't find "secondaryHadoopFileSystemUri" field and relevant
method in GridGgfsConfiguration.java.
> The gridgain version I used is gridgain-hadoop-os-6.6.2.
> So, does "secondaryHadoopFileSystemUri" have been renamed? or, could it be said that
the opensource version don't support secondary file system, only support ggfs PRIMARY mode?
> Thanks.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message