ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yaoqin <yao...@huawei.com>
Subject Hadoop File System Cache Setting Problems
Date Fri, 21 Aug 2015 03:35:08 GMT
HI! I'm looking for help from you to set and test Apache Ignite with Hadoop V2 and Spark 1.5.x.
I'm confused about the following questions, and hope to get your help.



I'd like to set IGFS as Hadoop FileSystem Cache. And I modified "$IGNITE_HOME/examples/config/filesystem/example-igfs.xml"
to "$IGNITE_HOME/examples/config/filesystem/yq-example-igfs.xml" by adding the following elements
to it:

[计算机生成了可选文字: 《I一 Exampleofconfiguredsharedmemoryendpoint. -一>
<!一 <propertyname=',ipcEndpointConfiguration"> <propertyname='"type''value=''SHME州‘,/>
</bean> </p户operty>一 -一> <beanclass="org.apache.ignite.igfs.IgfsIpcEndpointConfiguration
,,沪― <卜一propertyname='"secondaryFilesystem',> <beanclass=',org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFi1eSystem''>
<constructor一argvalue="hdfs://master:9000""/> -</bean> </property一> </bean>
万而二沂气月丫洁{ 厂月 只之一 二__准式_丫耳狱厂一― 、J丫、了一~勺厂一一“气一口压
<beanclass=,org.apache.ignite.configuration <propertyname="secondaryFilesystem",>
.FilesystemConfiguration"> 、入 <beanclass=',org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFi1eSystem">
<constructor一argvalue='hdfs://master:9000,'/>, </bean> be3n> </list>
</property> </property> 尸]



For detail, you can see the Appendix I: yq-example-igfs.xml



I can get ignite started by example-igfs.xml rightly, but when I use yq-example-igfs.xml,
it always fails  like this:







master:/home/software/apache-ignite # ignite.sh -i

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed
in 8.0

Available configuration files:

+----------------------------+

0:      config/default-config.xml

1:      (?) config/router/default-router.xml

2:      examples/config/example-cache.xml

3:      examples/config/example-ignite.xml

4:      examples/config/filesystem/example-igfs.xml

5:      examples/config/filesystem/yq-example-igfs.xml

6:      examples/src/main/java/org/apache/ignite/examples/misc/springbean/spring-bean.xml



Choose configuration file ('c' to cancel) [0]: 5



Using configuration: examples/config/filesystem/yq-example-igfs.xml



class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context
(make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/home/software/apache-ignite/examples/config/filesystem/yq-example-igfs.xml]

        at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:684)

        at org.apache.ignite.Ignition.start(Ignition.java:349)

        at org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:289)

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML
application context (make sure all classes used in Spring configuration are present at CLASSPATH)
[springUrl=file:/home/software/apache-ignite/examples/config/filesystem/yq-example-igfs.xml]

        at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:370)

        at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:87)

        at org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:81)

        at org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:547)

        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:740)

        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:671)

        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:527)

        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:497)

        at org.apache.ignite.Ignition.start(Ignition.java:346)

        ... 1 more

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with
name 'ignite.cfg' defined in URL [file:/home/software/apache-ignite/examples/config/filesystem/yq-example-igfs.xml<file:///\\home\software\apache-ignite\examples\config\filesystem\yq-example-igfs.xml>]:
Cannot create inner bean 'org.apache.ignite.configuration.FileSystemConfiguration#131276c2'
of type [org.apache.ignite.configuration.FileSystemConfiguration] while setting bean property
'fileSystemConfiguration' with key [1]; nested exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'org.apache.ignite.configuration.FileSystemConfiguration#131276c2'
defined in URL [file:/home/software/apache-ignite/examples/config/filesystem/yq-example-igfs.xml<file:///\\home\software\apache-ignite\examples\config\filesystem\yq-example-igfs.xml>]:
Cannot create inner bean 'org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem#1c3a4799'
of type [org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem] while setting bean
property 'secondaryFileSystem'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException:
Cannot find class [org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem] for bean
with name 'org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem#1c3a4799' defined
in URL [file:/home/software/apache-ignite/examples/config/filesystem/yq-example-igfs.xml<file:///\\home\software\apache-ignite\examples\config\filesystem\yq-example-igfs.xml>];
nested exception is java.lang.ClassNotFoundException: org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem







And in https://apacheignite.readme.io/v1.3/docs/file-system section Configure Hadoop

2) Hadoop must have Ignite JARS ${IGNITE_HOME}\libs\ignite-core-[version].jar and ${IGNITE_HOME}\libs\hadoop\ignite-hadoop-[version].ja"
in CLASSPATH.

Does ignite-hadoop-[version].ja mean ignite-yarn-[version].jar when I use Hadoop v2?





Mime
View raw message