ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Boudnik (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-218) Wrong staging permissions while running MR job under hadoop accelerator
Date Thu, 26 Feb 2015 00:20:04 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14337318#comment-14337318
] 

Konstantin Boudnik edited comment on IGNITE-218 at 2/26/15 12:19 AM:
---------------------------------------------------------------------

It is quite possible the configuration issue. Here's what I have done:
 - added 
{code}
        <property name="secondaryFileSystem">
          <bean class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
            <constructor-arg value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
            <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
          </bean>
        </property>
{code}
inside 
{code}
        <property name="ggfsConfiguration">
            <list>
                <bean class="org.gridgain.grid.ggfs.GridGgfsConfiguration" parent="ggfsCfgBase">
{code}
And now I can list the HDFS using gridgain-hadoop client configuration, and put files into
the file system. But once an MR job is ran I see that reported issue. 
The reason I did the configuration above instead of one explained here http://hadoop.gridgain.org/latest/Configuration
is that I couldn't make that latter work ;( Perhaps, it's my inability to read the documentation.
But nodes refuse to start and log files don't have any information that's might relevant to
the issue. By running {{ggstart.sh}} manually I found that
{noformat}
[root@vmhost05-hbase1 conf]# /usr/lib/gridgain-hadoop/bin/ggstart.sh
class org.gridgain.grid.GridException: Failed to instantiate Spring XML application context
[springUrl=file:/usr/lib/gridgain-hadoop/config/default-
config.xml, err=Error creating bean with name 'grid.cfg' defined in URL [file:/usr/lib/gridgain-hadoop/config/default-config.xml]:
Cannot create inn
er bean 'org.gridgain.grid.ggfs.GridGgfsConfiguration#29e700f6' of type [org.gridgain.grid.ggfs.GridGgfsConfiguration]
while setting bean property '
ggfsConfiguration' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'org.gri
dgain.grid.ggfs.GridGgfsConfiguration#29e700f6' defined in URL [file:/usr/lib/gridgain-hadoop/config/default-config.xml]:
Error setting property val
ues; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property
'secondaryHadoopFileSystemUri' of bean class [org.
gridgain.grid.ggfs.GridGgfsConfiguration]: Bean property 'secondaryHadoopFileSystemUri' is
not writable or has an invalid setter method. Does the pa
rameter type of the setter match the return type of the getter?]
For more information see:
    Troubleshooting:      http://bit.ly/GridGain-Troubleshooting
    Documentation Center: http://bit.ly/GridGain-Documentation

        at org.gridgain.grid.kernal.processors.spring.GridSpringProcessorImpl.loadConfigurations(GridSpringProcessorImpl.java:98)
        at org.gridgain.grid.kernal.GridGainEx.loadConfigurations(GridGainEx.java:560)
        at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:739)
        at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:677)
        at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:524)
        at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:494)
        at org.gridgain.grid.GridGain.start(GridGain.java:314)
        at org.gridgain.grid.startup.cmdline.GridCommandLineStartup.main(GridCommandLineStartup.java:293)
{noformat}
Perhaps it is because I am using config that isn't compatible with an older version? Anything
else? 

Another issue: when the underlying HDFS NN is in the safe mode, the GGFS error message is
wrong:
{{put: class org.gridgain.grid.ggfs.GridGgfsInvalidHdfsVersionException: HDFS version you
are connecting to differs from local version.}}


was (Author: cos):
It is quite possible the configuration issue. Here's what I have done:
 - added 
{code}
        <property name="secondaryFileSystem">
          <bean class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
            <constructor-arg value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
            <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
          </bean>
        </property>
{code}
inside 
{code}
        <property name="ggfsConfiguration">
            <list>
                <bean class="org.gridgain.grid.ggfs.GridGgfsConfiguration" parent="ggfsCfgBase">
{code}
And now I can list the HDFS using gridgain-hadoop client configuration, and put files into
the file system. But once an MR job is ran I see that reported issue. 
The reason I did the configuration above instead of one explained here http://hadoop.gridgain.org/latest/Configuration
is that I couldn't make that latter work ;( Perhaps, it's my inability to read the documentation.
But nodes refuse to start and log files don't have any information that's might relevant to
the issue.

Another issue: when the underlying HDFS NN is in the safe mode, the GGFS error message is
wrong:
{{put: class org.gridgain.grid.ggfs.GridGgfsInvalidHdfsVersionException: HDFS version you
are connecting to differs from local version.}}

> Wrong staging permissions while running MR job under hadoop accelerator
> -----------------------------------------------------------------------
>
>                 Key: IGNITE-218
>                 URL: https://issues.apache.org/jira/browse/IGNITE-218
>             Project: Ignite
>          Issue Type: Bug
>          Components: hadoop
>    Affects Versions: sprint-1
>         Environment: Hadoop 2.4.1
> gridgain-hadoop-6.5.6 (Bigtop trunk)
>            Reporter: Konstantin Boudnik
>            Assignee: Ivan Veselovsky
>             Fix For: sprint-2
>
>
> I am trying to run an MR example on gridgain-hadoop setup in Bigtop. The ggfs is configured
to be backed by HDFS with
> {code}
>         <property name="secondaryFileSystem">
>           <bean class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
>             <constructor-arg value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
>             <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
>           </bean>
>         </property>
> {code}
> set for {{ggfsConfiguration}}
> While all usual filesystem operations work properly, running an MR job produces the following
exception in the beginning of the run
> {noformat}
> HADOOP_CONF_DIR=/etc/hadoop/gridgain.client.conf hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar
pi 2 2
> Number of Maps  = 2
> Samples per Map = 2
> Wrote input for Map #0
> Wrote input for Map #1
> Starting Job
> java.io.IOException: The ownership on the staging directory /tmp/hadoop-yarn/staging/root/.staging
is not as expected. It is owned by gridgain. The directory must be owned by the submitter
root or by root
>         at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:112)
>         at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
>         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>         at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
>         at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {noformat}
> However the permissions seem to be correct.
> {noformat}
> [root@vmhost05-hbase2 gridgain.client.conf]# sudo -u hdfs hadoop fs -ls /tmp/hadoop-yarn/staging/root
                                            Found 1 items
> drwx------   - root root          0 2015-02-10 19:31 /tmp/hadoop-yarn/staging/root/.staging
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message