apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikram Patil <vik...@datatorrent.com>
Subject Re: Hdfs + apex-core
Date Mon, 15 May 2017 12:51:24 GMT
Hi Rohit,

Is this correct IP address 10.0.0.0 <http://10.0.0.0:8030/> ? Normally IP
addresses ending with 0 are network addresses.

Thanks & Regards,
Vikram


On Mon, May 15, 2017 at 4:20 PM, userguy <rohit.garg41@gmail.com> wrote:

> Hello ,
>
> Are you launching an application using apex-cli?
> Ans - Yes
>
> 2) If you are then is it same machine as before which you used WordCount
> example to launch from? Is it an example from Apache Malhar ? or Hadoop
> example?
>
> Ans - WordCount Example is from Hadoop Test Jar - ( I means Hadoop Map
> reduce Job for WordCount works Properly on Same Cluster )
>
> 3) Is your yarn-site.xml is same on all the nodes?
> yes
>
> 4) If possible can you provide yarn-site.xml from all nodes in the email
> thread?
> Sharing
>
> -----------YARN SITE----------
>
> -->
> <configuration>
>     <property>
>         <name>yarn.nodemanager.aux-services</name>
>         <value>mapreduce_shuffle</value>
>     </property>
>
>     <property>
>         <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
>         <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>     </property>
>     <property>
>        <name>yarn.resourcemanager.address</name>
>           <value>10.0.0.0:8032</value>
>      </property>
>       <property>
>      <name>yarn.resourcemanager.resource-tracker.address</name>
>     <value>10.0.0.0:8031</value>
>     </property>
>       <property>
>       <name>yarn.resourcemanager.scheduler.address</name>
>        <value>10.0.0.0:8030</value>
>       </property>
>     <property>
>         <name>yarn.nodemanager.local-dirs</name>
>         <value>file:///var/data/yarn/local</value>
>     </property>
>
>     <property>
>         <name>yarn.nodemanager.log-dirs</name>
>         <value>file:///var/data/yarn/logs</value>
>     </property>
>
>     <property>
>         <name>yarn.log.aggregation-enable</name>
>         <value>true</value>
>     </property>
>
>     <property>
>         <description>Where to aggregate logs</description>
>         <name>yarn.nodemanager.remote-app-log-dir</name>
>         <value>hdfs://10.0.0.0:9000/var/log/hadoop_yarn/apps</value>
>     </property>
>
>     <property>
>         <name>yarn.nodemanager.resource.memory-mb</name>
>         <value>40960</value>
>     </property>
>
>     <property>
>         <name>yarn.scheduler.minimum-allocation-mb</name>
>         <value>1024</value>
>     </property>
>
>     <property>
>         <name>yarn.scheduler.maximum-allocation-mb</name>
>         <value>40960</value>
>     </property>
>
>     <property>
>         <name>yarn.log-aggregation-enable</name>
>         <value>True</value>
>     </property>
>
>     <property>
>         <name>yarn.log-aggregation.retain-seconds</name>
>         <value>604800</value>
>     </property>
>
>   <property>
>         <name>yarn.application.classpath</name>
>         <value>
>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/etc/hadoop/*,
>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/common/*,
>
> /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/common/lib/*,
>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/hdfs/*,
>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/hdfs/
> lib/*,
>
> /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/mapreduce/*,
>
> /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/mapreduce/lib/*,
>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/yarn/*,
>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/yarn/
> lib/*
>         </value>
>     </property>
>
> </configuration>
>
> It is excatly same for all the servers
>
>
>
>
> --
> View this message in context: http://apache-apex-users-list.
> 78494.x6.nabble.com/Fwd-Hdfs-apex-core-tp1608p1614.html
> Sent from the Apache Apex Users list mailing list archive at Nabble.com.
>

Mime
View raw message