apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AJAY GUPTA <ajaygit...@gmail.com>
Subject Re: Hdfs + apex-core
Date Mon, 15 May 2017 13:20:31 GMT
Hi Rohit,

Can you send us the pom.xml of your application as well.


Ajay


On Mon, May 15, 2017 at 6:21 PM, Vikram Patil <vikram@datatorrent.com>
wrote:

> Hi Rohit,
>
> Is this correct IP address 10.0.0.0 <http://10.0.0.0:8030/> ? Normally IP
> addresses ending with 0 are network addresses.
>
> Thanks & Regards,
> Vikram
>
>
> On Mon, May 15, 2017 at 4:20 PM, userguy <rohit.garg41@gmail.com> wrote:
>
>> Hello ,
>>
>> Are you launching an application using apex-cli?
>> Ans - Yes
>>
>> 2) If you are then is it same machine as before which you used WordCount
>> example to launch from? Is it an example from Apache Malhar ? or Hadoop
>> example?
>>
>> Ans - WordCount Example is from Hadoop Test Jar - ( I means Hadoop Map
>> reduce Job for WordCount works Properly on Same Cluster )
>>
>> 3) Is your yarn-site.xml is same on all the nodes?
>> yes
>>
>> 4) If possible can you provide yarn-site.xml from all nodes in the email
>> thread?
>> Sharing
>>
>> -----------YARN SITE----------
>>
>> -->
>> <configuration>
>>     <property>
>>         <name>yarn.nodemanager.aux-services</name>
>>         <value>mapreduce_shuffle</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class<
>> /name>
>>         <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>>     </property>
>>     <property>
>>        <name>yarn.resourcemanager.address</name>
>>           <value>10.0.0.0:8032</value>
>>      </property>
>>       <property>
>>      <name>yarn.resourcemanager.resource-tracker.address</name>
>>     <value>10.0.0.0:8031</value>
>>     </property>
>>       <property>
>>       <name>yarn.resourcemanager.scheduler.address</name>
>>        <value>10.0.0.0:8030</value>
>>       </property>
>>     <property>
>>         <name>yarn.nodemanager.local-dirs</name>
>>         <value>file:///var/data/yarn/local</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.nodemanager.log-dirs</name>
>>         <value>file:///var/data/yarn/logs</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.log.aggregation-enable</name>
>>         <value>true</value>
>>     </property>
>>
>>     <property>
>>         <description>Where to aggregate logs</description>
>>         <name>yarn.nodemanager.remote-app-log-dir</name>
>>         <value>hdfs://10.0.0.0:9000/var/log/hadoop_yarn/apps</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.nodemanager.resource.memory-mb</name>
>>         <value>40960</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.scheduler.minimum-allocation-mb</name>
>>         <value>1024</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.scheduler.maximum-allocation-mb</name>
>>         <value>40960</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.log-aggregation-enable</name>
>>         <value>True</value>
>>     </property>
>>
>>     <property>
>>         <name>yarn.log-aggregation.retain-seconds</name>
>>         <value>604800</value>
>>     </property>
>>
>>   <property>
>>         <name>yarn.application.classpath</name>
>>         <value>
>>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/etc/hadoop/*,
>>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/common/
>> *,
>>
>> /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/common/lib/*,
>>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/hdfs/*,
>>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/hdfs/li
>> b/*,
>>
>> /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/mapreduce/*,
>>
>> /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/mapreduce/lib/*,
>>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/yarn/*,
>>             /var/data/hadoop/hadoop-2.6.0-cdh5.11.0/share/hadoop/yarn/li
>> b/*
>>         </value>
>>     </property>
>>
>> </configuration>
>>
>> It is excatly same for all the servers
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-apex-users-list.
>> 78494.x6.nabble.com/Fwd-Hdfs-apex-core-tp1608p1614.html
>> Sent from the Apache Apex Users list mailing list archive at Nabble.com.
>>
>
>

Mime
View raw message