ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SS <sausharm...@gmail.com>
Subject Hive on Ignited- Yarn
Date Tue, 20 Sep 2016 07:02:45 GMT
Hi 

I am runing a hadoop 2.6 cluster. 

mapred-site.xml 

  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>



Now I am trying to run hive-on-ignite. Based on configuration, my
hive-ignite is also working but I do not see any performance benefit in
hive-on-yarn vs hive-on-ignite. 

Anything which I am missing about Ignite (
apache-ignite-hadoop-1.7.0-bin)........ Here is my config files. 


[18:38:11] Ignite node started OK (id=97a46cec) 
[18:38:11] Topology snapshot [ver=1, servers=1, clients=0, CPUs=24,
heap=3.6GB] 
[18:40:09] Topology snapshot [ver=2, servers=2, clients=0, CPUs=48,
heap=7.1GB] 
[18:40:53] Topology snapshot [ver=3, servers=3, clients=0, CPUs=72,
heap=11.0GB] 
[18:41:31] Topology snapshot [ver=4, servers=4, clients=0, CPUs=96,
heap=14.0GB] 
[18:41:44] Topology snapshot [ver=5, servers=5, clients=0, CPUs=120,
heap=18.0GB] 
[18:42:02] Topology snapshot [ver=6, servers=6, clients=0, CPUs=144,
heap=21.0GB] 

default-config.xml 
============== 
   <property name="ipcEndpointConfiguration">
                        <bean
class="org.apache.ignite.igfs.IgfsIpcEndpointConfiguration">
                            <property name="type" value="TCP" />
                            <property name="host" value="hmaster1" />
                            <property name="port" value="10500" />
                        </bean>
                    </property>

 <property name="secondaryFileSystem">
                        <bean
class="org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem">
                            <constructor-arg name="uri"
value="hdfs://hmaster1:9000"/>
                            
                            <constructor-arg
name="cfgPath"><null/></constructor-arg>
                            
                        </bean>
                    </property>

mapred-site.xml 
=========== 

   <property>
        <name>mapreduce.framework.name</name>
        <value>ignite</value>
    </property>

    
    <property>
        <name>mapreduce.jobtracker.address</name>
        <value>hmaster1:11211</value>
    </property>


core-site.xml 
========== 

 <property>
        <name>fs.default.name</name>
        <value>igfs://igfs@hmaster1:10500</value>
    </property>

    
    <property>
        <name>fs.igfs.impl</name>
        <value>org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem</value>
    </property>

    
    <property>
        <name>fs.AbstractFileSystem.igfs.impl</name>
        <value>org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem</value>
    </property>

    
    <property>
       
<name>dfs.client.block.write.replace-datanode-on-failure.policy</name>
        <value>NEVER</value>
    </property>



Hive Job log 
========== 

Starting Job = job_1473330410172_0023, Tracking URL =
http://hmaster1:8088/proxy/application_1473330410172_0023/

8088 is default yarn job port which is not changing... Do I need to change
anything here. 


Regards



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Hive-on-Ignited-Yarn-tp7838.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message