hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AnushaGuntaka <anusha.gunt...@tcs.com>
Subject DataXceiver java.io.InterruptedIOException error on scannning Hbase table
Date Tue, 20 May 2014 14:10:28 GMT
Hi ,

Thanks in advance. Please help me out in figuring cause of the follwing
error and fixing it.

Am facing the below error while scanning a hbase table with partial RowKey
filter through MapReduce program.

Error: org.apache.hadoop.horg.apache.hadoop.hdfs.server.datanode.DataNode:
DatanodeRegistration():DataXceiver java.io.InterruptedIOException:
Interruped while waiting for IO on channel
java.nio.channels.SocketChannel[closed] 

Data node on the slave node is getting shutdown on this error.

My Map reduce program is running maptsks till 95% and then failing with this
error.

I have a hadoop cluster with two mechines ,

Table Size : 652 GB (223 GB in master Node  and 514GB in slave node)

System Disc details:

Node            space available 
---------------------------------
master   ----   22 GB  
slave     ----   210 GB

------------------------------- core-site.xml -----------------------
<configuration>
<property>

 <name>fs.tmp.dir</name>
  <value>/home/e521596/hadoop-1.1.1/full</value>
 </property>

 <property>
   <name>fs.default.name</name>
  <value>hdfs://172.20.193.234:9000</value>
   </property>

<property>
 <name>io.sort.factor</name>
   <value>15</value>
      <description>More streams merged at once while sorting
files.</description>
          </property>

<property>
<name>io.sort.mb</name>
<value>1000</value>
<description>Higher memory-limit while sorting data.</description>
</property>

<property>
<name>io.sort.record.percent</name>
<value>0.207</value>
<description>Higher memory-limit while sorting data.</description>
</property>

<property>
<name>io.sort.spill.percent</name>
 <value>1</value>
  <description>Higher memory-limit while sorting data.</description>
   </property>

</configuration>
------------------------------- mapred-site.xml -----------------------

<configuration>
  <property>
        <name>mapred.job.tracker</name>
        <value>fedora3:9001</value>
  </property>
  <property>
       <name>mapred.reduce.tasks</name>
       <value>6</value>
  </property>
  <property>
        <name>mapred.tasktracker.map.tasks.maximum</name>
        <value>6</value>
  </property>
  <property>
        <name>mapred.tasktracker.reduce.tasks.maximum</name>
        <value>6</value>
  </property>
  <property>
       <name>mapred.textoutputformat.separator</name>
       <value>#</value>
  </property>

 <property>
        <name>mapred.compress.map.output</name>
               <value>true</value>
                 </property>

 <property>
         <name>mapred.child.java.opts</name>
          <value>-Xms1024M -Xmx2048M</value>
  </property>


</configuration>
 ---------------------------------------- hdfs-site.xml--------------------

<configuration>
  <property>
        <name>dfs.name.dir</name>
        <value>/home/e521596/hadoop-1.1.1/full/dfs/name</value>
  </property>
  <property>
       <name>dfs.data.dir</name>
       <value>/home/e521596/hadoop-1.1.1/full/dfs/data</value>
  </property>
  <property>
     <name>dfs.replication</name>
       <value>1</value>
  </property>
<property>
     <name>dfs.datanode.max.xcievers</name>
          <value>5096</value>
            </property>

<property>
     <name>dfs.datanode.handler.count</name>
          <value>200</value>
            </property>
<property>
     <name>dfs.datanode.socket.write.timeout</name>
               <value>0</value>
                           </property>


</configuration>
---------------------------------------------------------------------






--
View this message in context: http://apache-hbase.679495.n3.nabble.com/DataXceiver-java-io-InterruptedIOException-error-on-scannning-Hbase-table-tp4059419.html
Sent from the HBase User mailing list archive at Nabble.com.

Mime
View raw message