hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From divye sheth <divs.sh...@gmail.com>
Subject Re: Exceptions in Hadoop and Hbase log files
Date Fri, 18 Oct 2013 10:33:56 GMT
I would recommend you to stop the cluster and then start the daemons one by
one.
1. stop-dfs.sh
2. hadoop-daemon.sh start namenode
3. hadoop-daemon.sh start datanode

This will show start up errors if any, also verify if the datanode is able
to communicate with the namenode.

Thanks
Divye Sheth


On Fri, Oct 18, 2013 at 3:51 PM, Vimal Jain <vkjk89@gmail.com> wrote:

> Hi,
> I am running Hbase in pseudo distributed mode.( Hbase 0.94.7 and Hadoop
> 1.1.2).
> I am getting certain exceptions in Hadoop's namenode and data node files
> which are :-
>
> Namenode :-
>
> 2013-10-18 10:33:37,218 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.heartbeatCheck: lost heartbeat from 192.168.20.30:50010
> 2013-10-18 10:33:37,242 INFO org.apache.hadoop.net.NetworkTopology:
> Removing a node: /default-rack/192.168.20.30:50010
> 2013-10-18 10:35:27,606 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
> transactions: 64 Total time for transactions(ms): 1Number
> of transactions batched in Syncs: 0 Number of syncs: 43 SyncTimes(ms): 86
> 2013-10-18 10:35:27,614 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:hadoop cause:java.io.IOException: File /h
>
> base/event_data/433b61f2a4ebff8f2e4b89890508a3b7/.tmp/99797a61a8f7471cb6df8f7b95f18e9e
> could only be replicated to 0 nodes, instead of 1
> 2013-10-18 10:35:27,895 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 9 on 9000, call
> addBlock(/hbase/event_data/433b61f2a4ebff8f2e4b89890508a
> 3b7/.tmp/99797a61a8f7471cb6df8f7b95f18e9e,
> DFSClient_hb_rs_hbase.rummycircle.com,60020,1382012725057, null) from
> 192.168.20.30:44990: error: java.io.I
> OException: File
>
> /hbase/event_data/433b61f2a4ebff8f2e4b89890508a3b7/.tmp/99797a61a8f7471cb6df8f7b95f18e9e
> could only be replicated to 0 nodes, instead
>  of 1
> java.io.IOException: File
>
> /hbase/event_data/433b61f2a4ebff8f2e4b89890508a3b7/.tmp/99797a61a8f7471cb6df8f7b95f18e9e
> could only be replicated to 0 nodes
> , instead of 1
>     at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1639)
>     at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:736)
>     at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:597)
>     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
>     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
>     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:396)
>     at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
>
>
> Data node :-
>
> 2013-10-18 06:13:14,499 WARN
> org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
> 192.168.20.30:50010, storageID=DS-1816106352-192.16
> 8.20.30-50010-1369314076237, infoPort=50075, ipcPort=50020):Got exception
> while serving blk_-3215981820534544354_52215 to /192.168.20.30:
> java.net.SocketTimeoutException: 480000 millis timeout while waiting for
> channel to be ready for write. ch :
> java.nio.channels.SocketChannel[connected
>  local=/192.168.20.30:50010 remote=/192.168.20.30:36188]
>     at
>
> org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
>     at
>
> org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
>     at
>
> org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
>     at
>
> org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:392)
>     at
>
> org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:490)
>     at
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:202)
>     at
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:104)
>     at java.lang.Thread.run(Thread.java:662)
>
>
>
>
>
>
>
> --
> Thanks and Regards,
> Vimal Jain
>

Mime
View raw message