hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaojinchao <gaojinc...@huawei.com>
Subject A question about HBase accesses the HDFS
Date Mon, 09 Jan 2012 05:13:19 GMT
In DN logs, There are a lot of "480000 millis timeout" exceptions when some scans finished,
But Region server have no exceptions.
I analyzed the flow of HBase and Hadoop. I found we are using API "readBuffer(byte buf[],
int off, int len)", It means we need read the whole block data, if we stop reading, some timeout
will happen in DN side.

I want to know whether is the bad effect?
For example leaking the socket, I found some "CLOSE_WAIT" in our machine, I did not know whether
it is the result.

Who has experience? Please help explain.

Thank You.

The HDFS API
2012-01-07 07:02:40,753 INFO org.apache.hadoop.hdfs.DFSClient: Hbase is invoking ....................
seek(long targetPos)
2012-01-07 07:02:40,753 INFO org.apache.hadoop.hdfs.DFSClient: Hbase is invoking.................................read(byte
buf[], int off, int len)
2012-01-07 07:02:40,754 INFO org.apache.hadoop.hdfs.DFSClient: Hbase is invoking.................................readBuffer(byte
buf[], int off, int len)

DN Logs:
2012-01-08 02:52:32,969 WARN  datanode.DataNode (DataXceiver.java:readBlock(274)) - DatanodeRegistration(158.1.130.33:10010,
storageID=DS-1985031385-158.1.130.33-10010-1318824003883, infoPort=10075, ipcPort=10020):Got
exception while serving blk_1325150650461_45153 to /158.1.130.33:
java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready
for write. ch : java.nio.channels.SocketChannel[connected local=/158.1.130.33:10010 remote=/158.1.130.33:53792]
         at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:247)
         at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
         at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
         at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:410)
         at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:508)
         at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:247)
         at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:130)
         at java.lang.Thread.run(Thread.java:662)

2012-01-08 02:52:32,969 ERROR datanode.DataNode (DataXceiver.java:run(183)) - org.apache.hadoop.hdfs.server.datanode.DataNode
(158.1.130.33:10010, storageID=DS-1985031385-158.1.130.33-10010-1318824003883, infoPort=10075,
ipcPort=10020):DataXceiver
java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready
for write. ch : java.nio.channels.SocketChannel[connected local=/158.1.130.33:10010 remote=/158.1.130.33:53792]
         at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:247)
         at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
         at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
         at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:410)
         at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:508)
         at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:247)
         at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:130)
         at java.lang.Thread.run(Thread.java:662)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message