hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tenghuan He <tenghua...@gmail.com>
Subject Directly reading from datanode using JAVA API got socketTimeoutException
Date Wed, 30 Dec 2015 17:29:37 GMT
​Hello,

I want to directly read from datanode blocks using JAVA API as the
following code, but I got socketTimeoutException

I use reflection to call the DFSClient private method connectToDN(...), and
get IOStreamPair of in and out, where in is used to read bytes from
datanode.
The workhorse code is

try {
    Method connectToDN;
    Class[] paraList = {DatanodeInfo.class, int.class, LocatedBlock.class};
    connectToDN =
dfsClient.getClass().getDeclaredMethod("connectToDN", paraList);
    connectToDN.setAccessible(true);
    IOStreamPair pair = (IOStreamPair) connectToDN.invoke(dfsClient,
datanode, timeout, lb);
    in = new DataInputStream(pair.in);
    System.out.println(in.getClass());
    byte[] b = new byte[10000];
    in.readFully(b);
} catch (Exception e) {
    e.printStackTrace();

}

and the exception is

java.net.SocketTimeoutException: 11000 millis timeout while waiting for
channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected local=/192.168.179.1:53765
remote=/192.168.179.135:50010]
at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at BlocksList.main(BlocksList.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)​

Could anyone tell me where the problem is?

Thanks & Begards

Tenghuan He

Mime
View raw message