hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayuresh <mayuresh.kshirsa...@gmail.com>
Subject Fwd: Coprocessor execution - HBaseServer: (responseTooSlow) java.nio.channels.ClosedChannelException
Date Fri, 23 Sep 2011 09:52:22 GMT
Hi,

I am using the AggregationClient to perform aggregate calculation on a
10 M row hbase table which is spread across around 27 regions.
Currently all the regions are on a single region server.

Here is briefly what the code is doing:
        Leases leases = new Leases(40 * 60 * 1000, 5 * 1000);
        leases.start();
        long startTime = System.currentTimeMillis();
        System.out.println("Start time: " + startTime);
        Configuration conf = new Configuration(true);
        AggregationClient aClient = new AggregationClient(conf);

        Scan scan = new Scan();
        scan.addColumn(TEST_FAMILY, TEST_QUALIFIER);
        final ColumnInterpreter<Long, Long> ci = new LongColumnInterpreter();

        double avg = aClient.avg(TEST_TABLE, ci,
                scan);
        leases.close();
        long endTime = System.currentTimeMillis();
        System.out.println("End time: " + endTime);
        System.out.println("Avg is: " + avg);
        System.out.println("Time taken is: " + (endTime - startTime));

However it is failing after some time with the following log messages
in the region server log:
2011-09-23 14:21:25,649 DEBUG
org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for
region Test,serv3-1315575650141,1316682342728.b3fe66994b0f2fb0643c88f6897474a1.
2011-09-23 14:21:25,650 DEBUG
org.apache.hadoop.hbase.io.hfile.HFileReaderV2: On close of file
15761056997240226 evicted 344 block(s)
2011-09-23 14:21:25,650 DEBUG
org.apache.hadoop.hbase.regionserver.Store: closed data
2011-09-23 14:21:25,650 INFO
org.apache.hadoop.hbase.regionserver.HRegion: Closed
Test,serv3-1315575650141,1316682342728.b3fe66994b0f2fb0643c88f6897474a1.
2011-09-23 14:21:25,650 DEBUG
org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler:
Closed region Test,serv3-1315575650141,1316682342728.b3fe66994b0f2fb0643c88f6897474a1.
2011-09-23 14:21:25,651 WARN org.apache.hadoop.ipc.HBaseServer: IPC
Server Responder, call execCoprocessor([B@6b6ab732,
getAvg(org.apache.hadoop.hbase.client.coprocessor.LongColumnInterpreter@2b216ab6,
{"timeRange":[0,9223372036854775807],"batch":-1,"startRow":"serv1-1315571539741","stopRow":"serv5-1315575834916","totalColumns":0,"cacheBlocks":true,"families":{"data":[]},"maxVersions":1,"caching":-1}))
from 137.72.240.177:47563: output error
2011-09-23 14:21:25,651 WARN org.apache.hadoop.ipc.HBaseServer: IPC
Server handler 9 on 60020 caught:
java.nio.channels.ClosedChannelException
        at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
        at org.apache.hadoop.hbase.ipc.HBaseServer.channelWrite(HBaseServer.java:1501)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.processResponse(HBaseServer.java:876)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Responder.doRespond(HBaseServer.java:955)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Call.sendResponseIfReady(HBaseServer.java:390)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1240)

And the following in the zookeeper log:
2011-09-23 14:19:50,978 INFO
org.apache.zookeeper.server.NIOServerCnxn: Established session
0x1329577cad70004 with negotiated timeout 180000 for client
/137.72.240.177:45431
2011-09-23 14:21:11,028 WARN
org.apache.zookeeper.server.NIOServerCnxn: EndOfStreamException:
Unable to read additional data from client sessionid
0x1329577cad70004, likely client has closed socket
2011-09-23 14:21:11,029 INFO
org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection
for client /137.72.240.177:45431 which had sessionid 0x1329577cad70004
2011-09-23 14:24:42,000 INFO
org.apache.zookeeper.server.ZooKeeperServer: Expiring session
0x1329577cad70004, timeout of 180000ms exceeded
2011-09-23 14:24:42,001 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Processed session
termination for sessionid: 0x1329577cad70004

Any ideas on how to resolve this?

Thanks in advance.

Regards,
Mayuresh

Mime
View raw message