incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HAMA-507) java.io.IOException: Connection reset by peer
Date Mon, 06 Feb 2012 01:19:59 GMT
java.io.IOException: Connection reset by peer
---------------------------------------------

                 Key: HAMA-507
                 URL: https://issues.apache.org/jira/browse/HAMA-507
             Project: Hama
          Issue Type: Improvement
          Components: bsp
    Affects Versions: 0.4.0
            Reporter: Edward J. Yoon
            Priority: Minor
             Fix For: 0.5.0


Occasionally, hadoop.ipc.Server throws exception when tasks are finished. like below.

IMO, it doesn't affect system stability at all, but it'd be nice if we can prevent that exception.

{code}
2012-02-05 18:58:08,870 INFO org.apache.hama.GroomServerRunner: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting GroomServer
STARTUP_MSG:   host = server2/192.168.0.110
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build =  -r ; compiled by 'chrisdo' on Wed Mar 10 13:22:54 PST 2010
************************************************************/
2012-02-05 18:58:08,936 INFO org.apache.hama.bsp.GroomServer: groom start
2012-02-05 18:58:08,984 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.3-1073969,
built on 02/23/2011 22:27 GMT
2012-02-05 18:58:08,984 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=server2.abcfood.org
2012-02-05 18:58:08,984 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_02
2012-02-05 18:58:08,984 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle
Corporation
2012-02-05 18:58:08,984 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/home/userA/jdk1.7.0_02/jre
2012-02-05 18:58:08,984 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/home/userA/hama-0.4.0-incubating/bin/../conf:/home/userA/jdk1.7.0_02/lib/tools.jar:/home/userA/hama-0.4.0-incubating/bin/../hama-core-0.4.0-incubating.jar:/home/userA/hama-0.4.0-incubating/bin/../hama-examples-0.4.0-incubating.jar:/home/userA/hama-0.4.0-incubating/bin/../hama-graph-0.4.0-incubating.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/ant-1.7.1.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/ant-launcher-1.7.1.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/commons-cli-1.2.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/commons-logging-1.1.1.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/hadoop-core-0.20.2.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/hadoop-test-0.20.2.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/jetty-6.1.14.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/jetty-annotations-6.1.14.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/jetty-util-6.1.14.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/jsp-2.1-6.1.14.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/junit-4.8.1.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/log4j-1.2.16.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/servlet-api-6.0.32.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/slf4j-api-1.5.8.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/slf4j-log4j12-1.5.8.jar:/home/userA/hama-0.4.0-incubating/bin/../lib/zookeeper-3.3.3.jar
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/i386:/lib:/usr/lib
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=i386
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-5-686
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=userA
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/userA
2012-02-05 18:58:08,985 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/userA/hama-0.4.0-incubating
2012-02-05 18:58:08,990 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection,
connectString=server4:21810 sessionTimeout=1200000 watcher=org.apache.hama.bsp.GroomServer@196978f
2012-02-05 18:58:09,020 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to
server server4/192.168.0.199:21810
2012-02-05 18:58:09,025 INFO org.apache.hama.bsp.GroomServer: /tmp/hadoop-userA/bsp/local
2012-02-05 18:58:09,040 INFO org.apache.zookeeper.ClientCnxn: Socket connection established
to server4/192.168.0.199:21810, initiating session
2012-02-05 18:58:09,141 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data
from server sessionid 0x0, likely server has closed socket, closing socket connection and
attempting reconnect
2012-02-05 18:58:09,174 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
with hostName=GroomServer, port=50000
2012-02-05 18:58:09,266 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2012-02-05 18:58:09,266 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50000: starting
2012-02-05 18:58:09,268 INFO org.apache.hama.bsp.GroomServer: Worker rpc server --> server2.abcfood.org:50000
2012-02-05 18:58:09,268 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 50000:
starting
2012-02-05 18:58:09,269 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
with hostName=GroomServer, port=53084
2012-02-05 18:58:09,273 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 53084: starting
2012-02-05 18:58:09,273 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2012-02-05 18:58:09,274 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 53084:
starting
2012-02-05 18:58:09,280 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 53084:
starting
2012-02-05 18:58:09,282 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 53084:
starting
2012-02-05 18:58:09,293 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 53084:
starting
2012-02-05 18:58:09,302 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 53084:
starting
2012-02-05 18:58:09,308 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 53084:
starting
2012-02-05 18:58:09,313 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 53084:
starting
2012-02-05 18:58:09,315 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 53084:
starting
2012-02-05 18:58:09,316 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 53084:
starting
2012-02-05 18:58:09,316 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 53084:
starting
2012-02-05 18:58:09,317 INFO org.apache.hama.bsp.GroomServer: GroomServer up at: localhost/127.0.0.1:53084
2012-02-05 18:58:09,317 INFO org.apache.hama.bsp.GroomServer: Starting groom: server2.abcfood.org:50000
2012-02-05 18:58:10,956 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to
server server4/192.168.0.199:21810
2012-02-05 18:58:10,957 INFO org.apache.zookeeper.ClientCnxn: Socket connection established
to server4/192.168.0.199:21810, initiating session
2012-02-05 18:58:10,966 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete
on server server4/192.168.0.199:21810, sessionid = 0x1354d29f1b30001, negotiated timeout =
1200000
2012-02-05 19:02:31,046 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:02:31,416 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0001_000001_0'
has started.
2012-02-05 19:02:31,416 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:02:32,210 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0001_000003_0'
has started.
2012-02-05 19:02:32,211 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:02:32,648 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0001_000005_0'
has started.
2012-02-05 19:02:36,282 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0001_000005_0
is done.
2012-02-05 19:02:36,478 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0001_000001_0
is done.
2012-02-05 19:02:36,514 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 53084: readAndProcess
threw exception java.io.IOException: Connection reset by peer. Count of bytes read: 0
java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
	at sun.nio.ch.IOUtil.read(IOUtil.java:191)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
	at org.apache.hadoop.ipc.Server.channelRead(Server.java:1211)
	at org.apache.hadoop.ipc.Server.access$2300(Server.java:77)
	at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:799)
	at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:419)
	at org.apache.hadoop.ipc.Server$Listener.run(Server.java:328)
2012-02-05 19:02:36,653 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0001_000003_0
is done.
2012-02-05 19:25:42,980 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:25:43,438 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0002_000000_0'
has started.
2012-02-05 19:25:43,439 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:25:44,056 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0002_000001_0'
has started.
2012-02-05 19:25:44,056 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:25:44,675 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0002_000002_0'
has started.
2012-02-05 19:25:50,170 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0002_000001_0
is done.
2012-02-05 19:25:50,372 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0002_000000_0
is done.
2012-02-05 19:25:50,377 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0002_000002_0
is done.
2012-02-05 19:26:03,539 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:26:03,710 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0003_000003_0'
has started.
2012-02-05 19:26:03,711 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:26:03,963 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0003_000004_0'
has started.
2012-02-05 19:26:03,964 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:26:04,170 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0003_000005_0'
has started.
2012-02-05 19:26:07,990 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0003_000005_0
is done.
2012-02-05 19:26:08,018 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0003_000004_0
is done.
2012-02-05 19:26:08,062 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0003_000003_0
is done.
2012-02-05 19:28:11,618 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:28:11,769 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0004_000003_0'
has started.
2012-02-05 19:28:11,782 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:28:12,258 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0004_000004_0'
has started.
2012-02-05 19:28:12,259 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 19:28:12,612 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0004_000005_0'
has started.
2012-02-05 19:28:15,971 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0004_000004_0
is done.
2012-02-05 19:28:16,023 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0004_000005_0
is done.
2012-02-05 19:28:16,143 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0004_000003_0
is done.
2012-02-05 20:54:55,866 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 20:54:56,017 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0005_000003_0'
has started.
2012-02-05 20:54:56,017 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 20:54:56,326 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0005_000004_0'
has started.
2012-02-05 20:54:56,327 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 20:54:56,655 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0005_000005_0'
has started.
2012-02-05 20:54:59,962 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0005_000004_0
is done.
2012-02-05 20:55:00,049 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0005_000005_0
is done.
2012-02-05 20:55:00,102 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0005_000003_0
is done.
2012-02-05 20:55:08,697 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 20:55:08,860 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0006_000001_0'
has started.
2012-02-05 20:55:08,865 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 20:55:09,185 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0006_000003_0'
has started.
2012-02-05 20:55:09,186 INFO org.apache.hama.bsp.GroomServer: Launch 3 tasks.
2012-02-05 20:55:09,497 INFO org.apache.hama.bsp.GroomServer: Task 'attempt_201202051858_0006_000005_0'
has started.
2012-02-05 20:55:13,539 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0006_000001_0
is done.
2012-02-05 20:55:13,582 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0006_000005_0
is done.
2012-02-05 20:55:13,620 INFO org.apache.hama.bsp.GroomServer: Task attempt_201202051858_0006_000003_0
is done.
2012-02-05 20:55:13,623 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 53084: readAndProcess
threw exception java.io.IOException: Connection reset by peer. Count of bytes read: 0
java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
	at sun.nio.ch.IOUtil.read(IOUtil.java:191)
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
	at org.apache.hadoop.ipc.Server.channelRead(Server.java:1211)
	at org.apache.hadoop.ipc.Server.access$2300(Server.java:77)
	at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:799)
	at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:419)
	at org.apache.hadoop.ipc.Server$Listener.run(Server.java:328)
2012-02-05 20:56:26,474 INFO org.apache.hama.GroomServerRunner: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down GroomServer at server2/192.168.0.110
************************************************************/
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message