Return-Path: X-Original-To: apmail-hama-user-archive@www.apache.org Delivered-To: apmail-hama-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 13019E2D1 for ; Wed, 26 Dec 2012 20:11:57 +0000 (UTC) Received: (qmail 56055 invoked by uid 500); 26 Dec 2012 20:11:56 -0000 Delivered-To: apmail-hama-user-archive@hama.apache.org Received: (qmail 56023 invoked by uid 500); 26 Dec 2012 20:11:56 -0000 Mailing-List: contact user-help@hama.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hama.apache.org Delivered-To: mailing list user@hama.apache.org Received: (qmail 56009 invoked by uid 99); 26 Dec 2012 20:11:56 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Dec 2012 20:11:56 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of sanchesbanhos@gmail.com designates 209.85.212.54 as permitted sender) Received: from [209.85.212.54] (HELO mail-vb0-f54.google.com) (209.85.212.54) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Dec 2012 20:11:51 +0000 Received: by mail-vb0-f54.google.com with SMTP id l1so9171489vba.27 for ; Wed, 26 Dec 2012 12:11:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=n7trHbce4372IVRs2U1BeSYesGoyQE4hLBt3PounAl4=; b=P49VvLIogrR7EoqBk58cdKeNjpFAQOZCg4ym1H+MtY62n+fUntEz+p5AY6DKlGe4be cEJCnqNR5kNryKvzY9dyu+y+nyEveeba+lehlFpv/uEsYd/wtBCJSMhzmTT+P8c69Kao Qu6Ojwue1KUb9hK/GHdMUalGtt0gd7ewvojffQb89LWPZL66NxFSYQAIkvSa6I16js8S l8XrbkBjdCHTc7P7GdVhjcwTTUPh0ELTjV7IbyBtC4HhYbaSPHIOM20aAqzQ08C3ucnl /zawYYfIPfOsmDBA7Zg3QPT2Ep26httGvqRb4DXXcsOd6UdGqPLuxSro+Zkk955Otpgc IOgQ== MIME-Version: 1.0 Received: by 10.52.175.163 with SMTP id cb3mr37674291vdc.76.1356552690084; Wed, 26 Dec 2012 12:11:30 -0800 (PST) Received: by 10.52.107.129 with HTTP; Wed, 26 Dec 2012 12:11:29 -0800 (PST) In-Reply-To: References: Date: Wed, 26 Dec 2012 17:11:29 -0300 Message-ID: Subject: Re: What is the best configuration for Cluster hama distributed mode From: Francisco Sanches To: user@hama.apache.org Content-Type: multipart/alternative; boundary=bcaec517a9c2362cc304d1c70976 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec517a9c2362cc304d1c70976 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks for the replies: Excuse me by the size of the log. But after I changed the parameter of hama-env.sh to "export HAMA_HEAPSIZE =3D 2000" My program stopped giving th= e memory error and displays the error below. Remembering that I am running for a graph with just over 65 000 vertices. What is it like to understand is that if I increase the number of nodes in the cluster can run my application with this graph of 65 000 vertices? LOG: hduser@mestre:/usr/local/hama$ bin/hama jar aplicpronto.jar 65537 /hdfs/disco/grafogrande /hdfs/disco/grafoout 36 12/12/26 17:00:19 INFO bsp.FileInputFormat: Total input paths to process : = 1 12/12/26 17:00:19 INFO bsp.FileInputFormat: Total # of splits: 36 12/12/26 17:00:23 INFO bsp.BSPJobClient: Running job: job_201212261658_0001 12/12/26 17:00:26 INFO bsp.BSPJobClient: Current supersteps number: 0 12/12/26 17:00:56 INFO bsp.BSPJobClient: Current supersteps number: 1 attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3D3.3.3-1073969, built on 02/23/2011 22:27 GMT attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:host.name=3Dmestre attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.version=3D1.6.0_24 attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.vendor=3DSun Microsystems Inc. attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.home=3D/usr/lib/jvm/java-6-openjdk-i386/jre attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.class.path=3D/usr/local/hama/bin/../conf:/usr/lib/jvm/java= -6-openjdk-i386/lib/tools.jar:/usr/local/hama/bin/../hama-core-0.5.0.jar:/u= sr/local/hama/bin/../hama-examples-0.5.0.jar:/usr/local/hama/bin/../hama-gr= aph-0.5.0.jar:/usr/local/hama/bin/../lib/ant-1.7.1.jar:/usr/local/hama/bin/= ../lib/ant-launcher-1.7.1.jar:/usr/local/hama/bin/../lib/avro-1.6.0.jar:/us= r/local/hama/bin/../lib/avro-ipc-1.6.0.jar:/usr/local/hama/bin/../lib/commo= ns-cli-1.2.jar:/usr/local/hama/bin/../lib/commons-configuration-1.7.jar:/us= r/local/hama/bin/../lib/commons-lang-2.6.jar:/usr/local/hama/bin/../lib/com= mons-logging-1.1.1.jar:/usr/local/hama/bin/../lib/commons-math3-3.0.jar:/us= r/local/hama/bin/../lib/guava-10.0.1.jar:/usr/local/hama/bin/../lib/hadoop-= core-1.0.0.jar:/usr/local/hama/bin/../lib/hadoop-test-1.0.0.jar:/usr/local/= hama/bin/../lib/jackson-core-asl-1.9.2.jar:/usr/local/hama/bin/../lib/jacks= on-mapper-asl-1.9.2.jar:/usr/local/hama/bin/../lib/jetty-6.1.14.jar:/usr/lo= cal/hama/bin/../lib/jetty-annotations-6.1.14.jar:/usr/local/hama/bin/../lib= /jetty-util-6.1.14.jar:/usr/local/hama/bin/../lib/jsp-2.1-6.1.14.jar:/usr/l= ocal/hama/bin/../lib/jsp-api-2.1-6.1.14.jar:/usr/local/hama/bin/../lib/juni= t-4.8.1.jar:/usr/local/hama/bin/../lib/log4j-1.2.16.jar:/usr/local/hama/bin= /../lib/netty-3.2.6.Final.jar:/usr/local/hama/bin/../lib/servlet-api-6.0.32= .jar:/usr/local/hama/bin/../lib/slf4j-api-1.5.8.jar:/usr/local/hama/bin/../= lib/slf4j-log4j12-1.5.8.jar:/usr/local/hama/bin/../lib/snappy-java-1.0.4.1.= jar:/usr/local/hama/bin/../lib/zookeeper-3.3.3.jar::/hdfs/disco/hama-hduser= /bsp/local/groomServer/attempt_201212261658_0001_000000_0/work/classes:/hdf= s/disco/hama-hduser/bsp/local/groomServer/attempt_201212261658_0001_000000_= 0/work attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.library.path=3D/usr/lib/jvm/java-6-openjdk-i386/jre/lib/i3= 86/server:/usr/lib/jvm/java-6-openjdk-i386/jre/lib/i386:/usr/lib/jvm/java-6= -openjdk-i386/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/i386-lin= ux-gnu/jni:/lib/i386-linux-gnu:/usr/lib/i386-linux-gnu:/usr/lib/jni:/lib:/u= sr/lib attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=3D/tmp attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:java.compiler=3D attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:os.name=3DLinux attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:os.arch=3Di386 attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:os.version=3D3.2.0-32-generic-pae attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:user.name=3Dhduser attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:user.home=3D/home/hduser attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Client environment:user.dir=3D/hdfs/disco/hama-hduser/bsp/local/groomServer/attemp= t_201212261658_0001_000000_0/work attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=3Dnode02:21810,node01:21810,mestre:21810 sessionTimeout=3D120= 0000 watcher=3Dorg.apache.hama.bsp.sync.ZooKeeperSyncClientImpl@1deeb40 attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ClientCnxn: Opening socket connection to server mestre/ 10.16.1.200:21810 attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO zookeeper.ClientCnxn: Socket connection established to mestre/ 10.16.1.200:21810, initiating session attempt_201212261658_0001_000000_0: 12/12/26 17:00:31 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At mestre/ 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:32 INFO zookeeper.ClientCnxn: Session establishment complete on server mestre/ 10.16.1.200:21810, sessionid =3D 0x3bd8cbb48a0006, negotiated timeout =3D 1200000 attempt_201212261658_0001_000000_0: SLF4J: Class path contains multiple SLF4J bindings. attempt_201212261658_0001_000000_0: SLF4J: Found binding in [jar:file:/usr/local/hama/lib/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/Stati= cLoggerBinder.class] attempt_201212261658_0001_000000_0: SLF4J: Found binding in [file:/hdfs/disco/hama-hduser/bsp/local/groomServer/attempt_201212261658_00= 01_000000_0/work/org/slf4j/impl/StaticLoggerBinder.class] attempt_201212261658_0001_000000_0: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node03/10.16.1.203:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x002f8116] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x002f8116, /10.16.1.200:47685 =3D> node03/ 10.16.1.203:61003] BOUND: /10.16.1.200:47685 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x002f8116, /10.16.1.200:47685 =3D> node03/ 10.16.1.203:61003] CONNECTED: node03/10.16.1.203:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node05/10.16.1.205:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00d8ca48] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00d8ca48, /10.16.1.200:52329 =3D> node05/ 10.16.1.205:61001] BOUND: /10.16.1.200:52329 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00d8ca48, /10.16.1.200:52329 =3D> node05/ 10.16.1.205:61001] CONNECTED: node05/10.16.1.205:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node04/10.16.1.204:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x018adae2] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x018adae2, /10.16.1.200:59179 =3D> node04/ 10.16.1.204:61002] BOUND: /10.16.1.200:59179 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x018adae2, /10.16.1.200:59179 =3D> node04/ 10.16.1.204:61002] CONNECTED: node04/10.16.1.204:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node06/10.16.1.206:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x003a835d] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x003a835d, /10.16.1.200:45011 =3D> node06/ 10.16.1.206:61001] BOUND: /10.16.1.200:45011 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x003a835d, /10.16.1.200:45011 =3D> node06/ 10.16.1.206:61001] CONNECTED: node06/10.16.1.206:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node04/10.16.1.204:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00061cd2] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00061cd2, /10.16.1.200:54578 =3D> node04/ 10.16.1.204:61003] BOUND: /10.16.1.200:54578 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00061cd2, /10.16.1.200:54578 =3D> node04/ 10.16.1.204:61003] CONNECTED: node04/10.16.1.204:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node05/10.16.1.205:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00fb6c5f] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00fb6c5f, /10.16.1.200:52964 =3D> node05/ 10.16.1.205:61002] BOUND: /10.16.1.200:52964 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00fb6c5f, /10.16.1.200:52964 =3D> node05/ 10.16.1.205:61002] CONNECTED: node05/10.16.1.205:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node03/10.16.1.203:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00cb42cf] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00cb42cf, /10.16.1.200:36765 =3D> node03/ 10.16.1.203:61001] BOUND: /10.16.1.200:36765 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00cb42cf, /10.16.1.200:36765 =3D> node03/ 10.16.1.203:61001] CONNECTED: node03/10.16.1.203:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node01/10.16.1.201:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x018bbf55] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x018bbf55, /10.16.1.200:45734 =3D> node01/ 10.16.1.201:61003] BOUND: /10.16.1.200:45734 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x018bbf55, /10.16.1.200:45734 =3D> node01/ 10.16.1.201:61003] CONNECTED: node01/10.16.1.201:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node02/10.16.1.202:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00ed5d9d] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00ed5d9d, /10.16.1.200:60724 =3D> node02/ 10.16.1.202:61002] BOUND: /10.16.1.200:60724 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00ed5d9d, /10.16.1.200:60724 =3D> node02/ 10.16.1.202:61002] CONNECTED: node02/10.16.1.202:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node03/10.16.1.203:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x014b84c7] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x014b84c7, /10.16.1.200:45839 =3D> node03/ 10.16.1.203:61002] BOUND: /10.16.1.200:45839 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x014b84c7, /10.16.1.200:45839 =3D> node03/ 10.16.1.203:61002] CONNECTED: node03/10.16.1.203:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node02/10.16.1.202:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x0160c4b0] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x0160c4b0, /10.16.1.200:44757 =3D> node02/ 10.16.1.202:61003] BOUND: /10.16.1.200:44757 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x0160c4b0, /10.16.1.200:44757 =3D> node02/ 10.16.1.202:61003] CONNECTED: node02/10.16.1.202:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node04/10.16.1.204:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x0029ea31] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x0029ea31, /10.16.1.200:40846 =3D> node04/ 10.16.1.204:61001] BOUND: /10.16.1.200:40846 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x0029ea31, /10.16.1.200:40846 =3D> node04/ 10.16.1.204:61001] CONNECTED: node04/10.16.1.204:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00158cc6, /10.16.1.210:36396 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00158cc6, /10.16.1.210:36396 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00158cc6, /10.16.1.210:36396 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.210:36396 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x016f5261, /10.16.1.200:41941 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x016f5261, /10.16.1.200:41941 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x016f5261, /10.16.1.200:41941 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.200:41941 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x008ddb93, /10.16.1.206:44493 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x008ddb93, /10.16.1.206:44493 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x008ddb93, /10.16.1.206:44493 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.206:44493 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01a9f87c, /10.16.1.202:48243 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01a9f87c, /10.16.1.202:48243 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01a9f87c, /10.16.1.202:48243 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.202:48243 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01f217ec, /10.16.1.200:41945 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01f217ec, /10.16.1.200:41945 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01f217ec, /10.16.1.200:41945 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.200:41945 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node01/10.16.1.201:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00a86d12] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00a86d12, /10.16.1.200:35763 =3D> node01/ 10.16.1.201:61001] BOUND: /10.16.1.200:35763 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00a86d12, /10.16.1.200:35763 =3D> node01/ 10.16.1.201:61001] CONNECTED: node01/10.16.1.201:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to mestre/10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x008932e8] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x016d3536, /10.16.1.200:41949 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x016d3536, /10.16.1.200:41949 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x016d3536, /10.16.1.200:41949 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.200:41949 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x008932e8, /10.16.1.200:41949 =3D> mestre/ 10.16.1.200:61002] BOUND: /10.16.1.200:41949 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x008932e8, /10.16.1.200:41949 =3D> mestre/ 10.16.1.200:61002] CONNECTED: mestre/10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: Connecting to node01/10.16.1.201:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00f2164f] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01aa0a15, /10.16.1.209:43193 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01aa0a15, /10.16.1.209:43193 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01aa0a15, /10.16.1.209:43193 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.209:43193 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00f2164f, /10.16.1.200:44908 =3D> node01/ 10.16.1.201:61002] BOUND: /10.16.1.200:44908 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyTransceiver: [id: 0x00f2164f, /10.16.1.200:44908 =3D> node01/ 10.16.1.201:61002] CONNECTED: node01/10.16.1.201:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01cf662f, /10.16.1.207:57476 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01cf662f, /10.16.1.207:57476 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x003eec1a, /10.16.1.207:57478 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01cf662f, /10.16.1.207:57476 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.207:57476 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01115152, /10.16.1.201:37455 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x003eec1a, /10.16.1.207:57478 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x003eec1a, /10.16.1.207:57478 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.207:57478 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01de6817, /10.16.1.210:36402 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01de6817, /10.16.1.210:36402 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01de6817, /10.16.1.210:36402 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.210:36402 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01115152, /10.16.1.201:37455 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01115152, /10.16.1.201:37455 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.201:37455 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00a56214, /10.16.1.207:57480 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00a56214, /10.16.1.207:57480 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00a56214, /10.16.1.207:57480 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.207:57480 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x004bb369, /10.16.1.205:56188 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x004bb369, /10.16.1.205:56188 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x004bb369, /10.16.1.205:56188 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.205:56188 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01abdac9, /10.16.1.206:44506 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01abdac9, /10.16.1.206:44506 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01abdac9, /10.16.1.206:44506 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.206:44506 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01f68272, /10.16.1.205:56189 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01f68272, /10.16.1.205:56189 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01f68272, /10.16.1.205:56189 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.205:56189 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x0103c29b, /10.16.1.202:48251 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x0103c29b, /10.16.1.202:48251 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x0103c29b, /10.16.1.202:48251 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.202:48251 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01e8c585, /10.16.1.205:56191 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01e8c585, /10.16.1.205:56191 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01e8c585, /10.16.1.205:56191 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.205:56191 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00ab835a, /10.16.1.208:50917 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x003cbb4b, /10.16.1.208:50918 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x003cbb4b, /10.16.1.208:50918 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x003cbb4b, /10.16.1.208:50918 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.208:50918 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00ab835a, /10.16.1.208:50917 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x00ab835a, /10.16.1.208:50917 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.208:50917 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01443800, /10.16.1.209:43200 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01443800, /10.16.1.209:43200 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x01443800, /10.16.1.209:43200 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.209:43200 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x011ff451, /10.16.1.209:43201 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x011ff451, /10.16.1.209:43201 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:51 INFO ipc.NettyServer: [id: 0x011ff451, /10.16.1.209:43201 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.209:43201 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01a73d30, /10.16.1.208:50924 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01a73d30, /10.16.1.208:50924 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01a73d30, /10.16.1.208:50924 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.208:50924 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01108727, /10.16.1.211:44099 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01108727, /10.16.1.211:44099 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01108727, /10.16.1.211:44099 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.211:44099 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x011415c8, /10.16.1.211:44100 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01a220e6, /10.16.1.206:44511 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01a220e6, /10.16.1.206:44511 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01a220e6, /10.16.1.206:44511 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.206:44511 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x011415c8, /10.16.1.211:44100 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x011415c8, /10.16.1.211:44100 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.211:44100 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x012f41a5, /10.16.1.211:44101 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x012f41a5, /10.16.1.211:44101 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x012f41a5, /10.16.1.211:44101 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.211:44101 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node02/10.16.1.202:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x019c4091] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x000e7d53, /10.16.1.201:37460 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x000e7d53, /10.16.1.201:37460 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x000e7d53, /10.16.1.201:37460 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.201:37460 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x019c4091, /10.16.1.200:54333 =3D> node02/ 10.16.1.202:61001] BOUND: /10.16.1.200:54333 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x019c4091, /10.16.1.200:54333 =3D> node02/ 10.16.1.202:61001] CONNECTED: node02/10.16.1.202:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x0053abbb, /10.16.1.203:46568 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x0053abbb, /10.16.1.203:46568 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x0053abbb, /10.16.1.203:46568 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.203:46568 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x009db992, /10.16.1.203:46569 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x009db992, /10.16.1.203:46569 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x009db992, /10.16.1.203:46569 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.203:46569 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to mestre/10.16.1.200:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e80761] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e80761, /10.16.1.200:56327 =3D> mestre/ 10.16.1.200:61003] BOUND: /10.16.1.200:56327 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e80761, /10.16.1.200:56327 =3D> mestre/ 10.16.1.200:61003] CONNECTED: mestre/10.16.1.200:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to mestre/10.16.1.200:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01dbb27d] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x0093c4f1, /10.16.1.203:46573 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x0093c4f1, /10.16.1.203:46573 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x0093c4f1, /10.16.1.203:46573 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.203:46573 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01dbb27d, /10.16.1.200:51740 =3D> mestre/ 10.16.1.200:61001] BOUND: /10.16.1.200:51740 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01dbb27d, /10.16.1.200:51740 =3D> mestre/ 10.16.1.200:61001] CONNECTED: mestre/10.16.1.200:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00b9b618, /10.16.1.204:35237 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00b9b618, /10.16.1.204:35237 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00b9b618, /10.16.1.204:35237 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.204:35237 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00dfc362, /10.16.1.204:35239 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00dfc362, /10.16.1.204:35239 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00dfc362, /10.16.1.204:35239 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.204:35239 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node11/10.16.1.211:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01971eb3] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01408a75, /10.16.1.202:48260 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01408a75, /10.16.1.202:48260 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x01408a75, /10.16.1.202:48260 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.202:48260 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01971eb3, /10.16.1.200:57399 =3D> node11/ 10.16.1.211:61003] BOUND: /10.16.1.200:57399 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01971eb3, /10.16.1.200:57399 =3D> node11/ 10.16.1.211:61003] CONNECTED: node11/10.16.1.211:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00c713d2, /10.16.1.201:37463 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00c713d2, /10.16.1.201:37463 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00c713d2, /10.16.1.201:37463 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.201:37463 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node11/10.16.1.211:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x005b78cf] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x005b78cf, /10.16.1.200:41092 =3D> node11/ 10.16.1.211:61001] BOUND: /10.16.1.200:41092 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x005b78cf, /10.16.1.200:41092 =3D> node11/ 10.16.1.211:61001] CONNECTED: node11/10.16.1.211:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00c28cb7, /10.16.1.204:35245 =3D> /10.16.1.200:61002] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00c28cb7, /10.16.1.204:35245 =3D> /10.16.1.200:61002] BOUND: / 10.16.1.200:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyServer: [id: 0x00c28cb7, /10.16.1.204:35245 =3D> /10.16.1.200:61002] CONNECTED: / 10.16.1.204:35245 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node10/10.16.1.210:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01f51e5c] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01f51e5c, /10.16.1.200:48018 =3D> node10/ 10.16.1.210:61002] BOUND: /10.16.1.200:48018 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01f51e5c, /10.16.1.200:48018 =3D> node10/ 10.16.1.210:61002] CONNECTED: node10/10.16.1.210:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node09/10.16.1.209:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00d36ff3] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00d36ff3, /10.16.1.200:34916 =3D> node09/ 10.16.1.209:61003] BOUND: /10.16.1.200:34916 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00d36ff3, /10.16.1.200:34916 =3D> node09/ 10.16.1.209:61003] CONNECTED: node09/10.16.1.209:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node11/10.16.1.211:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x018e899c] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x018e899c, /10.16.1.200:58261 =3D> node11/ 10.16.1.211:61002] BOUND: /10.16.1.200:58261 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x018e899c, /10.16.1.200:58261 =3D> node11/ 10.16.1.211:61002] CONNECTED: node11/10.16.1.211:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node10/10.16.1.210:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x014bc4e6] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x014bc4e6, /10.16.1.200:39073 =3D> node10/ 10.16.1.210:61003] BOUND: /10.16.1.200:39073 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x014bc4e6, /10.16.1.200:39073 =3D> node10/ 10.16.1.210:61003] CONNECTED: node10/10.16.1.210:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node08/10.16.1.208:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01be1041] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01be1041, /10.16.1.200:39758 =3D> node08/ 10.16.1.208:61002] BOUND: /10.16.1.200:39758 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01be1041, /10.16.1.200:39758 =3D> node08/ 10.16.1.208:61002] CONNECTED: node08/10.16.1.208:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node09/10.16.1.209:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e090ee] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e090ee, /10.16.1.200:56029 =3D> node09/ 10.16.1.209:61001] BOUND: /10.16.1.200:56029 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e090ee, /10.16.1.200:56029 =3D> node09/ 10.16.1.209:61001] CONNECTED: node09/10.16.1.209:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node07/10.16.1.207:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x0187f9f1] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x0187f9f1, /10.16.1.200:35151 =3D> node07/ 10.16.1.207:61003] BOUND: /10.16.1.200:35151 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x0187f9f1, /10.16.1.200:35151 =3D> node07/ 10.16.1.207:61003] CONNECTED: node07/10.16.1.207:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node08/10.16.1.208:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00c3358b] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00c3358b, /10.16.1.200:35301 =3D> node08/ 10.16.1.208:61003] BOUND: /10.16.1.200:35301 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00c3358b, /10.16.1.200:35301 =3D> node08/ 10.16.1.208:61003] CONNECTED: node08/10.16.1.208:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node10/10.16.1.210:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e82b72] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e82b72, /10.16.1.200:34819 =3D> node10/ 10.16.1.210:61001] BOUND: /10.16.1.200:34819 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01e82b72, /10.16.1.200:34819 =3D> node10/ 10.16.1.210:61001] CONNECTED: node10/10.16.1.210:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node09/10.16.1.209:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00d0deb5] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00d0deb5, /10.16.1.200:34684 =3D> node09/ 10.16.1.209:61002] BOUND: /10.16.1.200:34684 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00d0deb5, /10.16.1.200:34684 =3D> node09/ 10.16.1.209:61002] CONNECTED: node09/10.16.1.209:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node06/10.16.1.206:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01505258] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01505258, /10.16.1.200:35962 =3D> node06/ 10.16.1.206:61002] BOUND: /10.16.1.200:35962 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01505258, /10.16.1.200:35962 =3D> node06/ 10.16.1.206:61002] CONNECTED: node06/10.16.1.206:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node07/10.16.1.207:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01d8417c] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01d8417c, /10.16.1.200:41009 =3D> node07/ 10.16.1.207:61001] BOUND: /10.16.1.200:41009 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01d8417c, /10.16.1.200:41009 =3D> node07/ 10.16.1.207:61001] CONNECTED: node07/10.16.1.207:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node05/10.16.1.205:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x017c398e] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x017c398e, /10.16.1.200:53654 =3D> node05/ 10.16.1.205:61003] BOUND: /10.16.1.200:53654 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x017c398e, /10.16.1.200:53654 =3D> node05/ 10.16.1.205:61003] CONNECTED: node05/10.16.1.205:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node08/10.16.1.208:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00e13e7b] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00e13e7b, /10.16.1.200:55138 =3D> node08/ 10.16.1.208:61001] BOUND: /10.16.1.200:55138 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x00e13e7b, /10.16.1.200:55138 =3D> node08/ 10.16.1.208:61001] CONNECTED: node08/10.16.1.208:61001 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node06/10.16.1.206:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01503458] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01503458, /10.16.1.200:40540 =3D> node06/ 10.16.1.206:61003] BOUND: /10.16.1.200:40540 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01503458, /10.16.1.200:40540 =3D> node06/ 10.16.1.206:61003] CONNECTED: node06/10.16.1.206:61003 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: Connecting to node07/10.16.1.207:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01b963c4] OPEN attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01b963c4, /10.16.1.200:42816 =3D> node07/ 10.16.1.207:61002] BOUND: /10.16.1.200:42816 attempt_201212261658_0001_000000_0: 12/12/26 17:00:52 INFO ipc.NettyTransceiver: [id: 0x01b963c4, /10.16.1.200:42816 =3D> node07/ 10.16.1.207:61002] CONNECTED: node07/10.16.1.207:61002 attempt_201212261658_0001_000000_0: 12/12/26 17:00:53 INFO graph.GraphJobRunner: Loading finished at 1 steps. 12/12/26 17:03:17 INFO bsp.BSPJobClient: Job failed. 2012/12/26 Edward J. Yoon > BTW, This log looks like you ran out of memory while job was running > as a local mode. > > You can check the cluster status via web UI. > > 1. http://wiki.apache.org/hama/GettingStarted#Hama_Web_Interfaces > > On Wed, Dec 26, 2012 at 10:00 PM, Francisco Sanches > wrote: > > attempt_201212260904_0005_000031_0: Exception in thread "Thread-1" > > java.lang.OutOfMemoryError: GC overhead limit exceeded > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > --=20 Francisco Sanches --bcaec517a9c2362cc304d1c70976--