Return-Path: X-Original-To: apmail-incubator-hama-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-hama-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 51EAC7D1D for ; Thu, 3 Nov 2011 09:36:23 +0000 (UTC) Received: (qmail 50590 invoked by uid 500); 3 Nov 2011 09:36:23 -0000 Delivered-To: apmail-incubator-hama-dev-archive@incubator.apache.org Received: (qmail 50561 invoked by uid 500); 3 Nov 2011 09:36:23 -0000 Mailing-List: contact hama-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hama-dev@incubator.apache.org Delivered-To: mailing list hama-dev@incubator.apache.org Received: (qmail 50552 invoked by uid 99); 3 Nov 2011 09:36:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2011 09:36:23 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tommaso.teofili@gmail.com designates 209.85.161.175 as permitted sender) Received: from [209.85.161.175] (HELO mail-gx0-f175.google.com) (209.85.161.175) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2011 09:36:15 +0000 Received: by ggnj1 with SMTP id j1so1061179ggn.6 for ; Thu, 03 Nov 2011 02:35:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=cqe2L2UDxOqJPwSeXMh236po3xXqXLnC2D6+EHTlo9A=; b=jXSqbf5fX9GLfoVzI1kyrRLyOlHGWPUUBLWR1giLhKn4NgNCJhXCSsbGi9+507MFHP 7bJNrDQDD/RDo3xitEYJvIElvs0E2pS+/0H3pLQk4y5UqepWIm3xlIb1vUePUoAqP4Td /ib8+knrnvr75sE4UtEyaIMLm1EDMUKG+fKZ4= Received: by 10.236.116.9 with SMTP id f9mr12276479yhh.0.1320312954216; Thu, 03 Nov 2011 02:35:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.236.102.169 with HTTP; Thu, 3 Nov 2011 02:35:09 -0700 (PDT) In-Reply-To: References: From: Tommaso Teofili Date: Thu, 3 Nov 2011 10:35:09 +0100 Message-ID: Subject: Re: Experiences with large hama cluster. To: hama-dev@incubator.apache.org Content-Type: multipart/alternative; boundary=485b397dccfba0df9c04b0d15082 X-Virus-Checked: Checked by ClamAV on apache.org --485b397dccfba0df9c04b0d15082 Content-Type: text/plain; charset=ISO-8859-1 Thanks for sharing Edward, were you also able to debug why that was the case? Tommaso 2011/11/3 Edward J. Yoon > I tested Hama r.1197018 on 100 nodes cluster using 2 tasks per node. > > The all the functions are works well, but task launching and finishing > are *very slow*. > > [root@hnode-3-7 hama-trunk]# core/bin/start-bspd.sh > hnode-3-7: starting zookeeper, logging to > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-zookeeper-hnode-3-7.out > starting bspmaster, logging to > > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-bspmaster-hnode-3-7.out > 2011-11-03 16:29:14.382::INFO: Logging to STDERR via > org.mortbay.log.StdErrLog > 2011-11-03 16:29:14.430::INFO: jetty-6.1.14 > 2011-11-03 16:29:14.684::INFO: Started SelectChannelConnector@hnode-3-7 > :40013 > hnode-3-7: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-7.out > hnode-3-8: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-8.out > hnode-3-10: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-10.out > hnode-3-9: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-9.out > hnode-3-12: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-12.out > hnode-3-13: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-13.out > hnode-3-11: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-11.out > hnode-3-15: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-15.out > hnode-3-16: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-16.out > hnode-3-18: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-18.out > hnode-3-14: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-14.out > hnode-3-17: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-17.out > hnode-3-20: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-20.out > hnode-3-21: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-21.out > hnode-3-19: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-19.out > hnode-3-22: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-22.out > hnode-3-23: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-23.out > hnode-3-24: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-24.out > hnode-3-28: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-28.out > hnode-3-25: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-25.out > hnode-3-27: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-27.out > hnode-3-26: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-26.out > hnode-3-30: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-30.out > hnode-3-29: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-29.out > hnode-3-31: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-31.out > hnode-3-32: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-32.out > hnode-3-33: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-33.out > hnode-3-35: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-35.out > hnode-3-34: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-34.out > hnode-3-37: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-37.out > hnode-3-38: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-38.out > hnode-3-36: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-36.out > hnode-3-39: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-39.out > hnode-3-40: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-40.out > hnode-3-42: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-42.out > hnode-3-43: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-43.out > hnode-3-45: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-45.out > hnode-3-41: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-41.out > hnode-3-44: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-44.out > hnode-3-46: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-46.out > hnode-3-47: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-47.out > hnode-3-49: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-49.out > hnode-3-50: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-50.out > hnode-3-51: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-51.out > hnode-3-48: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-48.out > hnode-3-55: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-55.out > hnode-3-53: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-53.out > hnode-3-52: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-52.out > hnode-3-56: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-56.out > hnode-3-54: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-54.out > hnode-3-60: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-60.out > hnode-3-59: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-59.out > hnode-3-58: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-58.out > hnode-3-57: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-57.out > hnode-3-62: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-62.out > hnode-3-63: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-63.out > hnode-3-64: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-64.out > hnode-3-65: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-65.out > hnode-3-67: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-67.out > hnode-3-70: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-70.out > hnode-3-61: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-61.out > hnode-3-66: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-66.out > hnode-3-68: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-68.out > hnode-3-72: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-72.out > hnode-3-69: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-69.out > hnode-3-71: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-71.out > hnode-3-75: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-75.out > hnode-3-74: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-74.out > hnode-3-73: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-73.out > hnode-3-77: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-77.out > hnode-3-76: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-76.out > hnode-3-79: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-79.out > hnode-3-78: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-78.out > hnode-3-81: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-81.out > hnode-3-80: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-80.out > hnode-3-82: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-82.out > hnode-3-83: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-83.out > hnode-3-85: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-85.out > hnode-3-86: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-86.out > hnode-3-84: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-84.out > hnode-3-87: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-87.out > hnode-3-88: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-88.out > hnode-3-89: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-89.out > hnode-3-90: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-90.out > hnode-3-91: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-91.out > hnode-3-95: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-95.out > hnode-3-93: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-93.out > hnode-3-94: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-94.out > hnode-3-92: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-92.out > hnode-3-96: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-96.out > hnode-3-97: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-97.out > hnode-3-98: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-98.out > hnode-3-99: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-99.out > hnode-3-102: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-102.out > hnode-3-100: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-100.out > hnode-3-104: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-104.out > hnode-3-101: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-101.out > hnode-3-105: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-105.out > hnode-3-106: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-106.out > hnode-3-103: starting groom, logging to > /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-hnode-3-103.out > [root@hnode-3-7 hama-trunk]# ls > CHANGES.txt core DISCLAIMER.txt examples graph lib LICENSE.txt > NOTICE.txt pom.xml README.txt src target yarn > [root@hnode-3-7 hama-trunk]# core/bin/hama jar > examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar pi > 11/11/03 16:36:45 INFO bsp.BSPJobClient: Running job: job_201111031629_0001 > 11/11/03 16:36:48 INFO bsp.BSPJobClient: Current supersteps number: 0 > 11/11/03 16:38:09 INFO bsp.BSPJobClient: Current supersteps number: 1 > 11/11/03 16:38:12 INFO bsp.BSPJobClient: The total number of supersteps: 1 > Estimated value of PI is 3.1420720000000006 > Job Finished in 90.868 seconds > [root@hnode-3-7 hama-trunk]# core/bin/hama jar > examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16 > 1000 30 > 11/11/03 16:40:05 INFO bsp.BSPJobClient: Running job: job_201111031629_0002 > 11/11/03 16:40:08 INFO bsp.BSPJobClient: Current supersteps number: 0 > 11/11/03 16:41:47 INFO bsp.BSPJobClient: Current supersteps number: 1 > 11/11/03 16:41:50 INFO bsp.BSPJobClient: Current supersteps number: 3 > 11/11/03 16:41:53 INFO bsp.BSPJobClient: Current supersteps number: 5 > 11/11/03 16:41:59 INFO bsp.BSPJobClient: Current supersteps number: 7 > 11/11/03 16:42:02 INFO bsp.BSPJobClient: Current supersteps number: 10 > 11/11/03 16:42:05 INFO bsp.BSPJobClient: Current supersteps number: 14 > 11/11/03 16:42:08 INFO bsp.BSPJobClient: Current supersteps number: 19 > 11/11/03 16:42:11 INFO bsp.BSPJobClient: Current supersteps number: 24 > 11/11/03 16:42:14 INFO bsp.BSPJobClient: Current supersteps number: 28 > 11/11/03 16:42:17 INFO bsp.BSPJobClient: Current supersteps number: 30 > 11/11/03 16:42:20 INFO bsp.BSPJobClient: The total number of supersteps: 30 > Job Finished in 136.781 seconds > [root@hnode-3-7 hama-trunk]# core/bin/hama jar > examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16 > 10000 30 > 11/11/03 16:43:56 INFO bsp.BSPJobClient: Running job: job_201111031629_0003 > 11/11/03 16:43:59 INFO bsp.BSPJobClient: Current supersteps number: 0 > 11/11/03 16:45:42 INFO bsp.BSPJobClient: Current supersteps number: 1 > 11/11/03 16:45:48 INFO bsp.BSPJobClient: Current supersteps number: 2 > 11/11/03 16:45:54 INFO bsp.BSPJobClient: Current supersteps number: 3 > 11/11/03 16:45:57 INFO bsp.BSPJobClient: Current supersteps number: 4 > 11/11/03 16:46:03 INFO bsp.BSPJobClient: Current supersteps number: 5 > 11/11/03 16:46:06 INFO bsp.BSPJobClient: Current supersteps number: 6 > 11/11/03 16:46:09 INFO bsp.BSPJobClient: Current supersteps number: 7 > 11/11/03 16:46:12 INFO bsp.BSPJobClient: Current supersteps number: 8 > 11/11/03 16:46:15 INFO bsp.BSPJobClient: Current supersteps number: 9 > 11/11/03 16:46:18 INFO bsp.BSPJobClient: Current supersteps number: 10 > 11/11/03 16:46:21 INFO bsp.BSPJobClient: Current supersteps number: 11 > 11/11/03 16:46:24 INFO bsp.BSPJobClient: Current supersteps number: 12 > 11/11/03 16:46:27 INFO bsp.BSPJobClient: Current supersteps number: 13 > 11/11/03 16:46:30 INFO bsp.BSPJobClient: Current supersteps number: 14 > 11/11/03 16:46:33 INFO bsp.BSPJobClient: Current supersteps number: 15 > 11/11/03 16:46:36 INFO bsp.BSPJobClient: Current supersteps number: 16 > 11/11/03 16:46:39 INFO bsp.BSPJobClient: Current supersteps number: 17 > 11/11/03 16:46:42 INFO bsp.BSPJobClient: Current supersteps number: 19 > 11/11/03 16:46:45 INFO bsp.BSPJobClient: Current supersteps number: 20 > 11/11/03 16:46:51 INFO bsp.BSPJobClient: Current supersteps number: 21 > 11/11/03 16:46:54 INFO bsp.BSPJobClient: Current supersteps number: 22 > 11/11/03 16:46:57 INFO bsp.BSPJobClient: Current supersteps number: 23 > 11/11/03 16:47:00 INFO bsp.BSPJobClient: Current supersteps number: 24 > 11/11/03 16:47:03 INFO bsp.BSPJobClient: Current supersteps number: 25 > 11/11/03 16:47:06 INFO bsp.BSPJobClient: Current supersteps number: 26 > 11/11/03 16:47:09 INFO bsp.BSPJobClient: Current supersteps number: 27 > 11/11/03 16:47:12 INFO bsp.BSPJobClient: Current supersteps number: 28 > 11/11/03 16:47:15 INFO bsp.BSPJobClient: Current supersteps number: 29 > 11/11/03 16:47:18 INFO bsp.BSPJobClient: Current supersteps number: 30 > 11/11/03 16:47:24 INFO bsp.BSPJobClient: The total number of supersteps: 30 > Job Finished in 209.516 seconds > [root@hnode-3-7 hama-trunk]# core/bin/hama jar > examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16 > 100000 30 > 11/11/03 16:48:59 INFO bsp.BSPJobClient: Running job: job_201111031629_0004 > 11/11/03 16:49:02 INFO bsp.BSPJobClient: Current supersteps number: 0 > 11/11/03 16:50:50 INFO bsp.BSPJobClient: Current supersteps number: 1 > 11/11/03 16:51:08 INFO bsp.BSPJobClient: Current supersteps number: 2 > 11/11/03 16:51:26 INFO bsp.BSPJobClient: Current supersteps number: 3 > 11/11/03 16:51:44 INFO bsp.BSPJobClient: Current supersteps number: 4 > 11/11/03 16:51:59 INFO bsp.BSPJobClient: Current supersteps number: 5 > 11/11/03 16:52:17 INFO bsp.BSPJobClient: Current supersteps number: 6 > 11/11/03 16:52:39 INFO bsp.BSPJobClient: Current supersteps number: 7 > 11/11/03 16:52:54 INFO bsp.BSPJobClient: Current supersteps number: 8 > 11/11/03 16:53:12 INFO bsp.BSPJobClient: Current supersteps number: 9 > 11/11/03 16:53:39 INFO bsp.BSPJobClient: Current supersteps number: 10 > 11/11/03 16:53:57 INFO bsp.BSPJobClient: Current supersteps number: 11 > 11/11/03 16:54:12 INFO bsp.BSPJobClient: Current supersteps number: 12 > 11/11/03 16:54:30 INFO bsp.BSPJobClient: Current supersteps number: 13 > 11/11/03 16:54:48 INFO bsp.BSPJobClient: Current supersteps number: 14 > 11/11/03 16:55:03 INFO bsp.BSPJobClient: Current supersteps number: 15 > 11/11/03 16:55:18 INFO bsp.BSPJobClient: Current supersteps number: 16 > 11/11/03 16:55:36 INFO bsp.BSPJobClient: Current supersteps number: 17 > 11/11/03 16:55:51 INFO bsp.BSPJobClient: Current supersteps number: 18 > 11/11/03 16:56:09 INFO bsp.BSPJobClient: Current supersteps number: 19 > 11/11/03 16:56:24 INFO bsp.BSPJobClient: Current supersteps number: 20 > 11/11/03 16:56:42 INFO bsp.BSPJobClient: Current supersteps number: 21 > 11/11/03 16:56:57 INFO bsp.BSPJobClient: Current supersteps number: 22 > 11/11/03 16:57:15 INFO bsp.BSPJobClient: Current supersteps number: 23 > 11/11/03 16:57:30 INFO bsp.BSPJobClient: Current supersteps number: 24 > 11/11/03 16:57:48 INFO bsp.BSPJobClient: Current supersteps number: 25 > 11/11/03 16:58:09 INFO bsp.BSPJobClient: Current supersteps number: 26 > 11/11/03 16:58:24 INFO bsp.BSPJobClient: Current supersteps number: 27 > 11/11/03 16:58:39 INFO bsp.BSPJobClient: Current supersteps number: 28 > 11/11/03 16:58:57 INFO bsp.BSPJobClient: Current supersteps number: 29 > 11/11/03 16:59:12 INFO bsp.BSPJobClient: Current supersteps number: 30 > 11/11/03 16:59:36 INFO bsp.BSPJobClient: The total number of supersteps: 30 > Job Finished in 639.003 seconds > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > --485b397dccfba0df9c04b0d15082--