giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: more concurrency issues with requests management?
Date Wed, 20 Feb 2013 00:04:59 GMT
i'm running with the flags, nothing suspicious:


2013-02-20 00:59:28,208 INFO org.apache.giraph.utils.ExpectedBarrier:
waitForRequiredPermits: Waiting for times required to be 526 (currently 38)
2013-02-20 00:59:28,897 INFO org.apache.giraph.graph.GraphTaskManager:
execute: 95 partitions to process with 1 compute thread(s), originally 1
thread(s) on superstep 0
2013-02-20 00:59:28,905 INFO org.apache.giraph.comm.SendPartitionCache:
SendPartitionCache: maxVerticesPerTransfer = 10000
2013-02-20 00:59:28,905 INFO org.apache.giraph.comm.SendPartitionCache:
SendPartitionCache: maxEdgesPerTransfer = 80000
2013-02-20 00:59:42,700 INFO
org.apache.giraph.comm.netty.handler.RequestDecoder: decode: Server window
metrics MBytes/sec sent = 0.0051, MBytes/sec received = 1.6067, MBytesSent
= 0.2217, MBytesReceived = 69.8451, ave sent req MBytes = 0, ave received
req MBytes = 0.0047, secs waited = 43.471
2013-02-20 00:59:50,145 INFO org.apache.giraph.graph.ComputeCallable: call:
Computation took 21.244951 secs for 95 partitions on superstep 0.  Flushing
started
2013-02-20 00:59:50,913 INFO org.apache.giraph.worker.BspServiceWorker:
finishSuperstep: Waiting on all requests, superstep 0 Memory
(free/total/max) = 5478.25M / 9165.31M / 14346.69M
2013-02-20 00:59:50,914 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 414 open requests,
waiting for it to be <= 0, MBytes/sec sent = 9.6767, MBytes/sec received =
0.0001, MBytesSent = 175.2152, MBytesReceived = 0.0021, ave sent req MBytes
= 0.299, ave received req MBytes = 0, secs waited = 18.106
2013-02-20 01:00:07,920 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 37 open requests,
waiting for it to be <= 0, MBytes/sec sent = 4.99, MBytes/sec received =
0.0002, MBytesSent = 175.2152, MBytesReceived = 0.0068, ave sent req MBytes
= 0.299, ave received req MBytes = 0, secs waited = 35.112
2013-02-20 01:00:23,284 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 33 open requests,
waiting for it to be <= 0, MBytes/sec sent = 3.4713, MBytes/sec received =
0.0001, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes
= 0.299, ave received req MBytes = 0, secs waited = 50.476
2013-02-20 01:00:51,378 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 28 open requests,
waiting for it to be <= 0, MBytes/sec sent = 2.23, MBytes/sec received =
0.0001, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes
= 0.299, ave received req MBytes = 0, secs waited = 78.57
2013-02-20 01:01:06,379 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 28 open requests,
waiting for it to be <= 0, MBytes/sec sent = 1.8725, MBytes/sec received =
0.0001, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes
= 0.299, ave received req MBytes = 0, secs waited = 93.571
2013-02-20 01:01:21,380 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 28 open requests,
waiting for it to be <= 0, MBytes/sec sent = 1.6138, MBytes/sec received =
0.0001, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes
= 0.299, ave received req MBytes = 0, secs waited = 108.572
2013-02-20 01:01:51,381 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 28 open requests,
waiting for it to be <= 0, MBytes/sec sent = 1.2644, MBytes/sec received =
0, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes =
0.299, ave received req MBytes = 0, secs waited = 138.573
2013-02-20 01:02:21,382 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 28 open requests,
waiting for it to be <= 0, MBytes/sec sent = 1.0394, MBytes/sec received =
0, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes =
0.299, ave received req MBytes = 0, secs waited = 168.574
2013-02-20 01:02:51,383 INFO org.apache.giraph.comm.netty.NettyClient:
waitSomeRequests: Waiting interval of 15000 msecs, 28 open requests,
waiting for it to be <= 0, MBytes/sec sent = 0.8824, MBytes/sec received =
0, MBytesSent = 175.2152, MBytesReceived = 0.0069, ave sent req MBytes =
0.299, ave received req MBytes = 0, secs waited = 198.575

again hanging on the fixed number of requests.

Here's the GC output:
2013-02-20T00:58:01.879+0100: 41.285: [GC [PSYoungGen:
1835072K->4806K(2140864K)] 1835072K->4806K(7034240K), 0.0112590 secs]
[Times: user=0.05 sys=0.00, real=0.03 secs]
2013-02-20T00:58:16.062+0100: 55.468: [GC [PSYoungGen:
1839878K->145181K(2140864K)] 1839878K->145181K(7034240K), 0.6005940 secs]
[Times: user=1.01 sys=0.09, real=0.60 secs]
2013-02-20T00:58:22.530+0100: 61.936: [GC [PSYoungGen:
1980253K->292829K(2140864K)] 1980253K->292829K(7034240K), 1.5621790 secs]
[Times: user=1.58 sys=0.17, real=1.56 secs]
2013-02-20T00:58:29.440+0100: 68.846: [GC [PSYoungGen:
2127901K->305789K(3975936K)] 2127901K->445565K(8869312K), 1.0169950 secs]
[Times: user=2.53 sys=0.28, real=1.02 secs]
2013-02-20T00:58:46.687+0100: 86.093: [GC [PSYoungGen:
3975933K->305773K(3975936K)] 4115709K->748141K(8869312K), 2.5621070 secs]
[Times: user=3.93 sys=0.38, real=2.56 secs]
2013-02-20T00:59:29.630+0100: 129.036: [GC [PSYoungGen:
3975917K->665434K(4491904K)] 4418285K->1107802K(9385280K), 2.3398300 secs]
[Times: user=2.74 sys=0.36, real=2.34 secs]


doesn't look it's hanging there...


On Wed, Feb 20, 2013 at 12:18 AM, Avery Ching <aching@apache.org> wrote:

> Try these to start:
>
> -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails
> -XX:+PrintGCTimeStamps -verbose:gc
>
>
> On 2/19/13 3:04 PM, Claudio Martella wrote:
>
>> Looking at the second line of the logs, it does not look like the case,
>> but
>> it could be. do you have any suggestion for the flags in particular? I'm
>> running with -XX:+**HeapDumpOnOutOfMemoryError  now, and I don't get any
>> dump.
>>
>> It could be the case, but the behavior is quite consistent currently, I'd
>> get a Child error/OOM pretty soon in these cases.
>>
>>
>> On Tue, Feb 19, 2013 at 11:45 PM, Avery Ching <aching@apache.org> wrote:
>>
>>  Can you add GC flags?  Perhaps you are running into full GCs...
>>>
>>>
>>> On 2/19/13 1:45 PM, Claudio Martella wrote:
>>>
>>>  I have not investigated further yet, but this does not look connected to
>>>> ZK, but mostly to some concurrency on the queues. The number of requests
>>>> stays to 29...
>>>>
>>>>
>>>> On Tue, Feb 19, 2013 at 9:58 PM, Sebastian Schelter <ssc@apache.org>
>>>> wrote:
>>>>
>>>>   For me, Giraph on Hadoop 1.0.4 runs into some kind of deadlock after
>>>> the
>>>>
>>>>> input was loaded when using it with 200 workers on a billion-scale
>>>>> graph....
>>>>>
>>>>> I tried several versions of zookeeper, but nothing helped.
>>>>>
>>>>> Best,
>>>>> Sebastian
>>>>>
>>>>>
>>>>> On 19.02.2013 21:55, Claudio Martella wrote:
>>>>>
>>>>>  I'm running PR with trunk on yahoo webgraph across 528 workers and I
>>>>>> get
>>>>>> the workers hanging at the flushing:
>>>>>>
>>>>>> 2013-02-19 21:40:07,187 INFO org.apache.giraph.graph.****
>>>>>> ComputeCallable:
>>>>>>
>>>>>>  call:
>>>>>
>>>>>  Computation took 28.386961 secs for 95 partitions on superstep 0.
>>>>>>
>>>>>>     Flushing
>>>>>
>>>>>  started
>>>>>> 2013-02-19 21:40:07,604 INFO org.apache.giraph.worker.**
>>>>>>
>>>>>> BspServiceWorker:
>>>>>> finishSuperstep: Waiting on all requests, superstep 0 Memory
>>>>>> (free/total/max) = 5993.42M / 8661.44M / 14346.69M
>>>>>> 2013-02-19 21:40:07,605 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 558 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 6.4758, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0003, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 27.083
>>>>>> 2013-02-19 21:40:26,717 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 39 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 3.7965, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0.0001, MBytesSent = 175.3834, MBytesReceived = 0.0068, ave sent req
>>>>>>
>>>>>>  MBytes
>>>>>
>>>>>  = 0.2998, ave received req MBytes = 0, secs waited = 46.195
>>>>>> 2013-02-19 21:40:32,248 INFO
>>>>>> org.apache.giraph.comm.netty.****handler.RequestDecoder: decode:
>>>>>> Server
>>>>>>
>>>>>>  window
>>>>>
>>>>>  metrics MBytes/sec sent = 0.0002, MBytes/sec received = 3.9824,
>>>>>>
>>>>>>  MBytesSent
>>>>>
>>>>>  = 0.0066, MBytesReceived = 155.4787, ave sent req MBytes = 0, ave
>>>>>>
>>>>>>  received
>>>>>
>>>>>  req MBytes = 0.0155, secs waited = 39.04
>>>>>> 2013-02-19 21:40:42,041 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 36 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 2.8509, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0.0001, MBytesSent = 175.3834, MBytesReceived = 0.0068, ave sent req
>>>>>>
>>>>>>  MBytes
>>>>>
>>>>>  = 0.2998, ave received req MBytes = 0, secs waited = 61.519
>>>>>> 2013-02-19 21:41:07,456 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 2.0174, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0.0001, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>>
>>>>>>  MBytes
>>>>>
>>>>>  = 0.2998, ave received req MBytes = 0, secs waited = 86.933
>>>>>> 2013-02-19 21:41:22,456 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 1.7205, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0.0001, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>>
>>>>>>  MBytes
>>>>>
>>>>>  = 0.2998, ave received req MBytes = 0, secs waited = 101.934
>>>>>> 2013-02-19 21:41:37,457 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 1.4998, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0.0001, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>>
>>>>>>  MBytes
>>>>>
>>>>>  = 0.2998, ave received req MBytes = 0, secs waited = 116.935
>>>>>> 2013-02-19 21:41:52,458 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 1.3293, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0.0001, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>>
>>>>>>  MBytes
>>>>>
>>>>>  = 0.2998, ave received req MBytes = 0, secs waited = 131.936
>>>>>> 2013-02-19 21:42:22,459 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 1.083, MBytes/sec
>>>>>> received
>>>>>> =
>>>>>> 0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 161.937
>>>>>> 2013-02-19 21:42:37,460 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.9912, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 176.938
>>>>>> 2013-02-19 21:43:07,461 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.8475, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 206.939
>>>>>> 2013-02-19 21:43:37,462 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.7402, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 236.94
>>>>>> 2013-02-19 21:43:52,463 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.6961, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 251.941
>>>>>> 2013-02-19 21:44:07,464 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.657, MBytes/sec
>>>>>> received
>>>>>> =
>>>>>> 0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 266.942
>>>>>> 2013-02-19 21:44:37,465 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.5906, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 296.943
>>>>>> 2013-02-19 21:45:07,466 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.5364, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 326.944
>>>>>> 2013-02-19 21:45:22,467 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.5129, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 341.945
>>>>>> 2013-02-19 21:45:37,468 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.4913, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 356.946
>>>>>> 2013-02-19 21:46:07,469 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.4532, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 386.947
>>>>>> 2013-02-19 21:46:22,470 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.4363, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 401.948
>>>>>> 2013-02-19 21:46:52,471 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.406, MBytes/sec
>>>>>> received
>>>>>> =
>>>>>> 0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 431.949
>>>>>> 2013-02-19 21:47:07,472 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.3924, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 446.95
>>>>>> 2013-02-19 21:47:37,473 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.3677, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 476.951
>>>>>> 2013-02-19 21:48:07,474 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.346, MBytes/sec
>>>>>> received
>>>>>> =
>>>>>> 0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 506.952
>>>>>> 2013-02-19 21:48:22,475 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.336, MBytes/sec
>>>>>> received
>>>>>> =
>>>>>> 0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 521.953
>>>>>> 2013-02-19 21:48:52,476 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.3177, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 551.954
>>>>>> 2013-02-19 21:49:07,477 INFO org.apache.giraph.comm.netty.****
>>>>>>
>>>>>> NettyClient:
>>>>>> waitSomeRequests: Waiting interval of 15000 msecs, 29 open requests,
>>>>>> waiting for it to be <= 0, MBytes/sec sent = 0.3093, MBytes/sec
>>>>>> received
>>>>>>
>>>>>>  =
>>>>>
>>>>>  0, MBytesSent = 175.3834, MBytesReceived = 0.0069, ave sent req
>>>>>> MBytes =
>>>>>> 0.2998, ave received req MBytes = 0, secs waited = 566.955
>>>>>>
>>>>>> It looks similar to the problem we had earlier with concurrency.
>>>>>>
>>>>>> Are you guys experiencing something similar at large-scale?
>>>>>>
>>>>>>
>>>>>>
>>
>


-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message