hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How many bytes does heart beat message and return value contain?
Date Sun, 12 Jun 2011 04:52:18 GMT

Perhaps you could write some tests in Java to measure object sizes; or
you can run a profiler and measure it live (VisualVM lets you profile
with visuals)?

As Arun mentioned, it is really tough to say since every heartbeat
would carry variable sizes (I'd just provided empty-ish values)
depending on each of the Task's current, reported status and the
commands being issued back. But I believe you could form a formula of
sorts with careful-enough study.

On Sun, Jun 12, 2011 at 6:17 AM, Shen LI <geminialex007@gmail.com> wrote:
> Hi Harsh,
> Thank you very much for your reply. Do you know if there is any reference
> about calculating heart beat message size  that I can turn to? If I have 4
> living task under each TaskTracker,  how many bytes will one heart beat
> message contain?
> Thanks,
> Shen
> On Fri, Jun 10, 2011 at 5:06 PM, Harsh J <harsh@cloudera.com> wrote:
>> Shen,
>> It would really vary depending on how many TaskStatus objects the
>> TaskTracker has to report in the heartbeat. If there are none (zero
>> activity), I think the byte value (per Visual VM here) comes to about
>> 60B which you can perhaps add upon.
>> Note that I'm using OSX so perhaps the ResourceStats carried within
>> aren't properly for me (some of these are Linux specific). It
>> shouldn't exceed 100B in any case though, I think.
>> Similarly, the response carries variable numbers of TaskTrackerAction
>> objects, and other job related information. At zero activity I see it
>> reported as 32B.
>> HTH. These are in memory values -- network tx. is made with some form
>> of serialization applied, and I'm unsure as to how much the variance
>> would be.
>> On Fri, Jun 10, 2011 at 9:49 PM, Shen LI <geminialex007@gmail.com> wrote:
>> > Hi All,
>> > Does anyone know how many bytes does one heart beat message contain? And
>> > what about the return value of heart beat message?
>> > Thanks,
>> > Shen
>> --
>> Harsh J

Harsh J

View raw message