chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <>
Subject Re: units in MDL and HICC
Date Fri, 22 May 2009 18:19:07 GMT
On Fri, May 22, 2009 at 11:10 AM, Jiaqi Tan <> wrote:
> On Fri, May 22, 2009 at 11:06 AM, Eric Yang <> wrote:
>> On 5/22/09 10:35 AM, "Ariel Rabkin" <> wrote:
>>> I'm okay with doing this in the parser.  Particularly if it's
>>> declaratively driven with something like that conversion syntax,
>>> embedded in demux-conf.   But we should probably be clear to ourselves
>>> what constitutes "truth" when it comes to units and such.
>> I like to use base unit of bytes and raw time millis in HDFS, if possible.
>> The current base units are bytes, and per minute.  All data are in bytes at
>> yahoo except HDFS capacity in GB from hadoop metrics.
> Sure, but I think what Ari was referring to was when the sar-generated
> numbers are not in the base units? Then in that case I think the Demux
> processor should look at the column labels to determine what's the
> source unit and convert that to the base unit?

Yes, exactly.  I think we're all in agreement about that now, so I've
opened CHUKWA-260 for it.


Ari Rabkin
UC Berkeley Computer Science Department

View raw message