hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Synchronization problem in FSNameSystemMetrics
Date Thu, 21 Jan 2010 19:41:40 GMT
Hi Zlatin,

I agree that this access is unsynchronized and thus can be stale. Looking at
the implementation of HashMap.size, it doesn't need to do anything except
copy a single int, so it shouldn't be a cause for any errors beyond stale
data (ie it can't throw a ConcurrentModificationException).

In general, the contract for metrics throughout Hadoop doesn't imply
anything about consistency or exact up-to-date values. They're generally
meant for operational monitoring where exact values are less important.

Thanks
-Todd

On Wed, Jan 13, 2010 at 9:24 AM, <Zlatin.Balevsky@barclayscapital.com>wrote:

> Hello all,
>
> Unless I'm missing something, the synchronization in FSNameSystemMetrics
> on the 0.20 does not appear correct.  It locks on itself while pulling
> various metrics from the FSNameSystem object.  For example,
> FSNameSystem.getBlocksTotal() delegates to BlocksMap.size() which
> delegates to the contained HashMap object.  None of those methods are
> synchronized on FSNameSystem, so one cannot get accurate data for that
> metric.  Is this intentional?
>
> Zlatin
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient of
> this e-mail, do not duplicate or redistribute it by any means. Please delete
> it and any attachments and notify the sender that you have received it in
> error. Unless specifically indicated, this e-mail is not an offer to buy or
> sell or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays. Any views or opinions
> presented are solely those of the author and do not necessarily represent
> those of Barclays. This e-mail is subject to terms available at the
> following link: www.barcap.com/emaildisclaimer. By messaging with Barclays
> you consent to the foregoing.  Barclays Capital is the investment banking
> division of Barclays Bank PLC, a company registered in England (number
> 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.
>  This email may relate to or be sent from other members of the Barclays
> Group.
> _______________________________________________
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message