hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8213) DFSClient should not instantiate SpanReceiverHost
Date Wed, 22 Apr 2015 21:01:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14507888#comment-14507888

Colin Patrick McCabe commented on HDFS-8213:

bq. I think Billie Rinaldi is correct here; the client should not instantiate it's own SpanReceiverHost,
but instead depend on the process in which it resides to provide. This is how HBase client
works as well.

[~ndimiduk], what if I want to trace an HBase PUT all the way through the system?  You're
saying that the HBase client can't activate tracing on its own, so I have to make code changes
to the process doing the PUT (i.e. the user of the HBase client) in order to get that info?
 Seems like a limitation.

It's also worth pointing out that adding a {{SpanReceiverHost}} to the {{DFSClient}} is not
really a new change... it goes back to HDFS-7055, last October.  So it's been in there at
least 6 months.  Of course we can revisit it if that makes sense, but it's not really "new"
except in the sense that it took a very long time to do another Hadoop release with that in
it.  (We really should start being better with releases...)

Thinking about this a little more, another possible resolution here is to change the configuration
keys which the DFSClient looks for so that it's different than the ones which the NameNode
and DataNode look for.  Right now {{hadoop.htrace.spanreceiver.classes}} will activate span
receivers in both the NN and the DFSClient.  But the DFSClient could instead look for {{hdfs.client.htrace.spanreceiver.classes}}.
 Then [~billie.rinaldi] could use the same configuration file for everything, and the dfsclient
would never create its own span receivers or samplers.  And I could continue to trace the
dfsclient without modifying daemon code.  Seems like a good resolution.

> DFSClient should not instantiate SpanReceiverHost
> -------------------------------------------------
>                 Key: HDFS-8213
>                 URL: https://issues.apache.org/jira/browse/HDFS-8213
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Billie Rinaldi
>            Assignee: Brahma Reddy Battula
>            Priority: Critical
> DFSClient initializing SpanReceivers is a problem for Accumulo, which manages SpanReceivers
through its own configuration.  This results in the same receivers being registered multiple
times and spans being delivered more than once.  The documentation says SpanReceiverHost.getInstance
should be issued once per process, so there is no expectation that DFSClient should do this.

This message was sent by Atlassian JIRA

View raw message