hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6956) Allow dynamically changing the tracing level in Hadoop servers
Date Thu, 25 Sep 2014 18:39:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148117#comment-14148117

Colin Patrick McCabe commented on HDFS-6956:

bq. Should SpanReceiverInfoFactory be named SpanReceiverInfoBuilder?


bq. microscopic nit: Create linkedlist once only:

bq. I love all the ways we have of doing options in hadoop (smile). I notice here that -h
won't work but should be fine since no args gives you usage.

Yeah, let's add \-h as a synonym for \-help.

bq. Checks args are non-null before you put up the proxy?


bq. No need to close proxy? There seems to be a close in protocol? Just exiting the JVM is
fine? Any chance of someone including this little script inside a long-running app at all?
If hbase wants to on/off hdfs trace, won't it need to make use of this little client or something
like it

I think it would be more common for a long-running app to create its own {{TraceAdminProtocolPB}}
proxy object and make the RPC calls itself.  It should be pretty easy to do that directly
if you don't care about parsing argument strings like {{TraceAdmin}} does.  Still, I will
close the proxy here just for completeness.

bq. Or I suppose these are just admin commands... so hbase would just on/off tracing, NOT
on/off tracing sink. (IIRC stopping proxy hoses it so it can't be used again...that may be
my recollection of a childhood experience that involved me and an early hadoop...)

Sounds traumatic :P  I think HBase shouldn't need to make these RPCs to Hadoop.  I think the
most usual configuration to trace HBase + HDFS would be to configure HDFS to use {{NeverSampler}},
and then configure HBase to use {{ProbabilitySampler}}.  So some percentage of HBase requests
 to HDFS have tracing, and HDFS will trace only those requests.

bq. Does above mean who can be clients of this newly added protocol? If so, we should had
HBase since we'll be pulling on this chain for sure.

I haven't thought about whether HBase would ever call this API.  It's an interesting idea...
would you envision that the user would use an HBase command-line, and then HBase would add
a span receiver for itself and also for HDFS?  It seems simpler to set those up separately
in case a span receiver already exists on HDFS, but maybe doing both at once could be an option?
 I will add HBase anyway, and we can figure it out later.

bq. Patch is for hadoop trunk?

trunk and 2.6

bq. Does TraceAdminProtocolTranslatorPB (and other new classes in this package) need audience

yeah, will add

> Allow dynamically changing the tracing level in Hadoop servers
> --------------------------------------------------------------
>                 Key: HDFS-6956
>                 URL: https://issues.apache.org/jira/browse/HDFS-6956
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-6956.002.patch, HDFS-6956.003.patch, HDFS-6956.004.patch
> We should allow users to dynamically change the tracing level in Hadoop servers.  The
easiest way to do this is probably to have an RPC accessible only to the superuser that changes
tracing settings.  This would allow us to turn on and off tracing on the NameNode, DataNode,
etc. at runtime.

This message was sent by Atlassian JIRA

View raw message