hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-10949) Write metrics2 sink plugin for Apache Kafka -- KafkaSink
Date Thu, 03 Sep 2015 16:51:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729377#comment-14729377
] 

Allen Wittenauer edited comment on HADOOP-10949 at 9/3/15 4:50 PM:
-------------------------------------------------------------------

hadoop-tools still suffers from the same sorts of problems though, just on a smaller scale,
made worse by how it is laid out vs. the rest of Hadoop.  To be more specific, most of the
sub-projects do:

{code}
share/hadoop/(project)        <- hadoop bits
share/hadoop/(project)/lib   <- dependencies
{code}

tools, on the other hand, just does share/hadoop/tools/lib, with both the hadoop bits and
the dependencies intermingled.  Every time someone issues a command that needs tools support
(e.g., distcp), all of this stuff gets pulled in. This is a historical issue, and probably
one that should get broken in 3.x.

If we move all of the metrics sinks so their own universe, the file layout should be changed
appropriately.


was (Author: aw):
hadoop-tools still suffers from the same sorts of problems though, just on a smaller scale,
made worse by how it is laid out vs. the rest of Hadoop.  To be more specific, most of the
sub-projects do:

{code}
share/hadoop/(project)   <- hadoop bits
share/hadoop/(project)   <- dependencies
{code}

tools, on the other hand, just does share/hadoop/tools/lib, with both the hadoop bits and
the dependencies intermingled.  Every time someone issues a command that needs tools support
(e.g., distcp), all of this stuff gets pulled in. This is a historical issue, and probably
one that should get broken in 3.x.

If we move all of the metrics sinks so their own universe, the file layout should be changed
appropriately.

> Write metrics2 sink plugin for Apache Kafka -- KafkaSink
> --------------------------------------------------------
>
>                 Key: HADOOP-10949
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10949
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: metrics
>            Reporter: Babak Behzad
>            Assignee: Babak Behzad
>         Attachments: HADOOP-10949-1.patch, HADOOP-10949-2.patch, HADOOP-10949-4.patch,
HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch,
HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch, HADOOP-10949.patch,
HADOOP-10949.patch
>
>
> Write a metrics2 sink plugin for Hadoop to send metrics directly to Apache Kafka in addition
to the current, Graphite ([Hadoop-9704|https://issues.apache.org/jira/browse/HADOOP-9704]),
Ganglia and File sinks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message