chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-440) Custom processor classes not detected unless added to chukwa-core jar
Date Wed, 23 Dec 2009 18:20:29 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794132#action_12794132
] 

Jerome Boulon commented on CHUKWA-440:
--------------------------------------

-1 - Again this should not done by manually adding jars. 
There's a generic way to do that in hadoop world using the -libjars flag, there's no point
of adding chukwa specific code here, the problem has already been solved in hadoop.
So instead of adding some chukwa specific code to add the jar, we should enhance the demux
script to look for additional jars and use the -libjars flag.

if the -libjars is not working for demux that's another issue but since we are implementing
Tools it should already be supported.





> Custom processor classes not detected unless added to chukwa-core jar
> ---------------------------------------------------------------------
>
>                 Key: CHUKWA-440
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-440
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Bill Graham
>
> http://wiki.apache.org/hadoop/DemuxModification
> After implementing a custom parser as shown in the wiki link above and mapping it to
a data type in chukwa-demux.xml, there's no easy way to register the class so it gets included
in the job that's submitted to the hadoop cluster. I've added my jar containing the class
to the lib/ directory of the chukwa data processor install and verified that it's in the classapath,
but it's not submitted with the Hadoop job. On the Hadoop cluster ClassNotFoundExceptions
are thrown in my mapper.
> The only way I've been able to make this work is to do the following: 
> 1. put my class in the package org.apache.hadoop.chukwa.extraction.demux.processor.mapper
> 2. then manually add that class to the chukwa-core-0.3.0.jar that is on my data processor
> Instead, the class should be detected from whatever jar it lives in that's in the demux
class path, regardless of the package it lives in.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message