Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-dev@hadoop.apache.org
Message-ID: <68427133.1261609649397.JavaMail.jira@brutus>
Date: Wed, 23 Dec 2009 23:07:29 +0000 (UTC)
From: "Jerome Boulon (JIRA)" <jira@apache.org>
To: chukwa-dev@hadoop.apache.org
Subject: [jira] Commented: (CHUKWA-440) Custom processor classes not
 detected unless added to chukwa-core jar
In-Reply-To: <1657516270.1261550729417.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CHUKWA-440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12794262#action_12794262 ] 

Jerome Boulon commented on CHUKWA-440:
--------------------------------------

So if it's not working for Chukwa we need to fix it and I want to add the flag at the command line level since I'm no longer using DemuxManager because of other issues on S3, if we fix it inside DemuxManager then everyone that is only using Demux will still have the issue.

I'll do some testing but I don't have time right now, so if someone can test the libjar flag with a standard hadoop job that will be a good start.


> Custom processor classes not detected unless added to chukwa-core jar
> ---------------------------------------------------------------------
>
>                 Key: CHUKWA-440
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-440
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Bill Graham
>
> http://wiki.apache.org/hadoop/DemuxModification
> After implementing a custom parser as shown in the wiki link above and mapping it to a data type in chukwa-demux.xml, there's no easy way to register the class so it gets included in the job that's submitted to the hadoop cluster. I've added my jar containing the class to the lib/ directory of the chukwa data processor install and verified that it's in the classapath, but it's not submitted with the Hadoop job. On the Hadoop cluster ClassNotFoundExceptions are thrown in my mapper.
> The only way I've been able to make this work is to do the following: 
> 1. put my class in the package org.apache.hadoop.chukwa.extraction.demux.processor.mapper
> 2. then manually add that class to the chukwa-core-0.3.0.jar that is on my data processor
> Instead, the class should be detected from whatever jar it lives in that's in the demux class path, regardless of the package it lives in.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.