Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of msa@schor.com designates
 69.56.142.19 as permitted sender)
Message-ID: <49FA249E.3030106@schor.com>
Date: Thu, 30 Apr 2009 18:22:22 -0400
From: Marshall Schor <msa@schor.com>
User-Agent: Thunderbird 2.0.0.21 (Windows/20090302)
MIME-Version: 1.0
To: general@hadoop.apache.org
Subject: classpath for finding Key classes
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

In hadoop, one can define the classes to be used for Keys and Values.  I
do this.  When I make my giant Jar file holding everything needed for
running my application, I include these classes.

However, I've discovered that that is not enough it seems (in 0.19.1
version - in case that matters :-) ).  The job start up processes is
reading the configuration and finding the names of my Key classes, and
tries to load them.  But it is not using the giant Jar for my job,
(yet), so it doesn't find them.

A work-around that I've found is to include my giant Jar as the argument
to -libjars - that seems to get the class path set up so the startup /
validation code can find my classes.  This seems wasteful - having the
giant jar in two places...

Is there a best practices way to do this that's better than this?

Thanks. -Marshall Schor