hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marshall Schor <...@schor.com>
Subject classpath for finding Key classes
Date Thu, 30 Apr 2009 22:22:22 GMT
In hadoop, one can define the classes to be used for Keys and Values.  I
do this.  When I make my giant Jar file holding everything needed for
running my application, I include these classes.

However, I've discovered that that is not enough it seems (in 0.19.1
version - in case that matters :-) ).  The job start up processes is
reading the configuration and finding the names of my Key classes, and
tries to load them.  But it is not using the giant Jar for my job,
(yet), so it doesn't find them.

A work-around that I've found is to include my giant Jar as the argument
to -libjars - that seems to get the class path set up so the startup /
validation code can find my classes.  This seems wasteful - having the
giant jar in two places...

Is there a best practices way to do this that's better than this?

Thanks. -Marshall Schor

View raw message