From hadoop-user-return-2490-apmail-lucene-hadoop-user-archive=lucene.apache.org@lucene.apache.org Mon Oct 08 19:28:32 2007 Return-Path: Delivered-To: apmail-lucene-hadoop-user-archive@locus.apache.org Received: (qmail 90708 invoked from network); 8 Oct 2007 19:28:30 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 8 Oct 2007 19:28:30 -0000 Received: (qmail 12033 invoked by uid 500); 8 Oct 2007 19:28:16 -0000 Delivered-To: apmail-lucene-hadoop-user-archive@lucene.apache.org Received: (qmail 12014 invoked by uid 500); 8 Oct 2007 19:28:16 -0000 Mailing-List: contact hadoop-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-user@lucene.apache.org Delivered-To: mailing list hadoop-user@lucene.apache.org Delivered-To: moderator for hadoop-user@lucene.apache.org Received: (qmail 69252 invoked by uid 99); 8 Oct 2007 19:03:14 -0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of christophe.taton@gmail.com designates 64.233.182.187 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding:from; bh=CaqbeDUhI4PNxPDVJ11lQxlg3p8NnGUDzDiBRQSyUME=; b=cgPcjiqBw3/NEPw4aR6bp9Qnf3ORyXglPM8PAIPOGd0tGihATrkSXOlmz3+PV6vpST1RTHD98/u1QwJKaNKDMIcesPpSiGtuQ7NWSIFZV96hKBRiUW2J0OILMYUX2FL5pA7xcHo3ar021NQeX0pWcGCRF0zG/SRwqkKkC6ipRjM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding:from; b=Py8sLRQw7hAbsAcC17HMaScDStJa0FFMskRLuRtt8VUnYif4XMb/GX28mmnotCuv++ZJdAeDv4TB7GO35mxjPS1b3j/m43L8mfdzn2yJ3kp0k2H/pj6HbgnJVNlc3vWX6iCcdmXxJ+r9IRHYeIIwwdaC6WWxLq6mTExoD+YutUA= Message-ID: <470A7F12.2060602@gmail.com> Date: Mon, 08 Oct 2007 21:03:46 +0200 User-Agent: Thunderbird 2.0.0.6 (X11/20070926) MIME-Version: 1.0 To: hadoop-user@lucene.apache.org Subject: Re: External jars revisited. References: <4709E8C7.2050009@opera.com> In-Reply-To: <4709E8C7.2050009@opera.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit From: Christophe Taton X-Virus-Checked: Checked by ClamAV on apache.org Hi Daniel, Can you try to build and run a single jar file which contains all required class files directly (i.e. without including jar files inside the job jar file)? This should prevent classloading problems. If the error still persists, then you might suspect other problems. Chris Daniel Wressle wrote: > Hello Hadoopers! > > I have just recently started using Hadoop and I have a question that > has puzzled me for a couple of days now. > > I have already browsed the mailing list and found some relevant posts, > especially > http://mail-archives.apache.org/mod_mbox/lucene-hadoop-user/200708.mbox/%3c84ad79bb0708131649x3b94cc18x7a0910090f06a1e7@mail.gmail.com%3e, > but the solution eludes me. > > My Map/Reduce job relies on external jars and I had to modify my ant > script to include them in the lib/ directory of my jar file. So far, > so good. The job runs without any issues when I issue the job on my > local machine only. > > However, adding a second machine to the mini-cluster presents the > following problem: a NullPointerException being thrown as soon as I > call any function within a class I have imported from the external > jars. Please note that this will only happen on the other machine, the > maps on my main machine, which I submit the job on, will proceed > without any warnings. > > java.lang.NullPointerException at xxx.xxx.xxx (Unknown Source) is the > actual log output from hadoop. > > My jar file contains all the necessary jars in the lib/ directory. Do > I need to place them somewhere else on the slaves in order for my > submitted job to be able to use them? > > Any pointers would be much appreciated.