Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of mathias.herberts@gmail.com
 designates 209.85.212.178 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=fAtlDXz3Jm11GEBpTG917NnQSFgaed4zSILusoDplT0nPeQZMykDIl+UsGCHNjIcVj
         xRpDE0b/UW5UVRgzOsrml90Q5g5FezH+PtIsOW2eZVYyq+Ao/dNinwyLaCmemAlFJ77s
         2+3gRQgaLspAdpIL8763RpTDLeXXa6K0DqPyM=
MIME-Version: 1.0
In-Reply-To: <AANLkTimTfOEnsXvmX4ndXp9SHSHnyW-CXdUZ_MLi_cL1@mail.gmail.com>
References: <AANLkTikNd8ebmsc7DXiRGfJZZZ-UELMJ0wVaLjD3Rdw9@mail.gmail.com>
	<AANLkTikyTRM1KfieSRS+ei3WbWHKfVixk0US=R6HVQAJ@mail.gmail.com>
	<AANLkTimTfOEnsXvmX4ndXp9SHSHnyW-CXdUZ_MLi_cL1@mail.gmail.com>
Date: Fri, 10 Dec 2010 01:26:18 +0100
Message-ID: <AANLkTikparsP2d3KDyHhOY_YwSXR5N1eq99dbe_Mwrot@mail.gmail.com>
Subject: Re: Jcuda on Hadoop
From: Mathias Herberts <mathias.herberts@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Put the native libs (.so) with the other common libs that hadoop
already has (libhadoop, libcompression, ...). Or put them in a
specific 'lib/native' in your job jar and set 'java.library.path' to
'job.local.dir'/../jars/lib/native in your MR job.

Mathias.

On Thu, Dec 9, 2010 at 23:17, He Chen <airbots@gmail.com> wrote:
> I am still in the test stage. I mean I only start one JobTracker and one
> TaskTracker.
> I copied all jcuda.*.jar into =C2=A0HADOOP_HOME/lib/
>
> from the ps aux|grep java
>
> I can confirm the JT and TT processes all contain the jcuda.*.jar files
>
> On Thu, Dec 9, 2010 at 4:05 PM, Mathias Herberts <mathias.herberts@gmail.=
com
>> wrote:
>
>> You need to have the native libs on all tasktrackers and have
>> java.library.path correctly set.
>> On Dec 9, 2010 11:01 PM, "He Chen" <airbots@gmail.com> wrote:
>> > Hello everyone, I 've got a problem when I write some Jcuda program ba=
sed
>> on
>> > Hadoop MapReduce. I use the jcudaUtill. The KernelLauncherSample can b=
e
>> > successfully executed on my worker node. However, When I submit a prog=
ram
>> > containing jcuda to Hadoop MapReduce. I got following errors. Any repl=
y
>> will
>> > be appreciated! 10/12/09 15:41:39 INFO mapred.JobClient: Running job:
>> > job_201012091523_0002 10/12/09 15:41:40 INFO mapred.JobClient: map 0%
>> reduce
>> > 0% 10/12/09 15:41:53 INFO mapred.JobClient: Task Id :
>> > attempt_201012091523_0002_m_000000_0, Status : FAILED Error: Could not
>> load
>> > native library attempt_201012091523_0002_m_000000_0: [GC
>> > 7402K->1594K(12096K), 0.0045650 secs]
>> attempt_201012091523_0002_m_000000_0:
>> > [GC 108666K->104610K(116800K), 0.0106860 secs]
>> > attempt_201012091523_0002_m_000000_0: [Full GC 104610K->104276K(129856=
K),
>> > 0.0482530 secs] attempt_201012091523_0002_m_000000_0: Error while load=
ing
>> > native library with base name "JCudaDriver"
>> > attempt_201012091523_0002_m_000000_0: Operating system name: Linux
>> > attempt_201012091523_0002_m_000000_0: Architecture : amd64
>> > attempt_201012091523_0002_m_000000_0: Architecture bit size: 64 10/12/=
09
>> > 15:42:00 INFO mapred.JobClient: Task Id :
>> > attempt_201012091523_0002_m_000000_1, Status : FAILED Error: Could not
>> load
>> > native library attempt_201012091523_0002_m_000000_1: [GC
>> > 7373K->1573K(18368K), 0.0045230 secs]
>> attempt_201012091523_0002_m_000000_1:
>> > Error while loading native library with base name "JCudaDriver"
>> > attempt_201012091523_0002_m_000000_1: Operating system name: Linux
>> > attempt_201012091523_0002_m_000000_1: Architecture : amd64
>> > attempt_201012091523_0002_m_000000_1: Architecture bit size: 64 It loo=
ks
>> > like The jcuda library file can not be sucessfully loaded. Actually, I
>> tried
>> > many combinations. 1) I include all the jcuda library files in my
>> > TaskTracker's classpath, and also include jcuda library file into my
>> > mapreduce program. 2) TaskTracker's classpath w/o jcuda library, but m=
y
>> > program contains them 3) TaskTracker's classpath w/ jcuda librara, but=
 my
>> > program w/o them All of them report the same error above.
>>
>