hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adarsh Sharma <adarsh.sha...@orkash.com>
Subject Re: Library Issue
Date Mon, 28 Feb 2011 09:46:37 GMT
Harsh J wrote:
>> You're facing a permissions issue with a device, not a Hadoop-related
>> issue. Find a way to let users access the required devices
>> (/dev/nvidiactl is what's reported in your ST, for starters).
>>
>> On Mon, Feb 28, 2011 at 12:05 PM, Adarsh Sharma
>> <adarsh.sharma@orkash.com> wrote:
>>  
>>> Greetings to all,
>>>
>>> Today i came across a strange problem about non-root users in Linux 
>>> ( CentOS
>>> ).
>>>
>>> I am able to compile & run a Java Program through below commands 
>>> properly :
>>>
>>> [root@cuda1 hadoop-0.20.2]# javac EnumDevices.java
>>> [root@cuda1 hadoop-0.20.2]# java EnumDevices
>>> Total number of devices: 1
>>> Name: Tesla C1060
>>> Version: 1.3
>>> Clock rate: 1296000 MHz
>>> Threads per block: 512
>>>
>>> But I need to run it through other user [B]hadoop[/B]  in CentOS
>>>
>>> [hadoop@ws37-mah-lin hadoop-0.20.2]$ javac EnumDevices.java
>>> [hadoop@ws37-mah-lin hadoop-0.20.2]$ java EnumDevices
>>> NVIDIA: could not open the device file /dev/nvidiactl (Permission 
>>> denied).
>>> Exception in thread "main" CUDA Driver error: 100
>>>       at jcuda.CUDA.setError(CUDA.java:1874)
>>>       at jcuda.CUDA.init(CUDA.java:62)
>>>       at jcuda.CUDA.<init>(CUDA.java:42)
>>>       at EnumDevices.main(EnumDevices.java:20)

*I settled the above issue by setting permission to /dev/nvidia files to 
Hadoop user & group.
Now, I am able to compile programs from command

[hadoop@cuda1 hadoop-0.20.2]# javac EnumDevices.java
[hadoop@cuda1 hadoop-0.20.2]# java EnumDevices
Total number of devices: 1
Name: Tesla C1060
Version: 1.3
Clock rate: 1296000 MHz
Threads per block: 512

But Still don't know why it fails in Map-reduce job.

[hadoop@ws37-mah-lin hadoop-0.20.2]$ bin/hadoop jar wordcount1.jar 
org.myorg.WordCount /user/hadoop/gutenberg /user/hadoop/output1
11/02/28 15:01:45 INFO input.FileInputFormat: Total input paths to 
process : 3
11/02/28 15:01:45 INFO mapred.JobClient: Running job: job_201102281104_0006
11/02/28 15:01:46 INFO mapred.JobClient:  map 0% reduce 0%
11/02/28 15:01:56 INFO mapred.JobClient: Task Id : 
attempt_201102281104_0006_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
        ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
        at jcuda.CUDA.init(CUDA.java:62)
        at jcuda.CUDA.<init>(CUDA.java:42)
        at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
        ... 8 more

11/02/28 15:01:56 INFO mapred.JobClient: Task Id : 
attempt_201102281104_0006_m_000001_0, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
        ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
        at jcuda.CUDA.init(CUDA.java:62)
        at jcuda.CUDA.<init>(CUDA.java:42)
        at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
        ... 8 more

11/02/28 15:02:05 INFO mapred.JobClient: Task Id : 
attempt_201102281104_0006_m_000002_1, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
        ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
        at jcuda.CUDA.init(CUDA.java:62)
        at jcuda.CUDA.<init>(CUDA.java:42)
        at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
        ... 8 more

11/02/28 15:02:09 INFO mapred.JobClient: Task Id : 
attempt_201102281104_0006_m_000000_2, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
        ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
        at jcuda.CUDA.init(CUDA.java:62)
        at jcuda.CUDA.<init>(CUDA.java:42)
        at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
        ... 8 more

11/02/28 15:02:09 INFO mapred.JobClient: Task Id : 
attempt_201102281104_0006_m_000001_2, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
        ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
        at jcuda.CUDA.init(CUDA.java:62)
        at jcuda.CUDA.<init>(CUDA.java:42)
        at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
        ... 8 more

11/02/28 15:02:11 INFO mapred.JobClient: Task Id : 
attempt_201102281104_0006_m_000002_2, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
        ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
        at jcuda.CUDA.init(CUDA.java:62)
        at jcuda.CUDA.<init>(CUDA.java:42)
        at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
        ... 8 more

11/02/28 15:02:19 INFO mapred.JobClient: Job complete: job_201102281104_0006
11/02/28 15:02:19 INFO mapred.JobClient: Counters: 3
11/02/28 15:02:19 INFO mapred.JobClient:   Job Counters
11/02/28 15:02:19 INFO mapred.JobClient:     Launched map tasks=12
11/02/28 15:02:19 INFO mapred.JobClient:     Data-local map tasks=12
11/02/28 15:02:19 INFO mapred.JobClient:     Failed map tasks=1
[hadoop@ws37-mah-lin hadoop-0.20.2]$

Any suggestions.


Thanks

*
>>> [hadoop@ws37-mah-lin hadoop-0.20.2]$
>>>
>>> Actually I need to run a map-reduce code but first if it runs 
>>> through simple
>>> then I will go for it.
>>>
>>> Please guide me how to solve this issue as CLASSPATH is same through 
>>> all
>>> users.
>>>
>>>
>>>
>>> Thanks & best Regards,
>>> Adarsh Sharma
>>>
>>>
>>>     
>>
>>
>>
>>   
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message