hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wiley <kwi...@keithwiley.com>
Subject Re: streaming cacheArchive shared libraries
Date Fri, 05 Aug 2011 17:49:31 GMT
Right, so it was pushed down a level into the "testlink" directory.  That's why my shared libraries
were not linking properly to my mapper executable.  I can fix that by using "cmddev" to redirect
LD_LIBRARY_PATH.  I think that'll work.

On Aug 5, 2011, at 10:44 , Ramya Sunil wrote:

> Hi Keith,
> 
> I have tried the exact use case you have mentioned and it works fine for me.
> Below is the command line for the same:
> 
> [ramya]$ jar vxf samplelib.jar
> created: META-INF/
> inflated: META-INF/MANIFEST.MF
> inflated: libhdfs.so
> 
> [ramya]$ hadoop dfs -put samplelib.jar samplelib.jar
> 
> [ramya]$ hadoop jar hadoop-streaming.jar -input InputDir -mapper "ls
> testlink/libhdfs.so" -reducer NONE -output out -cacheArchive
> hdfs://<namenode>:<port>/user/ramya/samplelib.jar#testlink
> 
> [ramya]$ hadoop dfs -cat out/*
> testlink/libhdfs.so
> testlink/libhdfs.so
> testlink/libhdfs.so
> 
> 
> Hope it helps.
> 
> Thanks
> Ramya


________________________________________________________________________________
Keith Wiley     kwiley@keithwiley.com     keithwiley.com    music.keithwiley.com

"I used to be with it, but then they changed what it was.  Now, what I'm with
isn't it, and what's it seems weird and scary to me."
                                           --  Abe (Grandpa) Simpson
________________________________________________________________________________


Mime
View raw message