hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chandra Mohan, Ananda Vel Murugan" <Ananda.Muru...@honeywell.com>
Subject RE: Distributed cache in command line
Date Tue, 24 Sep 2013 09:26:49 GMT

Thanks for the response. I can create symlinks for the files. But I don't know how to add
jar to distributed cache. I found one way is by using libjars argument while running hadoop
job. Is it possible to add a jar file directly to distributed cache? Is there any specific
folder in HDFS which acts as distributed cache?


From: Omkar Joshi [mailto:ojoshi@hortonworks.com]
Sent: Monday, September 23, 2013 11:57 PM
To: user@hadoop.apache.org
Subject: Re: Distributed cache in command line


I have no idea about RHadoop but in general in YARN we do create symlinks for the files in
distributed cache in the current working directory of every container. You may be able to
use that somehow.

Omkar Joshi
Hortonworks Inc.<http://www.hortonworks.com>

On Mon, Sep 23, 2013 at 6:28 AM, Chandra Mohan, Ananda Vel Murugan <Ananda.Murugan@honeywell.com<mailto:Ananda.Murugan@honeywell.com>>

Is it possible to access distributed cache in command line? I have written a custom InputFormat
implementation which I want to add to distributed cache. Using libjars is not an option for
me as I am not running Hadoop job in command line. I am running it using RHadoop package in
R which internally uses Hadoop streaming. Please help. Thanks.


NOTICE: This message is intended for the use of the individual or entity to which it is addressed
and may contain information that is confidential, privileged and exempt from disclosure under
applicable law. If the reader of this message is not the intended recipient, you are hereby
notified that any printing, copying, dissemination, distribution, disclosure or forwarding
of this communication is strictly prohibited. If you have received this communication in error,
please contact the sender immediately and delete it from your system. Thank You.

View raw message