Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of markkerzner@gmail.com
 designates 209.85.214.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=WLNfdaVkr/xzTU4PceiDFDXAiNONVcCFwjsDsJyBb6JTiOHGl1tvVgEimNOzsqU39z
         3++re7z3cnYVvuKq7qpTRNGlXquYxttRJvbgqSXZNYLbmerud0uqy3h6UPbb9XBZoqNI
         IGak0CuG1n26lqL3mbQI3fEXLKUuorvw4Rimk=
MIME-Version: 1.0
In-Reply-To: <BANLkTi=q2EmTTBa_xGvQsEiMduDqgHh=2w@mail.gmail.com>
References: <BANLkTimMiffan4+rmqiEV86nFe3e6p8a-g@mail.gmail.com>
	<BANLkTi=9r2cSg8gQU5dURsyefLM9=pnjyA@mail.gmail.com>
	<BANLkTi=q2EmTTBa_xGvQsEiMduDqgHh=2w@mail.gmail.com>
Date: Mon, 4 Apr 2011 11:40:15 -0500
Message-ID: <BANLkTik8Q39E2wJE1FMX0qrqRQK7QzsuSA@mail.gmail.com>
Subject: Re: Including Additional Jars
From: Mark Kerzner <markkerzner@gmail.com>
To: Shuja Rehman <shujamughal@gmail.com>, common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e6d6452c10653b04a01a6a6b

--0016e6d6452c10653b04a01a6a6b
Content-Type: text/plain; charset=ISO-8859-1

Then it seems you want to do the opposite of what I have done in this
script. I AM combining all the jars in one jar, and you already have that.

Rather, you want to distribute only your app jar, and put the other ones in
the lib folder on the server.

I know that when you run a standard MR job, you only need to mention your
jar, and the other Hadoop jars already come from the lib. In other words,
you should be able to run it like this:

hadoop jar your-jar parameters

Since you are using Cloudera distro, this runs the following

/usr/bin/hadoop-0.20

which in turn runs this script

#!/bin/sh
export HADOOP_HOME=/usr/lib/hadoop-0.20
exec /usr/lib/hadoop-0.20/bin/hadoop "$@"

Since HADOOP_HOME is set, it knows that the libraries are in here

/usr/lib/hadoop-0.20/lib/

therefore, I think that if you put your additional libraries in the same
folder, it should just pick them up.

Sincerely,
Mark


On Mon, Apr 4, 2011 at 11:31 AM, Shuja Rehman <shujamughal@gmail.com> wrote:

> hi,
> i do not understand it. can u take my explain it with my example?
>
> I have following jars in lib folder of dist created by netbeans
> (dist/lib/).
>
> commons-logging-1.1.1.jar
> guava-r07.jar
> hadoop-0.20.2+737-core.jar
> hbase.jar
> hbase-0.89.20100924+28.jar
> log4j-1.2.15.jar
> mysql-connector-java-5.1.7-bin.jar
> UIDataTransporter.jar
> zookeeper.jar
>
> and dist folder contains only
>
> MyProgram.jar
>
>
> at the moment, i am combining all jars files to produce the single file.
> but now i want to just put the dist/lib/ *.jars for once on server and only
> MyProgram.jar should be copied everytime i change the code.
>
> so can u transfer ur code according to my example???
> Thanks
>
>
>
>
> On Mon, Apr 4, 2011 at 8:17 PM, Mark Kerzner <markkerzner@gmail.com>wrote:
>
>> Shuja,
>>
>> here is what I do in NB environment
>>
>> #!/bin/sh
>> cd ../dist
>> jar -xf Chapter1.jar
>> jar -cmf META-INF/MANIFEST.MF  ../Chapter3-for-Hadoop.jar *
>> cd ../bin
>> echo "Repackaged for Hadoop"
>>
>> and it does the job. I run it only when I want to build this jar.
>>
>> Mark
>>
>> On Mon, Apr 4, 2011 at 10:06 AM, Shuja Rehman <shujamughal@gmail.com>wrote:
>>
>>> Hi All
>>>
>>> I have created a map reduce job and to run on it on the cluster, i have
>>> bundled all jars(hadoop, hbase etc) into single jar which increases the
>>> size
>>> of overall file. During the development process, i need to copy again and
>>> again this complete file which is very time consuming so is there any way
>>> that i just copy the program jar only and do not need to copy the lib
>>> files
>>> again and again. i am using net beans to develop the program.
>>>
>>> kindly let me know how to solve this issue?
>>>
>>> Thanks
>>>
>>> --
>>> Regards
>>> Shuja-ur-Rehman Baig
>>> <http://pk.linkedin.com/in/shujamughal>
>>>
>>
>>
>
>
> --
> Regards
> Shuja-ur-Rehman Baig
> <http://pk.linkedin.com/in/shujamughal>
>
>

--0016e6d6452c10653b04a01a6a6b--