hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rodrigo Schmidt <rodr...@facebook.com>
Subject Re: Building offline
Date Thu, 18 Mar 2010 07:31:02 GMT

Thanks Chaitanya!

Didn't work at first. I tried many times in different ways, connected to the
Internet and not.

When I was about to give up I realized ivy might be caching some stuff,
which turned out to be true. I had to delete the hadoop-core.jar file inside
~/ivy2/cache for it to work.

This whole ivy2 and maven dependencies look too complicated for offline use
and cross-project debugging. Isn't there an easier way to do things?

Cheers,
Rodrigo


On 3/17/10 11:27 PM, "chaitanya krishna" <chaitanyavv.iiith@gmail.com>
wrote:

> In case you want to use hadoop-core.jar containing your modifications in
> common, i think the following works:
>
> 1. ant mvn-install in common project. This will create hadoop-core.jar and
> installs it in the local m2 repository.
> 2. ant test -Dresolvers=internal in mapreduce project. This will make sure
> that the hadoop-core.jar is taken from local m2 repository.
>
> Hope this helps.
>
> -Chaitanya.
>
> On Thu, Mar 18, 2010 at 11:53 AM, Rodrigo Schmidt <rodrigo@facebook.com>wrote:
>
>>
>> I tried that but my logs still didn't show up. To test if it really worked,
>> I disabled my Internet connection and tried it, which failed. Did I miss
>> anything?
>>
>>
>> Output below:
>>
>>
>> $ant -Doffline=yes test
>> Buildfile: build.xml
>>
>> ivy-download:
>>
>> ivy-init-dirs:
>>
>> ivy-probe-antlib:
>>
>> ivy-init-antlib:
>>
>> ivy-init:
>> [ivy:configure] :: Ivy 2.0.0-rc2 - 20081028224207 ::
>> http://ant.apache.org/ivy/ ::
>> :: loading settings :: file =
>> /Users/rschmidt/workspace/hadoop-mapreduce-trunk/ivy/ivysettings.xml
>>
>> ivy-resolve-common:
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve] You probably access the destination server through a proxy
>> server that is not well configured.
>> [ivy:resolve]
>> [ivy:resolve] :: problems summary ::
>> [ivy:resolve] :::: WARNINGS
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
>> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core/0.22.0
>> -SNAPSHOT/maven-metadata.xml>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
>> HOT/hadoop-core-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPS
>> HOT/hadoop-core-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-core/0.22.0-SNAPS%0AHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]         module not found:
>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT
>> [ivy:resolve]     ==== apache-snapshot: tried
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-co
>> re/0.22.0-SNAPSHOT/hadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     ==== maven2: tried
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
>> hadoop-core-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT!hadoop-core.jar:
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core/0.22.0-SNAPSHOT/
>> hadoop-core-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-core/0.22.0-SNAPSHOT/%0Ahadoop-core-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository
>> .apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-core-t
>> est/0.22.0-SNAPSHOT/maven-metadata.xml>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<
>> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
>> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>>
>> hadoop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<
>> https://repository.apache.org/content/repositories/snapshots/org/apache/%0Aha
>> doop/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/o
>> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
>> SNAPSHOT.pom>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-
>> SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/o
>> rg/apache/hadoop/hadoop-core-test/0.22.0-%0ASNAPSHOT/hadoop-core-test-0.22.0-
>> SNAPSHOT.jar>
>> [ivy:resolve]         module not found:
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT
>> [ivy:resolve]     ==== apache-snapshot: tried
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http
>> s://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
>> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http
>> s://repository.apache.org/content/repositories/snapshots/org/apache/hado%0Aop
>> /hadoop-core-test/0.22.0-SNAPSHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     ==== maven2: tried
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/a
>> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
>> SHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT!hadoop-core-test.jar:
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-core-test/0.22.0-SNAP
>> SHOT/hadoop-core-test-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/a
>> pache/hadoop/hadoop-core-test/0.22.0-SNAP%0ASHOT/hadoop-core-test-0.22.0-SNAP
>> SHOT.jar>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/maven-metadata.xml<https://repository.apac
>> he.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoop-hdfs/0.22.0
>> -SNAPSHOT/maven-metadata.xml>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repository.apache.org not found.
>> url=
>> https://repository.apache.org/content/repositories/snapshots/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<https://re
>> pository.apache.org/content/repositories/snapshots/org/apache/%0Ahadoop/hadoo
>> p-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]     Host repo1.maven.org not found.
>> url=
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPS
>> HOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/
>> hadoop/hadoop-hdfs/0.22.0-SNAPS%0AHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]         module not found:
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT
>> [ivy:resolve]     ==== apache-snapshot: tried
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar<https://reposi
>> tory.apache.org/content/repositories/snapshots/org/apache/hado%0Aop/hadoop-hd
>> fs/0.22.0-SNAPSHOT/hadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]     ==== maven2: tried
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
>> hadoop-hdfs-0.22.0-SNAPSHOT.pom<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.pom>
>> [ivy:resolve]       -- artifact
>> org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT!hadoop-hdfs.jar:
>> [ivy:resolve]
>>
>> http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.22.0-SNAPSHOT/
>> hadoop-hdfs-0.22.0-SNAPSHOT.jar<http://repo1.maven.org/maven2/org/apache/hado
>> op/hadoop-hdfs/0.22.0-SNAPSHOT/%0Ahadoop-hdfs-0.22.0-SNAPSHOT.jar>
>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]         ::          UNRESOLVED DEPENDENCIES         ::
>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]         :: org.apache.hadoop#hadoop-core;0.22.0-SNAPSHOT: not
>> found
>> [ivy:resolve]         ::
>> org.apache.hadoop#hadoop-core-test;0.22.0-SNAPSHOT:
>> not found
>> [ivy:resolve]         :: org.apache.hadoop#hadoop-hdfs;0.22.0-SNAPSHOT: not
>> found
>> [ivy:resolve]         ::::::::::::::::::::::::::::::::::::::::::::::
>> [ivy:resolve]
>> [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>>
>>
>> On 3/17/10 11:14 PM, "Amar Kamat" <amarrk@yahoo-inc.com> wrote:
>>
>>> Use -Doffline=yes switch while running the tests.
>>> Amar
>>>
>>> On 3/18/10 11:32 AM, "Rodrigo Schmidt" <rodrigo@facebook.com> wrote:
>>>
>>>
>>>
>>> How do I disable the ivy download to run unit tests? I need to replace
>>> hadoop-core.jar for one I created with some extra logging to debug a unit
>> test
>>> for mapreduce.
>>>
>>> Right now I'm just erasing some lines from build.xml, but I'm quite sure
>> there
>>> is an easier way to do that.
>>>
>>> Thanks,
>>> Rodrigo
>>>
>>
>>


Mime
View raw message