giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: Failed to compile Giraph for Hadoop YARN
Date Mon, 15 Jul 2013 17:53:41 GMT
I think something else is wrong then, this same build works fine for me. I
think you need to do mvn targets "clean install" the first time, and
"compile" from then on?


On Mon, Jul 8, 2013 at 7:45 AM, Chui-Hui Chiu <cchiu1@tigers.lsu.edu> wrote:

> I run
>
> mvn -Dhadoop.version=2.0.3-alpha -Phadoop_yarn compile
>
> on the source code of the Giraph-1.1.0-SNAPSHOT, which is released on
> 05/21/2013, but still see the following message.
>
> =====
>
> [INFO] Executed tasks
> [INFO]
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building Apache Giraph Examples 1.1.0-SNAPSHOT
> [INFO]
> ------------------------------------------------------------------------
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Giraph Parent .............................. SUCCESS [1.335s]
> [INFO] Apache Giraph Core ................................ SUCCESS
> [41.165s]
> [INFO] Apache Giraph Examples ............................ FAILURE [0.122s]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 43.270s
> [INFO] Finished at: Mon Jul 08 09:37:16 CDT 2013
> [INFO] Final Memory: 43M/724M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal on project giraph-examples: Could not
> resolve dependencies for project
> org.apache.giraph:giraph-examples:jar:1.1.0-SNAPSHOT: Failure to find
> org.apache.giraph:giraph-core:jar:tests:1.1.0-SNAPSHOT in
> http://repo1.maven.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of central has
> elapsed or updates are forced -> [Help 1]
>
> =====
>
> When I apply the GIRAPH-688-1.patch, the patch utility reports that the
> patch has applied to the source code.  I just compile it.
>
>
> On Thu, Jul 4, 2013 at 11:56 AM, Eli Reisman <apache.mailbox@gmail.com>wrote:
>
>> Yes, glad this was brought up, I'll post to the site wiki with my
>> explicit apologies for not doing it sooner.
>>
>> I posted the most recent "correct" way to build pure-YARN Giraph against
>> any hadoop version on GIRAPH-688 here:
>>
>> https://issues.apache.org/jira/browse/GIRAPH-688
>>
>> The trick is, you have to put the -D mvn option first, and set the
>> "hadoop.version" Maven var. The versions as seen in "hadoop.version" are
>> -not- the same as the version profiles (-P option) that we build against.
>> So the correct format is:
>>
>> mvn -Dhadoop.version=2.0.3-alpha -Phadoop_yarn clean install
>>
>> if you want to just compile and not create our build products as in your
>> example, replace "clean install" with "compile". -P profile should be
>> "hadoop_yarn" for pure YARN builds, and "hadoop.version" appears as the
>> part of the hadoop version you see on the Apache Hadoop download site,
>> -after- the "hadoop-" part. So "hadoop-2.0.3-alpha" becomes for us
>> "-Dhadoop.version=2.0.3-alpha"
>>
>> To build Giraph so that it can run on a YARN-enabled Hadoop cluster that
>> -also- uses MapReduce v2 as the interface to the cluster (Giraph will still
>> think its running on MapReduce, not YARN) you might do this:
>>
>> mvn -Phadoop_2.0.3 clean install
>>
>> If you want to see how we label build profiles vs. "hadoop.version" in
>> Giraph builds, check our POM.xml files
>>
>> Thanks! Sorry for the confusion, this is a confusing thing since there
>> are 2 ways to "run Giraph on YARN" as detailed above.
>>
>>
>>
>>
>> On Wed, Jul 3, 2013 at 10:16 AM, Avery Ching <aching@apache.org> wrote:
>>
>>> Eli, any thoughts?
>>>
>>>
>>> On 7/3/13 9:27 AM, Chui-Hui Chiu wrote:
>>>
>>>> Hello,
>>>>
>>>> I tried to compile the Giraph-1.1.0-SNAPSHOT for hadoop_2.0.3 or
>>>> hadoop_yarn but all failed.
>>>>
>>>> The error message while the compile command is "mvn -Phadoop_yarn
>>>> compile" is
>>>> =====
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [INFO] Reactor Summary:
>>>> [INFO]
>>>> [INFO] Apache Giraph Parent .............................. SUCCESS
>>>> [1.320s]
>>>> [INFO] Apache Giraph Core ..............................**.. FAILURE
>>>> [12.508s]
>>>> [INFO] Apache Giraph Examples ............................ SKIPPED
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [INFO] BUILD FAILURE
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [INFO] Total time: 14.473s
>>>> [INFO] Finished at: Wed Jul 03 11:05:44 CDT 2013
>>>> [INFO] Final Memory: 14M/216M
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [ERROR] Failed to execute goal on project giraph-core: Could not
>>>> resolve dependencies for project org.apache.giraph:giraph-core:**jar:1.1.0-SNAPSHOT:
>>>> The following artifacts could not be resolved: org.apache.hadoop:hadoop-
>>>> **common:jar:SET_HADOOP_VERSION_**USING_MVN_DASH_D_OPTION,
>>>> org.apache.hadoop:hadoop-**mapreduce-client-common:jar:**
>>>> SET_HADOOP_VERSION_USING_MVN_**DASH_D_OPTION, org.apache.hadoop:hadoop-
>>>> **mapreduce-client-core:jar:SET_**HADOOP_VERSION_USING_MVN_DASH_**D_OPTION,
>>>> org.apache.hadoop:hadoop-yarn-**common:jar:SET_HADOOP_VERSION_**USING_MVN_DASH_D_OPTION,
>>>> org.apache.hadoop:hadoop-yarn-**server-common:jar:SET_HADOOP_**
>>>> VERSION_USING_MVN_DASH_D_**OPTION, org.apache.hadoop:hadoop-yarn-**
>>>> server-resourcemanager:jar:**SET_HADOOP_VERSION_USING_MVN_**DASH_D_OPTION,
>>>> org.apache.hadoop:hadoop-yarn-**server-nodemanager:jar:SET_**
>>>> HADOOP_VERSION_USING_MVN_DASH_**D_OPTION,
>>>> org.apache.hadoop:hadoop-yarn-**server-tests:jar:tests:SET_**
>>>> HADOOP_VERSION_USING_MVN_DASH_**D_OPTION: Could not find artifact
>>>> org.apache.hadoop:hadoop-**common:jar:SET_HADOOP_VERSION_**USING_MVN_DASH_D_OPTION
>>>> in central (http://repo1.maven.org/maven2**) -> [Help 1]
>>>> =====
>>>>
>>>>
>>>> The error message while the compile command is "mvn -Phadoop_2.0.3
>>>> compile" is
>>>> =====
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [INFO] Reactor Summary:
>>>> [INFO]
>>>> [INFO] Apache Giraph Parent .............................. SUCCESS
>>>> [12.695s]
>>>> [INFO] Apache Giraph Core ..............................**.. SUCCESS
>>>> [2:10.916s]
>>>> [INFO] Apache Giraph Examples ............................ FAILURE
>>>> [2.286s]
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [INFO] BUILD FAILURE
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [INFO] Total time: 2:26.530s
>>>> [INFO] Finished at: Wed Jul 03 11:03:25 CDT 2013
>>>> [INFO] Final Memory: 34M/348M
>>>> [INFO] ------------------------------**------------------------------**
>>>> ------------
>>>> [ERROR] Failed to execute goal on project giraph-examples: Could not
>>>> resolve dependencies for project org.apache.giraph:giraph-**examples:jar:1.1.0-SNAPSHOT:
>>>> Could not find artifact org.apache.giraph:giraph-core:**jar:tests:1.1.0-SNAPSHOT
>>>> in central (http://repo1.maven.org/maven2**) -> [Help 1]
>>>> =====
>>>>
>>>> Do I miss anything?
>>>>
>>>>
>>>> I also noticed that my Maven 3 download many files from the maven2
>>>> folder on a remote server with the following prompt.
>>>>
>>>> Downloading: http://repo1.maven.org/maven2/**org/apache/hadoop/.<http://repo1.maven.org/maven2/org/apache/hadoop/.>
>>>> ..
>>>>
>>>> Is this a problem?
>>>>
>>>>
>>>> Thanks,
>>>> Chui-hui
>>>>
>>>
>>>
>>
>

Mime
View raw message