giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manuel Lagang <manuellag...@gmail.com>
Subject Re: Compiling latest code for cdh4
Date Tue, 03 Dec 2013 01:38:08 GMT
I'm compiling against the latest trunk (Git
SHA: 37bc2c80564b45d7e4ce95db76f5411a6b8bdb3a).

Without any changes, the command:

mvn -Dmaven.test.skip=true -Phadoop_cdh4.1.2 clean install

gives this error:

[ERROR] Found a problem with the dependency
commons-httpclient:commons-httpclient
  Resolved version is 3.0.1
  Version 3.1 was expected by artifact: org.apache.hadoop:hadoop-common
[ERROR] Found a problem with the dependency commons-logging:commons-logging
  Resolved version is 1.0.4
  Version 1.1.1 was expected by artifacts: org.apache.hadoop:hadoop-client,
org.apache.hadoop:hadoop-common
[ERROR] Found a problem with the dependency org.apache.zookeeper:zookeeper
  Resolved version is 3.3.3
  Version 3.4.3-cdh4.1.2 was expected by artifacts:
org.apache.hadoop:hadoop-client, org.apache.hadoop:hadoop-common,
org.apache.hadoop:hadoop-mapreduce-client-common,
org.apache.hadoop:hadoop-mapreduce-client-hs,
org.apache.hadoop:hadoop-mapreduce-client-jobclient,
org.apache.hadoop:hadoop-mapreduce-client-shuffle

It seems there's some dependency versions in Giraph's POM that conflict
with the indirect dependencies of the same libraries from the hadoop
version. See https://github.com/apache/giraph/blob/trunk/pom.xml#L262-L297.
In particular, the versions of commons-httpclient:commons-httpclient,
commons-logging:commons-logging, and org.apache.zookeeper:zookeeper seem to
conflict.


On Mon, Dec 2, 2013 at 5:16 PM, Ameya Vilankar <ameya.vilankar@gmail.com>wrote:

> Are you compiling the release branch?
> Checkout the trunk branch of giraph and try compiling. It should work.
>
>
> On Mon, Dec 2, 2013 at 5:13 PM, Manuel Lagang <manuellagang@gmail.com>wrote:
>
>> Ameya, thanks for the reply. It seems that your suggestion is to
>> basically copy the 4.1.2 profile and point it to 4.2.0. I've seen similar
>> suggestions, so I've tried this before, but it doesn't get around the
>> compilation issues that I have. I'll have to do this anyway to get versions
>> as close as possible.
>>
>> I've gotten Giraph to build without errors on a cdh4 maven profile, but
>> I've had to disable some maven compile checks and modify some code to catch
>> and rewrap some missed checked exceptions. Was this necessary, or should
>> the cdh4 profile work as-is?
>>
>>
>> On Mon, Dec 2, 2013 at 4:45 PM, Ameya Vilankar <ameya.vilankar@gmail.com>wrote:
>>
>>>
>>> Check out this link:
>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_31.html
>>>
>>> You have to change the cloudera hadoop profile in your pom.xml file to
>>> make it work.
>>>
>>> In you hadoop profile for the cloudera distribution, do the following
>>> changes
>>>
>>>    1. Change this line(Line number 714):
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> <profile>
>>>
>>>       <id>hadoop_cdh4.1.2</id>
>>>
>>> to
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> <profile>
>>>
>>>       <id>hadoop_cdh4.4.0</id>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2. Change this line (Line number 716):
>>>
>>>
>>>
>>>
>>>
>>>
>>> <properties>
>>>
>>>         <hadoop.version>2.0.0-cdh4.1.2</hadoop.version>
>>>
>>>
>>>
>>>
>>>
>>>         <munge.symbols>HADOOP_1_SECRET_MANAGER</munge.symbols>
>>>
>>>
>>>
>>>
>>>
>>>
>>> to
>>>
>>>
>>> <properties>
>>>         <hadoop.version>2.0.0-cdh4.4.0</hadoop.version>
>>>
>>>
>>>
>>>
>>>         <munge.symbols>HADOOP_1_SECRET_MANAGER</munge.symbols>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Let me know if this works!!!
>>> Thanks,
>>> Ameya
>>>
>>>
>>>
>>> On Mon, Dec 2, 2013 at 12:58 PM, Manuel Lagang <manuellagang@gmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm trying to move my giraph code to a cluster using cdh4, so I would
>>>> to compile against that hadoop version. I tried using the cdh4.1.2 profile
>>>> when compiling against the latest 1.1.0-SNAPSHOT
>>>> (SHA: 37bc2c80564b45d7e4ce95db76f5411a6b8bdb3a) via this command:
>>>>
>>>> mvn -Dmaven.test.skip=true -Phadoop_cdh4.1.2 clean install
>>>>
>>>> I had to ignore any dependency version conflicts by changing 2 lines in
>>>> the pom.xml that now look like:
>>>>
>>>> <failBuildInCaseOfConflict>false</failBuildInCaseOfConflict>
>>>>
>>>> I still get compilation errors though:
>>>>
>>>> [ERROR]
>>>> /home/manuel/code/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[78,49]
>>>> unreported exception java.lang.InterruptedException; must be caught or
>>>> declared to be thrown
>>>> [ERROR]
>>>> /home/manuel/code/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[81,50]
>>>> unreported exception java.lang.InterruptedException; must be caught or
>>>> declared to be thrown
>>>>
>>>> Can anyone confirm if they can get the cdh4.1.2 profile to compile
>>>> as-is? The actual version I want to eventually compile for is cdh4.2.0,
>>>> does anyone have any tips on how to get Giraph working on that?
>>>>
>>>> Thanks,
>>>> Manuel Lagang
>>>>
>>>
>>>
>>
>

Mime
View raw message