giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Negro <alenegr...@yahoo.it>
Subject Re: Compiling Giraph with Hadoop 2.5.1 & YARN
Date Mon, 27 Oct 2014 22:02:27 GMT
Hi Nicola,
how are you running the test? Could you help me to test it on yarn?
I would like to see the error.

Thanks,
Alessandro

Il giorno 27/ott/2014, alle ore 17:34, Nicola Vitucci <nicola.vitucci@gmail.com> ha
scritto:

> Hi Alessandro,
> 
> I can now complete the compiling process and get the full .jar. I can't
> get it working right away, but I think it's more an Hadoop problem than
> a Giraph problem (can't get past "superstep -1" in the PageRank
> example). I'll keep you updated.
> 
> Thanks again,
> 
> Nicola
> 
> Il 27/10/2014 17:15, Alessandro Negro ha scritto:
>> Hi Nicola,
>> I have just fixed this other issue, but I’m not sure that it will work
>> completely.
>> After I checked the code I notice that the issue is related to the munge
>> symbol STATIC_SASL_SYMBOL.
>> Try removing STATIC_SASL_SYMBOL from the munge symbols list in the
>> hadoop_yarn profile.
>> 
>> My profile was:
>> 
>> <munge.symbols>PURE_YARN,STATIC_SASL_SYMBOL</munge.symbols>
>> 
>> now it is:
>> 
>> <munge.symbols>PURE_YARN</munge.symbols>
>> 
>> 
>> Now I’m able to continue. As I said I’m not sure that this will produce
>> a working jar.
>> I checked into the code and it seems so. If you can please test and let
>> me know.
>> 
>> Cheers,
>> Alessandro
>> 
>> 
>> Il giorno 27/ott/2014, alle ore 15:52, Nicola Vitucci
>> <nicola.vitucci@gmail.com <mailto:nicola.vitucci@gmail.com>> ha scritto:
>> 
>>> Hi Alessandro,
>>> 
>>> I can confirm that after your addition I'm able to build Giraph for
>>> Hadoop 2.4.0, so this works:
>>> 
>>> mvn package -DskipTests -Dhadoop.version=2.4.0 -Phadoop_yarn clean install
>>> 
>>> Do you know whether there are any plans to support Hadoop 2.5.x too?
>>> 
>>> Thank you very much,
>>> 
>>> Nicola
>>> 
>>> Il 27/10/2014 15:38, Alessandro Negro ha scritto:
>>>> Hi Nicola,
>>>> it’s true I got the same error but also before changing the plugin
>>>> configuration.
>>>> I was able to compile only for version 2.4.0. I tested with 2.5.0 as
>>>> well.
>>>> 
>>>> I tested everything on the main trunk.
>>>> 
>>>> Cheers,
>>>> Alessandro
>>>> 
>>>> 
>>>> 
>>>> Il giorno 27/ott/2014, alle ore 15:00, Nicola Vitucci
>>>> <nicola.vitucci@gmail.com <mailto:nicola.vitucci@gmail.com>>
ha scritto:
>>>> 
>>>>> Hi Alessandro,
>>>>> 
>>>>> thanks for the reply. In order to start fresh, before adding the line
>>>>> you suggested I re-cloned the repository and tried to compile from trunk
>>>>> using the same command:
>>>>> 
>>>>> mvn package -DskipTests -Dhadoop.version=2.5.1 -Phadoop_yarn clean
>>>>> install
>>>>> 
>>>>> I get an error (on the variable SASL_PROPS) that I could get rid of
>>>>> before by applying the patch GIRAPH-947, but now the patch is not
>>>>> applicable any longer, so I can't even get to the stage where the
>>>>> 'munged' directory was the problem. Can you tell me which branch have
>>>>> you used and what is the exact sequence of changes that you applied to
>>>>> get Giraph compiled?
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Nicola
>>>>> 
>>>>> Il 26/10/2014 23:34, Alessandro Negro ha scritto:
>>>>>> Hi Nicola,
>>>>>> I was able to fix this issue adding the following line into the
>>>>>> munge maven plugin:
>>>>>> 
>>>>>> <mungedDirectory>${project.build.directory}/munged</mungedDirectory>
>>>>>> 
>>>>>> Now it is:
>>>>>> 
>>>>>> <plugin>
>>>>>>        <groupId>org.sonatype.plugins</groupId>
>>>>>>        <artifactId>munge-maven-plugin</artifactId>
>>>>>>        <version>1.0</version>
>>>>>>        <executions>
>>>>>>          <execution>
>>>>>>            <id>munge</id>
>>>>>>            <phase>generate-sources</phase>
>>>>>>            <goals>
>>>>>>              <goal>munge</goal>
>>>>>>            </goals>
>>>>>>            <configuration>
>>>>>>              <mungedDirectory>${project.build.directory}/munged</mungedDirectory>
>>>>>>              <symbols>${munge.symbols}</symbols>
>>>>>>            </configuration>
>>>>>>          </execution>
>>>>>>        </executions>
>>>>>>      </plugin>
>>>>>> 
>>>>>> I hope that this will help you.
>>>>>> 
>>>>>> Regards,
>>>>>> Alessandro


Mime
View raw message