giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: Compiling Giraph for Hadoop 2.5.x and 2.6.0 -- SASL_PROPS variable error
Date Wed, 28 Jan 2015 08:30:36 GMT
This makes sense thanks Eugene! I would advise keeping the YARN profile for
now especially if the fix is this minor. I do feel like the advantages and
opportunities for Giraph once development focuses on a post Hadoop1 world
will be huge. Seems like more companies are making the upgrade every day,
mine included. Now that we''re on YARN I'm hoping to run Giraph again
myself.

Just me though and I can certainly see the other side of the argument too.
On Jan 12, 2015 8:05 AM, "Claudio Martella" <claudio.martella@gmail.com>
wrote:

> I see more and more people getting into this. I guess whether we should
> add the fix to the pure_yarn profile by default, as it feel it's going to
> stay.
> Ideas?
>
> On Sat, Jan 10, 2015 at 7:38 PM, Eugene Koontz <ekoontz@hiro-tan.org>
> wrote:
>
>>  Hi Allesio and Eli,
>>
>> Compiling with "mvn -Phadoop_yarn -Dhadoop.version=2.6.0 clean" will
>> avoid the below "SASL_PROPS" compilation error if you remove the
>> STATIC_SASL_SYMBOL from the munge.symbols of the hadoop_yarn profile as
>> follows:
>>
>> diff --git a/pom.xml b/pom.xml
>> index cf0e1f9..8c2a561 100644
>> --- a/pom.xml
>> +++ b/pom.xml
>> @@ -1194,7 +1194,7 @@ under the License.
>>        </modules>
>>        <properties>
>>
>> <hadoop.version>SET_HADOOP_VERSION_USING_MVN_DASH_D_OPTION</hadoop.version>
>> -        <munge.symbols>PURE_YARN,STATIC_SASL_SYMBOL</munge.symbols>
>> +        <munge.symbols>PURE_YARN</munge.symbols>
>>          <!-- TODO: add these checks eventually -->
>>          <project.enforcer.skip>true</project.enforcer.skip>
>>
>> <giraph.maven.dependency.plugin.skip>true</giraph.maven.dependency.plugin.skip>
>>
>> In other words, when compiling Giraph against newer releases of Hadoop,
>> there is no need for this munge symbol.
>>
>> The distinction between newer and older seems to be release 2.4.0 of
>> Hadoop, as given here:
>>
>> https://issues.apache.org/jira/browse/HADOOP-10221 " Add a plugin to
>> specify SaslProperties for RPC protocol based on connection properties".
>>
>>  It seems like we need to add some additional profiles to make the
>> pre-2.4 Hadoop (which requires the munge symbol STATIC_SASL_SYMBOL) and
>> newer (which should not).
>>
>> -Eugene
>>
>>
>> On 1/8/15, 11:13 PM, Eugene Koontz wrote:
>>
>> Hi Alessio,
>>
>> I am able to reproduce your problem:
>>
>> https://gist.github.com/ekoontz/7dbaaf6218abb4fd7832
>>
>> I'll try building Hadoop 2.6.0 and getting Giraph to work with it.
>>
>> -Eugene
>>
>>
>> On 1/8/15, 10:55 AM, Eli Reisman wrote:
>>
>> This looks like a munge symbol that needs to be added to the hadoop_yarn
>> profile in the pom.xml. I'm thinking this is an issue a couple people have
>> been having on 2.5 and 2.6 trying to build the hadoop_yarn profile?
>>
>> On Thu, Dec 4, 2014 at 1:01 PM, Dr. Alessio Arleo <ingarleo@icloud.com>
>> wrote:
>>
>>> Hello everybody
>>>
>>> I am trying to compile Giraph release-1.1 for Hadoop 2.5.x and Hadoop
>>> 2.6.0 with Maven profile hadoop_yarn. It works fine up to Hadoop 2.4.1, but
>>> when trying with a newer version of Hadoop the following error comes up. I
>>> am working with jdk 1.7 and Maven 3.2.1.
>>> ST
>>> [ERROR] COMPILATION ERROR :
>>> [INFO] -------------------------------------------------------------
>>> [ERROR]
>>> /home/hadoop/git/giraph/1.1/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyClient.java:[84,68]
>>> cannot find symbol
>>>   symbol:   variable SASL_PROPS
>>>   location: class org.apache.hadoop.security.SaslRpcServer
>>> [ERROR]
>>> /home/hadoop/git/giraph/1.1/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyServer.java:[105,62]
>>> cannot find symbol
>>>   symbol:   variable SASL_PROPS
>>>   location: class org.apache.hadoop.security.SaslRpcServer
>>>
>>> Do you have any suggestions? Any would be much appreciated :)
>>>
>>> Kind regards,
>>> Alessio
>>>
>>
>>
>>
>>
>
>
> --
>    Claudio Martella
>
>

Mime
View raw message