Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 40C1910FB4 for ; Sat, 10 Jan 2015 18:38:39 +0000 (UTC) Received: (qmail 72694 invoked by uid 500); 10 Jan 2015 18:38:34 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 72637 invoked by uid 500); 10 Jan 2015 18:38:34 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 72626 invoked by uid 99); 10 Jan 2015 18:38:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Jan 2015 18:38:34 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ekoontz@hiro-tan.org designates 104.187.107.145 as permitted sender) Received: from [104.187.107.145] (HELO hiro.hiro-tan.org) (104.187.107.145) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Jan 2015 18:38:30 +0000 Received: from Eugene-Koontzs-Mac-Pro.local (WR702N [192.168.1.67]) (Authenticated sender: ekoontz) by hiro.hiro-tan.org (Postfix) with ESMTPSA id AF580BA2699 for ; Sat, 10 Jan 2015 10:38:09 -0800 (PST) Message-ID: <54B17196.4040107@hiro-tan.org> Date: Sat, 10 Jan 2015 10:38:14 -0800 From: Eugene Koontz User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: user@giraph.apache.org Subject: Re: Compiling Giraph for Hadoop 2.5.x and 2.6.0 -- SASL_PROPS variable error References: <9CCB7954-7267-4E13-94C8-37A5F2C45C78@icloud.com> <54AF7F80.6070803@hiro-tan.org> In-Reply-To: <54AF7F80.6070803@hiro-tan.org> Content-Type: multipart/alternative; boundary="------------070301030705090403090309" X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------070301030705090403090309 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Allesio and Eli, Compiling with "mvn -Phadoop_yarn -Dhadoop.version=2.6.0 clean" will avoid the below "SASL_PROPS" compilation error if you remove the STATIC_SASL_SYMBOL from the munge.symbols of the hadoop_yarn profile as follows: diff --git a/pom.xml b/pom.xml index cf0e1f9..8c2a561 100644 --- a/pom.xml +++ b/pom.xml @@ -1194,7 +1194,7 @@ under the License. SET_HADOOP_VERSION_USING_MVN_DASH_D_OPTION - PURE_YARN,STATIC_SASL_SYMBOL + PURE_YARN true true In other words, when compiling Giraph against newer releases of Hadoop, there is no need for this munge symbol. The distinction between newer and older seems to be release 2.4.0 of Hadoop, as given here: https://issues.apache.org/jira/browse/HADOOP-10221 " Add a plugin to specify SaslProperties for RPC protocol based on connection properties". It seems like we need to add some additional profiles to make the pre-2.4 Hadoop (which requires the munge symbol STATIC_SASL_SYMBOL) and newer (which should not). -Eugene On 1/8/15, 11:13 PM, Eugene Koontz wrote: > Hi Alessio, > > I am able to reproduce your problem: > > https://gist.github.com/ekoontz/7dbaaf6218abb4fd7832 > > I'll try building Hadoop 2.6.0 and getting Giraph to work with it. > > -Eugene > > > On 1/8/15, 10:55 AM, Eli Reisman wrote: >> This looks like a munge symbol that needs to be added to the >> hadoop_yarn profile in the pom.xml. I'm thinking this is an issue a >> couple people have been having on 2.5 and 2.6 trying to build the >> hadoop_yarn profile? >> >> On Thu, Dec 4, 2014 at 1:01 PM, Dr. Alessio Arleo >> > wrote: >> >> Hello everybody >> >> I am trying to compile Giraph release-1.1 for Hadoop 2.5.x and >> Hadoop 2.6.0 with Maven profile hadoop_yarn. It works fine up to >> Hadoop 2.4.1, but when trying with a newer version of Hadoop the >> following error comes up. I am working with jdk 1.7 and Maven 3.2.1. >> ST >> [ERROR] COMPILATION ERROR : >> [INFO] ------------------------------------------------------------- >> [ERROR] >> /home/hadoop/git/giraph/1.1/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyClient.java:[84,68] >> cannot find symbol >> symbol: variable SASL_PROPS >> location: class org.apache.hadoop.security.SaslRpcServer >> [ERROR] >> /home/hadoop/git/giraph/1.1/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyServer.java:[105,62] >> cannot find symbol >> symbol: variable SASL_PROPS >> location: class org.apache.hadoop.security.SaslRpcServer >> >> Do you have any suggestions? Any would be much appreciated :) >> >> Kind regards, >> Alessio >> >> > --------------070301030705090403090309 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi Allesio and Eli,

Compiling with "mvn -Phadoop_yarn -Dhadoop.version=2.6.0 clean" will avoid the below "SASL_PROPS" compilation error if you remove the STATIC_SASL_SYMBOL from the munge.symbols of the hadoop_yarn profile as follows:

diff --git a/pom.xml b/pom.xml
index cf0e1f9..8c2a561 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1194,7 +1194,7 @@ under the License.
       </modules>
       <properties>
         <hadoop.version>SET_HADOOP_VERSION_USING_MVN_DASH_D_OPTION</hadoop.version>
-        <munge.symbols>PURE_YARN,STATIC_SASL_SYMBOL</munge.symbols>
+        <munge.symbols>PURE_YARN</munge.symbols>
         
         <project.enforcer.skip>true</project.enforcer.skip>
         <giraph.maven.dependency.plugin.skip>true</giraph.maven.dependency.plugin.skip>

In other words, when compiling Giraph against newer releases of Hadoop, there is no need for this munge symbol.

The distinction between newer and older seems to be release 2.4.0 of Hadoop, as given here:

https://issues.apache.org/jira/browse/HADOOP-10221 " Add a plugin to specify SaslProperties for RPC protocol based on connection properties".

It seems like we need to add some additional profiles to make the pre-2.4 Hadoop (which requires the munge symbol STATIC_SASL_SYMBOL) and newer (which should not).

-Eugene

On 1/8/15, 11:13 PM, Eugene Koontz wrote:

Hi Alessio,

I am able to reproduce your problem:

https://gist.github.com/ekoontz/7dbaaf6218abb4fd7832

I'll try building Hadoop 2.6.0 and getting Giraph to work with it.

-Eugene

On 1/8/15, 10:55 AM, Eli Reisman wrote:

This looks like a munge symbol that needs to be added to the hadoop_yarn profile in the pom.xml. I'm thinking this is an issue a couple people have been having on 2.5 and 2.6 trying to build the hadoop_yarn profile?

On Thu, Dec 4, 2014 at 1:01 PM, Dr. Alessio Arleo <ingarleo@icloud.com> wrote:

Hello everybody

I am trying to compile Giraph release-1.1 for Hadoop 2.5.x and Hadoop 2.6.0 with Maven profile hadoop_yarn. It works fine up to Hadoop 2.4.1, but when trying with a newer version of Hadoop the following error comes up. I am working with jdk 1.7 and Maven 3.2.1.
ST
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/hadoop/git/giraph/1.1/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyClient.java:[84,68] cannot find symbol
symbol: variable SASL_PROPS
location: class org.apache.hadoop.security.SaslRpcServer
[ERROR] /home/hadoop/git/giraph/1.1/giraph-core/target/munged/main/org/apache/giraph/comm/netty/SaslNettyServer.java:[105,62] cannot find symbol
symbol: variable SASL_PROPS
location: class org.apache.hadoop.security.SaslRpcServer

Do you have any suggestions? Any would be much appreciated :)

Kind regards,
Alessio

--------------070301030705090403090309--