avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tony Reix (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1567) Avro java tools tests fail with IBM JVM
Date Wed, 27 Aug 2014 15:37:58 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112351#comment-14112351
] 

Tony Reix commented on AVRO-1567:
---------------------------------

Hummmm Since I'm using top build.sh, I did not do mvn package.
Anyway, I've tried, on my RHEL6.5/x86_64 machine, with IBM JVM .

I'm there:
avro-src-1.7.4/lang/java   (Avro 1.7.4 version that I got from Avro GitHub).

I did:
rm -rf /root/.m2/repository/org/apache/avro
 mvn clean
find ~/.m2 -name "avro*"  ---> Nothing
mvn -Phadoop2 -Dhadoop.version=2 package -DskipTests
  [INFO] Apache Avro Mapred API ............................ FAILURE [  3.990 s]
 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
(default-compile) on project avro-mapred: Compilation failure
  [ERROR] /home/reixt/AVRO/avro-src-1.7.4/lang/java/mapred/src/main/java/org/apache/avro/mapred/tether/TetheredProcess.java:[157,21]
error: method captureOutAndError in class TaskLog cannot be applied to given types;

find ~/.m2 -name "avro*"
/root/.m2/repository/org/apache/avro
/root/.m2/repository/org/apache/avro/avro-toplevel
/root/.m2/repository/org/apache/avro/avro-toplevel/1.7.3/avro-toplevel-1.7.3.pom
...........

So, it did download some Avro jar files from the Web.


Doing the same (after mvn clean and rm -rf ~/.m2/....avro) with install instead of package:
   mvn -Phadoop2 -Dhadoop.version=2 install -DskipTests
I also got Avro 1.7.4 jars being downloaded in my .m2 repository :

$ find ~/.m2 -name "avro*"
/root/.m2/repository/org/apache/avro
/root/.m2/repository/org/apache/avro/avro-compiler
/root/.m2/repository/org/apache/avro/avro-compiler/1.7.4/avro-compiler-1.7.4.pom
/root/.m2/repository/org/apache/avro/avro-compiler/1.7.4/avro-compiler-1.7.4.jar
....

> Avro java tools tests fail with IBM JVM
> ---------------------------------------
>
>                 Key: AVRO-1567
>                 URL: https://issues.apache.org/jira/browse/AVRO-1567
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.4, 1.7.7
>         Environment: RHEL 6.5 on x86_64
> IBM JVM 7.1.1.1
> HADOOP 2.4.1
>            Reporter: Tony Reix
>            Priority: Blocker
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When using IBM JVM, compared to Oracle JVM, 25 of the Avro Tools tests fail.
> This is due to Avro using Hadoop which uses class:
>    org/apache/hadoop/security/UserGroupInformation.java
> which makes use of:
>    com.sun.security.auth.module.UnixLoginModule
> which does not exist in IBM JVM.
> Instead there is the class:
>    com.ibm.security.auth.module.LinuxLoginModule
> that can be used in UserGroupInformation.java if the JVM is IBM.
> With a IBM-JVM patched version of Hadoop that takes care of the kind of JVM, these 25
Avro Java Tools tests still fail because the pom.xml file of:
>     lang/java/tools/
> says unconditionnaly (starting line 146 in Avro 1.7.7) :
>       <dependency>
>       <groupId>org.apache.hadoop</groupId>
>       <artifactId>hadoop-core</artifactId>
> Using:
>   mvn -Phadoop2 -Dhadoop.version=2 test
> is of no help.
> In fact, hadoop-core exists only for old Hadoop versions (here version 0.20.205.0 is
used by Avro), and not for Hadoop 2.4.1 .
> Replacing hadoop-core by hadoop-client in lang/java/tools/pom.xml file does fix the issue,
as a work-around.
> However, a more rigorous solution is required, like it is done in lang/java/mapred/pom.xm
, where hadoop-core is associated with hadoop1 and hadoop-client is associated with hadoop2
.
> I'm not an expert of Maven/pom.xml, and the pom.xml file of tools contains <exclusions>
tags I have no idea. So, I'm not sure I can provide a correct patch.
> I guess that a Maven/pom.xml expert should be able to fix this in some minutes, plus
testing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message