accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Molek <mmo...@clearedgeit.com>
Subject Tests failing when building 1.5.0 source with -Dhadoop.profile=2.0
Date Wed, 22 Jan 2014 18:52:25 GMT
I'm having trouble building accumulo 1.5.0 with the hadoop 2.0 profile. I'm
using the source release accumulo-1.5.0-src.tar.gz from
http://accumulo.apache.org/downloads/ . I'm building on Centos 6.3 with
oracle java 1.7 and maven 3.0.5.

I think I'm running into two separate issues.

First, when building with the default hadoop version for the 2.0 profile
with this command: 'mvn -Dhadoop.profile=2.0
-Dmaven.test.failure.ignore=true clean install'

I get two test failures.

testFileMonitor(org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest)
 Time elapsed: 0.385 sec  <<< ERROR!
testGetClass(org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest)
 Time elapsed: 0.084 sec  <<< ERROR!

Both errors have the same cause:

java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCrc32.nativeVerifyChunkedSums(IILjava/nio/ByteBuffer;ILjava/nio/ByteBuffer;IILjava/lang/String;J)V
        at
org.apache.hadoop.util.NativeCrc32.nativeVerifyChunkedSums(Native Method)
        at
org.apache.hadoop.util.NativeCrc32.verifyChunkedSums(NativeCrc32.java:57)
        at
org.apache.hadoop.util.DataChecksum.verifyChunkedSums(DataChecksum.java:291)
        at
org.apache.hadoop.hdfs.RemoteBlockReader2.readNextPacket(RemoteBlockReader2.java:187)
        at
org.apache.hadoop.hdfs.RemoteBlockReader2.read(RemoteBlockReader2.java:122)
        at
org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:542)
        at
org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:594)
        at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:648)
        at
org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
        at java.io.DataInputStream.read(DataInputStream.java:149)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at
org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:100)
        at java.io.FilterInputStream.read(FilterInputStream.java:107)
        at org.apache.commons.vfs2.FileUtil.writeContent(FileUtil.java:85)
        at org.apache.commons.vfs2.FileUtil.copyContent(FileUtil.java:114)
        at
org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:1053)
        at
org.apache.commons.vfs2.impl.DefaultFileReplicator.replicateFile(DefaultFileReplicator.java:249)
        at
org.apache.commons.vfs2.provider.AbstractFileSystem.doReplicateFile(AbstractFileSystem.java:467)
        at
org.apache.commons.vfs2.provider.AbstractFileSystem.replicateFile(AbstractFileSystem.java:423)
        at
org.apache.commons.vfs2.provider.zip.ZipFileSystem.<init>(ZipFileSystem.java:61)
        at
org.apache.commons.vfs2.provider.jar.JarFileSystem.<init>(JarFileSystem.java:50)
        at
org.apache.commons.vfs2.provider.jar.JarFileProvider.doCreateFileSystem(JarFileProvider.java:82)
        at
org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.createFileSystem(AbstractLayeredFileProvider.java:89)
        at
org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:914)
        at
org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:933)
        at
org.apache.commons.vfs2.impl.VFSClassLoader.addFileObjects(VFSClassLoader.java:153)
        at
org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:116)
        at
org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:98)
        at
org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest.setup(VfsClassLoaderTest.java:58)
        ... and a bunch more

Second problem: If I try to build with a more recent hadoop release
(2.1.0-beta and up) command: 'mvn -Dhadoop.profile=2.0
-Dhadoop.version=2.1.0-beta -Dmaven.test.failure.ignore=true clean install'

I end up with 140+ test errors. Many are like this:

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
        at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
        at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
        at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:234)
        at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:213)
        at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:662)
        at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:570)
        at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2586)
        at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2578)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2444)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:363)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:165)
        at
org.apache.accumulo.core.client.mock.MockInstance.getDefaultFileSystem(MockInstance.java:69)
        at
org.apache.accumulo.core.client.mock.MockInstance.<init>(MockInstance.java:60)
        at
org.apache.accumulo.server.util.CloneTest.testMerge(CloneTest.java:344)
        ... 25 more (my truncation)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
        ... 39 more
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
        at
org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
Method)
        at
org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
        at
org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
        ... 44 more


Lastly, I can build with no errors if I use the default hadoop-1.0 profile.

Both of my problems seem to boil down to UnsatisfiedLinkErrors related to
the underlying Hadoop dependencies. I can't find much information to help
resolve this other than some general java related discussions saying that
UnsatisfiedLinkErrors usually result from missing native libraries. Can
anyone point me in the right direction? Thanks!

Mime
View raw message