accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Newton <eric.new...@gmail.com>
Subject Re: Tests failing when building 1.5.0 source with -Dhadoop.profile=2.0
Date Wed, 22 Jan 2014 18:57:06 GMT
Be sure to "mvn clean" when switching between hadoop versions.

-Eric



On Wed, Jan 22, 2014 at 1:52 PM, Matthew Molek <mmolek@clearedgeit.com>wrote:

> I'm having trouble building accumulo 1.5.0 with the hadoop 2.0 profile.
> I'm using the source release accumulo-1.5.0-src.tar.gz from
> http://accumulo.apache.org/downloads/ . I'm building on Centos 6.3 with
> oracle java 1.7 and maven 3.0.5.
>
> I think I'm running into two separate issues.
>
> First, when building with the default hadoop version for the 2.0 profile
> with this command: 'mvn -Dhadoop.profile=2.0
> -Dmaven.test.failure.ignore=true clean install'
>
> I get two test failures.
>
> testFileMonitor(org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest)
>  Time elapsed: 0.385 sec  <<< ERROR!
> testGetClass(org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest)
>  Time elapsed: 0.084 sec  <<< ERROR!
>
> Both errors have the same cause:
>
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCrc32.nativeVerifyChunkedSums(IILjava/nio/ByteBuffer;ILjava/nio/ByteBuffer;IILjava/lang/String;J)V
>         at
> org.apache.hadoop.util.NativeCrc32.nativeVerifyChunkedSums(Native Method)
>         at
> org.apache.hadoop.util.NativeCrc32.verifyChunkedSums(NativeCrc32.java:57)
>         at
> org.apache.hadoop.util.DataChecksum.verifyChunkedSums(DataChecksum.java:291)
>         at
> org.apache.hadoop.hdfs.RemoteBlockReader2.readNextPacket(RemoteBlockReader2.java:187)
>         at
> org.apache.hadoop.hdfs.RemoteBlockReader2.read(RemoteBlockReader2.java:122)
>         at
> org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:542)
>         at
> org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:594)
>         at
> org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:648)
>         at
> org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
>         at java.io.DataInputStream.read(DataInputStream.java:149)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
>         at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:100)
>         at java.io.FilterInputStream.read(FilterInputStream.java:107)
>         at org.apache.commons.vfs2.FileUtil.writeContent(FileUtil.java:85)
>         at org.apache.commons.vfs2.FileUtil.copyContent(FileUtil.java:114)
>         at
> org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:1053)
>         at
> org.apache.commons.vfs2.impl.DefaultFileReplicator.replicateFile(DefaultFileReplicator.java:249)
>         at
> org.apache.commons.vfs2.provider.AbstractFileSystem.doReplicateFile(AbstractFileSystem.java:467)
>         at
> org.apache.commons.vfs2.provider.AbstractFileSystem.replicateFile(AbstractFileSystem.java:423)
>         at
> org.apache.commons.vfs2.provider.zip.ZipFileSystem.<init>(ZipFileSystem.java:61)
>         at
> org.apache.commons.vfs2.provider.jar.JarFileSystem.<init>(JarFileSystem.java:50)
>         at
> org.apache.commons.vfs2.provider.jar.JarFileProvider.doCreateFileSystem(JarFileProvider.java:82)
>         at
> org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.createFileSystem(AbstractLayeredFileProvider.java:89)
>         at
> org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:914)
>         at
> org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:933)
>         at
> org.apache.commons.vfs2.impl.VFSClassLoader.addFileObjects(VFSClassLoader.java:153)
>         at
> org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:116)
>         at
> org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:98)
>         at
> org.apache.accumulo.start.classloader.vfs.providers.VfsClassLoaderTest.setup(VfsClassLoaderTest.java:58)
>         ... and a bunch more
>
> Second problem: If I try to build with a more recent hadoop release
> (2.1.0-beta and up) command: 'mvn -Dhadoop.profile=2.0
> -Dhadoop.version=2.1.0-beta -Dmaven.test.failure.ignore=true clean install'
>
> I end up with 140+ test errors. Many are like this:
>
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
>         at org.apache.hadoop.security.Groups.<init>(Groups.java:55)
>         at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
>         at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:234)
>         at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:213)
>         at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:662)
>         at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:570)
>         at
> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2586)
>         at
> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2578)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2444)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:363)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:165)
>         at
> org.apache.accumulo.core.client.mock.MockInstance.getDefaultFileSystem(MockInstance.java:69)
>         at
> org.apache.accumulo.core.client.mock.MockInstance.<init>(MockInstance.java:60)
>          at
> org.apache.accumulo.server.util.CloneTest.testMerge(CloneTest.java:344)
>         ... 25 more (my truncation)
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
>         ... 39 more
> Caused by: java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V
>         at
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native
> Method)
>         at
> org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49)
>         at
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.<init>(JniBasedUnixGroupsMappingWithFallback.java:38)
>         ... 44 more
>
>
> Lastly, I can build with no errors if I use the default hadoop-1.0 profile.
>
> Both of my problems seem to boil down to UnsatisfiedLinkErrors related to
> the underlying Hadoop dependencies. I can't find much information to help
> resolve this other than some general java related discussions saying that
> UnsatisfiedLinkErrors usually result from missing native libraries. Can
> anyone point me in the right direction? Thanks!
>
>

Mime
View raw message