accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Verkuylen <...@verkuylen.net>
Subject Re: vfs classpath could not replicate
Date Thu, 27 Sep 2018 22:27:59 GMT
Nothing in VFS except my jar file.

On Thu, Sep 27, 2018 at 8:50 PM Michael Wall <mjwall@gmail.com> wrote:

>  java.lang.ClassNotFoundException: IO Error loading class
>> org.apache.accumulo.tserver.compaction.DefaultCompactionStrategy
>
>
> Do you have your accumulo jars in VFS?  I would recommend against that,
> put them in ACCUMULO_HOME/lib.  Those should not be reloaded between
> restarts and makes a lot of extra calls against HDFS than are necessary.
>
> On Thu, Sep 27, 2018 at 2:36 PM Rob Verkuylen <rob@verkuylen.net> wrote:
>
>> Hmm 'jinfo <tabletserver_pid> | grep tmp'  shows:
>>
>> java.io.tmpdir = /tmp
>>
>>
>> "ls -hal /" shows 777 on /tmp
>>
>>
>> Grepping for 'tmp' in tserver startup:
>>
>> [zookeeper.ZooKeeper] INFO : Client environment:java.io.tmpdir=/tmp
>>
>> [server.Accumulo] INFO : general.vfs.cache.dir =
>> /tmp/accumulo-vfs-cache-accumulo
>>
>> [server.Accumulo] INFO : tserver.dir.memdump = /tmp
>>
>>
>> I'm guessing the relevant code to get the dir would be in:
>> AccumuloVFSClassLoader.java
>>
>> String cacheDirPath = AccumuloClassLoader.getAccumuloString(VFS_CACHE_DIR,
>> System.getProperty("java.io.tmpdir"));
>>
>> with
>>
>> public static final String VFS_CACHE_DIR = "general.vfs.cache.dir";
>>
>>
>> which then would be ' /tmp/accumulo-vfs-cache-accumulo', but it seems
>> this dir does not not need to exist and will be created. Could there be any
>> other reason this would pop up? Can these settings be overridden somewhere?
>>
>>
>>
>>
>> On Wed, Sep 26, 2018 at 10:17 PM Christopher <ctubbsii@apache.org> wrote:
>>
>>> It looks like the error is when vfs tries to create a temporary file in
>>> your tmpdir. I would check that your java.io.tmpdir points to a directory
>>> that exists and has the appropriate permissions for the user running the
>>> Accumulo process.
>>>
>>> On Wed, Sep 26, 2018, 16:08 Rob Verkuylen <rob@verkuylen.net> wrote:
>>>
>>>> Hi,
>>>>
>>>>
>>>> I'm trying to get vfs loading of my jar working in 1.7.2. On a test
>>>> cluster of the same version this works fine and I see the jar being
>>>> replicated in the cache in '/tmp/accumulo-vfs*/fstore-filters-1.3.2.jar',
>>>> but when I do the exact same thing on prod, I get the stacktrace below. Any
>>>> ideas?
>>>>
>>>>
>>>> Setup I used:
>>>>
>>>> config -s
>>>> general.vfs.context.classpath.fstore=hdfs://nameservice1/libs/accumulo/fstore/.*jar
>>>> config -t test.fstore_index -s table.classpath.context=fstore
>>>>
>>>> Stacktrace on tablet server:
>>>> Failed to load class
>>>>             java.lang.ClassNotFoundException: IO Error loading class
>>>> org.apache.accumulo.tserver.compaction.DefaultCompactionStrategy
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.ContextManager.loadClass(ContextManager.java:188)
>>>>                         at
>>>> org.apache.accumulo.core.conf.Property.createInstance(Property.java:873)
>>>>                         at
>>>> org.apache.accumulo.core.conf.Property.createTableInstanceFromPropertyName(Property.java:910)
>>>>                         at
>>>> org.apache.accumulo.tserver.TabletServerResourceManager$TabletResourceManager.needsMajorCompaction(TabletServerResourceManager.java:647)
>>>>                         at
>>>> org.apache.accumulo.tserver.tablet.Tablet.needsMajorCompaction(Tablet.java:1594)
>>>>                         at
>>>> org.apache.accumulo.tserver.tablet.Tablet.initiateMajorCompaction(Tablet.java:1574)
>>>>                         at
>>>> org.apache.accumulo.tserver.TabletServer$MajorCompactor.run(TabletServer.java:1875)
>>>>                         at
>>>> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
>>>>                         at java.lang.Thread.run(Thread.java:745)
>>>>             Caused by: org.apache.commons.vfs2.FileSystemException:
>>>> Could not replicate
>>>> "hdfs://nameservice1/libs/accumulo/fstore/fstore-filters-1.3.2.jar".
>>>>                         at
>>>> org.apache.commons.vfs2.provider.AbstractFileSystem.replicateFile(AbstractFileSystem.java:426)
>>>>                         at
>>>> org.apache.commons.vfs2.provider.zip.ZipFileSystem.<init>(ZipFileSystem.java:66)
>>>>                         at
>>>> org.apache.commons.vfs2.provider.jar.JarFileSystem.<init>(JarFileSystem.java:48)
>>>>                         at
>>>> org.apache.commons.vfs2.provider.jar.JarFileProvider.doCreateFileSystem(JarFileProvider.java:80)
>>>>                         at
>>>> org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.createFileSystem(AbstractLayeredFileProvider.java:87)
>>>>                         at
>>>> org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:1022)
>>>>                         at
>>>> org.apache.commons.vfs2.impl.DefaultFileSystemManager.createFileSystem(DefaultFileSystemManager.java:1042)
>>>>                         at
>>>> org.apache.commons.vfs2.impl.VFSClassLoader.addFileObjects(VFSClassLoader.java:156)
>>>>                         at
>>>> org.apache.commons.vfs2.impl.VFSClassLoader.<init>(VFSClassLoader.java:119)
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.AccumuloReloadingVFSClassLoader.<init>(AccumuloReloadingVFSClassLoader.java:147)
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.AccumuloReloadingVFSClassLoader.<init>(AccumuloReloadingVFSClassLoader.java:162)
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.ContextManager$Context.getClassLoader(ContextManager.java:46)
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.ContextManager.getClassLoader(ContextManager.java:174)
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.ContextManager.loadClass(ContextManager.java:186)
>>>>                         ... 8 more
>>>>             Caused by: org.apache.commons.vfs2.FileSystemException:
>>>> Unknown message with code "No such file or directory".
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.UniqueFileReplicator.replicateFile(UniqueFileReplicator.java:68)
>>>>                         at
>>>> org.apache.commons.vfs2.provider.AbstractFileSystem.doReplicateFile(AbstractFileSystem.java:473)
>>>>                         at
>>>> org.apache.commons.vfs2.provider.AbstractFileSystem.replicateFile(AbstractFileSystem.java:422)
>>>>                         ... 21 more
>>>>             Caused by: java.io.IOException: No such file or directory
>>>>                         at
>>>> java.io.UnixFileSystem.createFileExclusively(Native Method)
>>>>                         at java.io.File.createTempFile(File.java:2024)
>>>>                         at
>>>> org.apache.accumulo.start.classloader.vfs.UniqueFileReplicator.replicateFile(UniqueFileReplicator.java:60)
>>>>                         ... 23 more
>>>>
>>>>
>>>>
>>>>
>>>

Mime
View raw message