accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Marion (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2911) setscaniter and setshelliter unable to load class.
Date Sun, 15 Jun 2014 16:07:01 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031961#comment-14031961
] 

Dave Marion commented on ACCUMULO-2911:
---------------------------------------

Looking at the code, the SetIterCommand uses the ContextManager and the AccumuloVFSClassLoader,
the SetScanIterCommand and SetShellIterCommand do not. What you are trying to do is not currently
supported. Looking at the code I think setScanIterCommand should use a table context classpath
and the setShellIterCommand should not. We could create the notion of a shell context classloader
with very little work or do something that [~bills] suggested which is to use a table context
classloader when in a table context. Not sure if there are cases where we need/want a shell
context classloader when not in a table context.

> setscaniter and setshelliter unable to load class.
> --------------------------------------------------
>
>                 Key: ACCUMULO-2911
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2911
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.5.0
>            Reporter: David Medinets
>            Priority: Trivial
>
> Problem:
> I can use a custom iterator using the setiter command but the same iterator does not
work using the setscaniter or setshelliter commands.
> References:
>   https://blogs.apache.org/accumulo/entry/the_accumulo_classloader
>   http://accumulo.apache.org/1.5/examples/classpath.html
> Description:
> I am using my https://github.com/medined/D4M_Schema project to start Accumulo. So the
environment that I am using can be duplicated exactly if needed. I am using
> Accumulo: 1.5.0
> Hadoop: 1.2.1
> The classpath settings in accumulo-site.xml are the following (which I think are the
default):
>     <property>
>       <name>general.classpaths</name>
>       <value>
>     $ACCUMULO_HOME/server/target/classes/,
>     $ACCUMULO_HOME/core/target/classes/,
>     $ACCUMULO_HOME/start/target/classes/,
>     $ACCUMULO_HOME/examples/target/classes/,
>         $ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
>         $ACCUMULO_HOME/lib/[^.].*.jar,
>         $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
>         $HADOOP_HOME/conf,
>         $HADOOP_HOME/[^.].*.jar,
>         $HADOOP_HOME/lib/[^.].*.jar,
>       </value>
>       <description>Classpaths that accumulo checks for updates and class files.
>       When using the Security Manager, please remove the ".../target/classes/" values.
>       </description>
>     </property>
> I can load my iterator using setiter but not with setscaniter or setshelliter.
> Here is my do-nothing iterator:
> public class MyIterator extends WrappingIterator implements OptionDescriber {
>     @Override
>     public IteratorOptions describeOptions() {
>         String name = "dummy";
>         String description = "Dummy Description";
>         Map<String, String> namedOptions = new HashMap<String, String>();
>         List<String> unnamedOptionDescriptions = null;
>         return new IteratorOptions(name, description, namedOptions, unnamedOptionDescriptions);
>     }
>     @Override
>     public boolean validateOptions(Map<String, String> options) {
>         return true;
>     }
>     
> }
> I copy the jar file out to HDFS:
> hadoop fs -mkdir /user/vagrant/d4m/classpath
> hadoop fs -put /vagrant/schema/target/d4m_schema-0.0.1-SNAPSHOT.jar /user/vagrant/classpath
> I set the table-specific classpath context:
> createtable atest
> table atest
> insert row cf cq value
> config -s general.vfs.context.classpath.d4m=hdfs://affy-master:9000/user/vagrant/classpath
> config -t atest -s table.classpath.context=d4m
> Now I can configure the iterator and scan over the single row without a problem:
> setiter -n MyIterator -p 10 -scan -minc -majc -class com.codebits.d4m.iterator.MyIterator
> scan
> deleteiter -n MyIterator -scan -minc -majc 
> However, the setscaniter commands fails:
> root@instance atest> setscaniter -n MyIterator -p 10 -class com.codebits.d4m.iterator.MyIterator
> 2014-06-15 02:54:14,098 [shell.Shell] WARN : Deprecated, use setshelliter
> Dummy Description
> 2014-06-15 02:54:14,126 [shell.Shell] ERROR: org.apache.accumulo.core.util.shell.ShellCommandException:
Command could not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
> As does the setshelliter:
> root@instance atest> setshelliter -pn d4m -n MyIterator -p 10 -class com.codebits.d4m.iterator.MyIterator
> Dummy Description
> 2014-06-15 02:55:07,025 [shell.Shell] ERROR: org.apache.accumulo.core.util.shell.ShellCommandException:
Command could not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
> I don't see any messages in the log files. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message