accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vicky Kak <vicky....@gmail.com>
Subject Re: Unable to load Iterator with setscaniter and setshelliter
Date Sun, 15 Jun 2014 13:02:15 GMT
>>setiter -n MyIterator -p 10 -scan -minc -majc -class
com.codebits.d4m.iterator.MyIterator
>>scan

The above line fails for me with the similar kind of error i.e
ClassNotFoundException

root@accumulo atest> setiter -n MyIterator -p 10 -scan -minc -majc -class
org.dallaybatta.MyIterator
2014-06-15 18:20:18,061 [shell.Shell] ERROR:
org.apache.accumulo.shell.ShellCommandException: Command could not be
initialized (Unable to load org.dallaybatta.MyIterator; class not found.)


My hdfs contains the corresponding jars but it yet fails.
After digging a code for a while I figured that the error is coming from
org.apache.accumulo.shell.commands.SetIterCommand::execute

*********************************************************
    try {
      clazz =
classloader.loadClass(className).asSubclass(SortedKeyValueIterator.class);
      untypedInstance = clazz.newInstance();
    } catch (ClassNotFoundException e) {
      StringBuilder msg = new StringBuilder("Unable to load
").append(className);
      if (className.indexOf('.') < 0) {
        msg.append("; did you use a fully qualified package name?");
      } else {
        msg.append("; class not found.");
      }
      throw new ShellCommandException(ErrorCode.INITIALIZATION_FAILURE,
msg.toString());
    } catch (InstantiationException e) {

*********************************************************

Typically the ClassNotFoundException can appear also if the dependent
classes are not present, here SortedKeyValueIterator could be the use case,
I moved the accumulo core to the classpath folder in the hdfs but still
could not get it working. May be some other dependent classes are required,
needs more time to analyse in this direction.

The document states the following so it should ideally work,
"although the VFS classloader allows for classpath manipulation using a
variety of schemes including URLs and HDFS URIs."

I find it strange that your test and my tests results differ as you are
able to set Iterator for a single row but I am not.

Thanks,
Vicky




On Sun, Jun 15, 2014 at 8:25 AM, David Medinets <david.medinets@gmail.com>
wrote:

> I'm sure that I'm overlooking something simple. I can load my iterator
> using setiter but not with setscaniter or setshelliter.
>
> Here is my do-nothing iterator:
>
> public class MyIterator extends WrappingIterator implements
> OptionDescriber {
>
>     @Override
>     public IteratorOptions describeOptions() {
>         String name = "dummy";
>         String description = "Dummy Description";
>         Map<String, String> namedOptions = new HashMap<String, String>();
>         List<String> unnamedOptionDescriptions = null;
>         return new IteratorOptions(name, description, namedOptions,
> unnamedOptionDescriptions);
>     }
>
>     @Override
>     public boolean validateOptions(Map<String, String> options) {
>         return true;
>     }
>
> }
>
> I copy the jar file out to HDFS:
>
> hadoop fs -mkdir /user/vagrant/d4m/classpath
> hadoop fs -put /vagrant/schema/target/d4m_schema-0.0.1-SNAPSHOT.jar
> /user/vagrant/classpath
>
> I set the table-specific classpath context:
>
> createtable atest
> table atest
> insert row cf cq value
> config -s
> general.vfs.context.classpath.d4m=hdfs://affy-master:9000/user/vagrant/classpath
> config -t atest -s table.classpath.context=d4m
>
> Now I can configure the iterator and scan over the single row without a
> problem:
>
> setiter -n MyIterator -p 10 -scan -minc -majc -class
> com.codebits.d4m.iterator.MyIterator
> scan
> deleteiter -n MyIterator -scan -minc -majc
>
> However, the setscaniter commands fails:
>
> root@instance atest> setscaniter -n MyIterator -p 10 -class
> com.codebits.d4m.iterator.MyIterator
> 2014-06-15 02:54:14,098 [shell.Shell] WARN : Deprecated, use setshelliter
> Dummy Description
> 2014-06-15 02:54:14,126 [shell.Shell] ERROR:
> org.apache.accumulo.core.util.shell.ShellCommandException: Command could
> not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
>
> As does the setshelliter:
>
> root@instance atest> setshelliter -pn d4m -n MyIterator -p 10 -class
> com.codebits.d4m.iterator.MyIterator
> Dummy Description
> 2014-06-15 02:55:07,025 [shell.Shell] ERROR:
> org.apache.accumulo.core.util.shell.ShellCommandException: Command could
> not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
>
> I don't see any messages in the log files.
>
> Any suggestions to resolve these issues?
>
>

Mime
View raw message