accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Unable to load Iterator with setscaniter and setshelliter
Date Sun, 15 Jun 2014 14:31:00 GMT
Naw, the commons-vfs loader should be loading those resources using a
second classloader.

Maybe it's just a problem with the HDFS code? Does it work if you have the
jar with your iterator in lib/ or lib/ext? Or, maybe something is wrong
like you defined a private constructor which is throwing that Exception?
On Jun 15, 2014 8:50 AM, "William Slacum" <wilhelm.von.cloud@accumulo.net>
wrote:

> Wouldn't the iterator have to be on the classpath for the JVM that
> launches the shell command?
>
>
> On Sun, Jun 15, 2014 at 9:02 AM, Vicky Kak <vicky.kak@gmail.com> wrote:
>
>>
>> >>setiter -n MyIterator -p 10 -scan -minc -majc -class
>> com.codebits.d4m.iterator.MyIterator
>> >>scan
>>
>> The above line fails for me with the similar kind of error i.e
>> ClassNotFoundException
>>
>> root@accumulo atest> setiter -n MyIterator -p 10 -scan -minc -majc
>> -class org.dallaybatta.MyIterator
>> 2014-06-15 18:20:18,061 [shell.Shell] ERROR:
>> org.apache.accumulo.shell.ShellCommandException: Command could not be
>> initialized (Unable to load org.dallaybatta.MyIterator; class not found.)
>>
>>
>> My hdfs contains the corresponding jars but it yet fails.
>> After digging a code for a while I figured that the error is coming from
>> org.apache.accumulo.shell.commands.SetIterCommand::execute
>>
>> *********************************************************
>>     try {
>>       clazz =
>> classloader.loadClass(className).asSubclass(SortedKeyValueIterator.class);
>>       untypedInstance = clazz.newInstance();
>>     } catch (ClassNotFoundException e) {
>>       StringBuilder msg = new StringBuilder("Unable to load
>> ").append(className);
>>       if (className.indexOf('.') < 0) {
>>         msg.append("; did you use a fully qualified package name?");
>>       } else {
>>         msg.append("; class not found.");
>>       }
>>       throw new ShellCommandException(ErrorCode.INITIALIZATION_FAILURE,
>> msg.toString());
>>     } catch (InstantiationException e) {
>>
>> *********************************************************
>>
>> Typically the ClassNotFoundException can appear also if the dependent
>> classes are not present, here SortedKeyValueIterator could be the use case,
>> I moved the accumulo core to the classpath folder in the hdfs but still
>> could not get it working. May be some other dependent classes are required,
>> needs more time to analyse in this direction.
>>
>> The document states the following so it should ideally work,
>> "although the VFS classloader allows for classpath manipulation using a
>> variety of schemes including URLs and HDFS URIs."
>>
>> I find it strange that your test and my tests results differ as you are
>> able to set Iterator for a single row but I am not.
>>
>> Thanks,
>> Vicky
>>
>>
>>
>>
>> On Sun, Jun 15, 2014 at 8:25 AM, David Medinets <david.medinets@gmail.com
>> > wrote:
>>
>>> I'm sure that I'm overlooking something simple. I can load my iterator
>>> using setiter but not with setscaniter or setshelliter.
>>>
>>> Here is my do-nothing iterator:
>>>
>>> public class MyIterator extends WrappingIterator implements
>>> OptionDescriber {
>>>
>>>     @Override
>>>     public IteratorOptions describeOptions() {
>>>         String name = "dummy";
>>>         String description = "Dummy Description";
>>>         Map<String, String> namedOptions = new HashMap<String, String>();
>>>         List<String> unnamedOptionDescriptions = null;
>>>         return new IteratorOptions(name, description, namedOptions,
>>> unnamedOptionDescriptions);
>>>     }
>>>
>>>     @Override
>>>     public boolean validateOptions(Map<String, String> options) {
>>>         return true;
>>>     }
>>>
>>> }
>>>
>>> I copy the jar file out to HDFS:
>>>
>>> hadoop fs -mkdir /user/vagrant/d4m/classpath
>>> hadoop fs -put /vagrant/schema/target/d4m_schema-0.0.1-SNAPSHOT.jar
>>> /user/vagrant/classpath
>>>
>>> I set the table-specific classpath context:
>>>
>>> createtable atest
>>> table atest
>>> insert row cf cq value
>>> config -s
>>> general.vfs.context.classpath.d4m=hdfs://affy-master:9000/user/vagrant/classpath
>>> config -t atest -s table.classpath.context=d4m
>>>
>>> Now I can configure the iterator and scan over the single row without a
>>> problem:
>>>
>>> setiter -n MyIterator -p 10 -scan -minc -majc -class
>>> com.codebits.d4m.iterator.MyIterator
>>> scan
>>> deleteiter -n MyIterator -scan -minc -majc
>>>
>>> However, the setscaniter commands fails:
>>>
>>> root@instance atest> setscaniter -n MyIterator -p 10 -class
>>> com.codebits.d4m.iterator.MyIterator
>>> 2014-06-15 02:54:14,098 [shell.Shell] WARN : Deprecated, use setshelliter
>>> Dummy Description
>>> 2014-06-15 02:54:14,126 [shell.Shell] ERROR:
>>> org.apache.accumulo.core.util.shell.ShellCommandException: Command could
>>> not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
>>>
>>> As does the setshelliter:
>>>
>>> root@instance atest> setshelliter -pn d4m -n MyIterator -p 10 -class
>>> com.codebits.d4m.iterator.MyIterator
>>> Dummy Description
>>> 2014-06-15 02:55:07,025 [shell.Shell] ERROR:
>>> org.apache.accumulo.core.util.shell.ShellCommandException: Command could
>>> not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
>>>
>>> I don't see any messages in the log files.
>>>
>>> Any suggestions to resolve these issues?
>>>
>>>
>>
>

Mime
View raw message