accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Medinets <david.medin...@gmail.com>
Subject Re: Unable to load Iterator with setscaniter and setshelliter
Date Sun, 15 Jun 2014 14:51:04 GMT
Opps.

Accumulo: 1.5.0


On Sun, Jun 15, 2014 at 10:50 AM, David Medinets <david.medinets@gmail.com>
wrote:

> I am using my https://github.com/medined/D4M_Schema project to start
> Accumulo. So the environment that I am using can be duplicated exactly if
> needed. I am using
>
> Accumulo: 1.5.50
> Hadoop: 1.2.1
>
> On Sun, Jun 15, 2014 at 9:50 AM, William Slacum <
> wilhelm.von.cloud@accumulo.net> wrote:
>
>> Wouldn't the iterator have to be on the classpath for the JVM that
>> launches the shell command?
>>
>>
>> On Sun, Jun 15, 2014 at 9:02 AM, Vicky Kak <vicky.kak@gmail.com> wrote:
>>
>>>
>>> >>setiter -n MyIterator -p 10 -scan -minc -majc -class
>>> com.codebits.d4m.iterator.MyIterator
>>> >>scan
>>>
>>> The above line fails for me with the similar kind of error i.e
>>> ClassNotFoundException
>>>
>>> root@accumulo atest> setiter -n MyIterator -p 10 -scan -minc -majc
>>> -class org.dallaybatta.MyIterator
>>> 2014-06-15 18:20:18,061 [shell.Shell] ERROR:
>>> org.apache.accumulo.shell.ShellCommandException: Command could not be
>>> initialized (Unable to load org.dallaybatta.MyIterator; class not found.)
>>>
>>>
>>> My hdfs contains the corresponding jars but it yet fails.
>>> After digging a code for a while I figured that the error is coming from
>>> org.apache.accumulo.shell.commands.SetIterCommand::execute
>>>
>>> *********************************************************
>>>     try {
>>>       clazz =
>>> classloader.loadClass(className).asSubclass(SortedKeyValueIterator.class);
>>>       untypedInstance = clazz.newInstance();
>>>     } catch (ClassNotFoundException e) {
>>>       StringBuilder msg = new StringBuilder("Unable to load
>>> ").append(className);
>>>       if (className.indexOf('.') < 0) {
>>>         msg.append("; did you use a fully qualified package name?");
>>>       } else {
>>>         msg.append("; class not found.");
>>>       }
>>>       throw new ShellCommandException(ErrorCode.INITIALIZATION_FAILURE,
>>> msg.toString());
>>>     } catch (InstantiationException e) {
>>>
>>> *********************************************************
>>>
>>> Typically the ClassNotFoundException can appear also if the dependent
>>> classes are not present, here SortedKeyValueIterator could be the use case,
>>> I moved the accumulo core to the classpath folder in the hdfs but still
>>> could not get it working. May be some other dependent classes are required,
>>> needs more time to analyse in this direction.
>>>
>>> The document states the following so it should ideally work,
>>> "although the VFS classloader allows for classpath manipulation using a
>>> variety of schemes including URLs and HDFS URIs."
>>>
>>> I find it strange that your test and my tests results differ as you are
>>> able to set Iterator for a single row but I am not.
>>>
>>> Thanks,
>>> Vicky
>>>
>>>
>>>
>>>
>>> On Sun, Jun 15, 2014 at 8:25 AM, David Medinets <
>>> david.medinets@gmail.com> wrote:
>>>
>>>> I'm sure that I'm overlooking something simple. I can load my iterator
>>>> using setiter but not with setscaniter or setshelliter.
>>>>
>>>> Here is my do-nothing iterator:
>>>>
>>>> public class MyIterator extends WrappingIterator implements
>>>> OptionDescriber {
>>>>
>>>>     @Override
>>>>     public IteratorOptions describeOptions() {
>>>>         String name = "dummy";
>>>>         String description = "Dummy Description";
>>>>         Map<String, String> namedOptions = new HashMap<String,
>>>> String>();
>>>>         List<String> unnamedOptionDescriptions = null;
>>>>         return new IteratorOptions(name, description, namedOptions,
>>>> unnamedOptionDescriptions);
>>>>     }
>>>>
>>>>     @Override
>>>>     public boolean validateOptions(Map<String, String> options) {
>>>>         return true;
>>>>     }
>>>>
>>>> }
>>>>
>>>> I copy the jar file out to HDFS:
>>>>
>>>> hadoop fs -mkdir /user/vagrant/d4m/classpath
>>>> hadoop fs -put /vagrant/schema/target/d4m_schema-0.0.1-SNAPSHOT.jar
>>>> /user/vagrant/classpath
>>>>
>>>> I set the table-specific classpath context:
>>>>
>>>> createtable atest
>>>> table atest
>>>> insert row cf cq value
>>>> config -s
>>>> general.vfs.context.classpath.d4m=hdfs://affy-master:9000/user/vagrant/classpath
>>>> config -t atest -s table.classpath.context=d4m
>>>>
>>>> Now I can configure the iterator and scan over the single row without a
>>>> problem:
>>>>
>>>> setiter -n MyIterator -p 10 -scan -minc -majc -class
>>>> com.codebits.d4m.iterator.MyIterator
>>>> scan
>>>> deleteiter -n MyIterator -scan -minc -majc
>>>>
>>>> However, the setscaniter commands fails:
>>>>
>>>> root@instance atest> setscaniter -n MyIterator -p 10 -class
>>>> com.codebits.d4m.iterator.MyIterator
>>>> 2014-06-15 02:54:14,098 [shell.Shell] WARN : Deprecated, use
>>>> setshelliter
>>>> Dummy Description
>>>> 2014-06-15 02:54:14,126 [shell.Shell] ERROR:
>>>> org.apache.accumulo.core.util.shell.ShellCommandException: Command could
>>>> not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
>>>>
>>>> As does the setshelliter:
>>>>
>>>> root@instance atest> setshelliter -pn d4m -n MyIterator -p 10 -class
>>>> com.codebits.d4m.iterator.MyIterator
>>>> Dummy Description
>>>> 2014-06-15 02:55:07,025 [shell.Shell] ERROR:
>>>> org.apache.accumulo.core.util.shell.ShellCommandException: Command could
>>>> not be initialized (Unable to load com.codebits.d4m.iterator.MyIterator)
>>>>
>>>> I don't see any messages in the log files.
>>>>
>>>> Any suggestions to resolve these issues?
>>>>
>>>>
>>>
>>
>

Mime
View raw message