Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E50CAD281 for ; Wed, 27 Jun 2012 20:18:57 +0000 (UTC) Received: (qmail 43933 invoked by uid 500); 27 Jun 2012 20:18:57 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 43912 invoked by uid 500); 27 Jun 2012 20:18:57 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 43903 invoked by uid 99); 27 Jun 2012 20:18:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Jun 2012 20:18:57 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jreucypoda@gmail.com designates 74.125.82.169 as permitted sender) Received: from [74.125.82.169] (HELO mail-we0-f169.google.com) (74.125.82.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Jun 2012 20:18:49 +0000 Received: by wefh52 with SMTP id h52so1256570wef.0 for ; Wed, 27 Jun 2012 13:18:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Jrs0SrTinMHIn4DCPI1jEWbHhEi567QKjGK24Bi8VYk=; b=bzoavt/+6QPKxeYPD3QaY+Bgc7jIrGeblJN0J+YIW7b39ViRKjTmXFLu480gXLlBNY Kgwb8J0JohraW1gvdx0nxJNBrzuCDpeK2R7heoLliJfr0dJtPPPfpIegDSnT7yhs0+dO H08xYvDMq2ufYroc3rUkT7SF/caDWapvc9IFxMqqg5kEfka6LshMxwnb2n5eQc+wd1d9 9BKWPxw4q/iO0sM+yWaUSWtCXGw/19ztqiPACiPWVW3v5N90p0W5pgd9jCPcr9KIGDTN 420XkWQYJaT0TGLo2y57XD/jx0+Bh1VJNUV8/Ty3OAbY7tljFLJmK7LBJyrXP98/R2g5 xzww== MIME-Version: 1.0 Received: by 10.180.109.129 with SMTP id hs1mr7496025wib.0.1340828307284; Wed, 27 Jun 2012 13:18:27 -0700 (PDT) Received: by 10.180.7.34 with HTTP; Wed, 27 Jun 2012 13:18:27 -0700 (PDT) In-Reply-To: References: Date: Wed, 27 Jun 2012 16:18:27 -0400 Message-ID: Subject: Re: Problem with using iterators to access table From: Jonathan Hsu To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=e89a8f13ea88f5e43f04c379ea44 --e89a8f13ea88f5e43f04c379ea44 Content-Type: text/plain; charset=ISO-8859-1 Hey Jim, I got it to work. I had to download the individual jar files for the external classes that I was using, and it works now. Thanks so much for your help! Jonathan Hsu On Wed, Jun 27, 2012 at 3:55 PM, Jim Klucar wrote: > Jonathan, > > You have to deploy all the dependencies of your iterator into the > lib/ext also, except whatever is on the Accumulo classpath (accumulo, > hadoop, zookeeper, etc). I seem to remember that the mega-jar with > all dependencies built into one file doesn't work. The classloader > won't search the lib path inside the jar for other jars and load them. > > What's going on here is that the tserver process has a separate > classloader for jars in lib/ext. When it detects a change in the > directory, it dynamically reloads all the jars in the directory. This > is how "user" jars can get incorporated into the tablet server > iterator stack without restarting the tservers. > > On Wed, Jun 27, 2012 at 3:48 PM, Jonathan Hsu > wrote: > > Thanks for your quick reply. > > > > I didn't deploy my iterator jar to the directory, so I did that and > tried to > > run it again. I looked in the .debug.log file, and now I'm getting > > NoClassDefFoundErrors for classes that are not in the same package. For > > example, I imported and am using the imported Jama Matrix class, and it > > threw a ClassNotFoundException on that class. Am I missing something > > important with regard to importing external classes? > > > > Thanks again, > > > > Jonathan Hsu > > > > > > On Wed, Jun 27, 2012 at 1:16 PM, Jim Klucar wrote: > >> > >> My first thought would be that you didn't deploy your iterator jar to > >> the accumulo lib/ext directory. That wouldn't be causing the > >> "built-in" iterators to produce an error also. Unfortunately this > >> error message isn't very informative because all the client knows is > >> that a Thrift service call somehow failed. The error message you want > >> to look at will be in the tserver_.log file (or .debug.log > >> file) on the tserver. This is in $ACCUMULO_HOME/logs by default. > >> > >> On Wed, Jun 27, 2012 at 1:03 PM, Jonathan Hsu > >> wrote: > >> > I'm trying to use iterators to access a table that I created. When I > >> > run a > >> > MockInstance, the iterators work perfectly, but they don't work when I > >> > try > >> > to use them on an actual table. Also, if I don't add the iterators, > the > >> > scanner works fine as well. I wrote both iterators, but the error > >> > doesn't > >> > seem to be with my iterators, because I also attempted the same thing > >> > with > >> > one of accumulo's iterators, and got the same result. I know that the > >> > table > >> > is created correctly (by checking the accumulo shell). > >> > > >> > Here is my code : > >> > > >> > /* created the table first */ > >> > > >> > Scanner scann = conn.createScanner(table, new Authorizations()); > >> > scann.setRange(new Range("term1")); > >> > > >> > IteratorSetting cfg = new IteratorSetting(10, MyIterator1.class, > >> > properties); > >> > scann.addScanIterator(cfg); > >> > > >> > IteratorSetting cfg2 = new IteratorSetting(11, MyIterator2.class, > >> > properties); > >> > scann.addScanIterator(cfg2); > >> > > >> > > >> > > >> > I'm getting this error : > >> > > >> > Exception in thread "main" java.lang.RuntimeException: > >> > org.apache.accumulo.core.client.impl.AccumuloServerException: Error on > >> > server127.0.0.1:9997 > >> > at > >> > > >> > > org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:186) > >> > at > >> > > >> > > com.bah.applefox.test.plugins.fulltextindex.DebugTest.main(DebugTest.java:151) > >> > > >> > Caused by: > org.apache.accumulo.core.client.impl.AccumuloServerException: > >> > Error on server 127.0.0.1:9997 > >> > at > >> > > >> > > org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:302) > >> > at > >> > > >> > > org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:94) > >> > at > >> > > >> > > org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:176) > >> > ... 1 more > >> > > >> > Caused by: org.apache.thrift.TApplicationException: Internal error > >> > processing startScan > >> > at > >> > > >> > > org.apache.thrift.TApplicationException.read(TApplicationException.java:108) > >> > at > >> > > >> > > org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(TabletClientService.java:184) > >> > at > >> > > >> > > org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(TabletClientService.java:157) > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> > at > >> > > >> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > >> > at > >> > > >> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > >> > at java.lang.reflect.Method.invoke(Method.java:597) > >> > at > >> > > >> > > org.apache.accumulo.cloudtrace.instrument.thrift.TraceWrap$2.invoke(TraceWrap.java:84) > >> > at $Proxy2.startScan(Unknown Source) > >> > at > >> > > >> > > org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:415) > >> > at > >> > > >> > > org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:295) > >> > ... 3 more > >> > > >> > > >> > Thanks for your time, > >> > > >> > Jonathan Hsu > > > > > > > > > > -- > > - Jonathan Hsu > -- - Jonathan Hsu --e89a8f13ea88f5e43f04c379ea44 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hey Jim,

I got it to work. =A0I had to download the indi= vidual jar files for the external classes that I was using, and it works no= w. =A0

Thanks so much for your help!
Jonathan Hsu

On Wed, Ju= n 27, 2012 at 3:55 PM, Jim Klucar <klucar@gmail.com> wrote:
Jonathan,

You have to deploy all the dependencies of your iterator into the
lib/ext also, except whatever is on the Accumulo classpath (accumulo,
hadoop, zookeeper, etc). =A0I seem to remember that the mega-jar with
all dependencies built into one file doesn't work. The classloader
won't search the lib path inside the jar for other jars and load them.<= br>
What's going on here is that the tserver process has a separate
classloader for jars in lib/ext. When it detects a change in the
directory, it dynamically reloads all the jars in the directory. This
is how "user" jars can get incorporated into the tablet server iterator stack without restarting the tservers.

On Wed, Jun 27, 2012 at 3:48 PM, Jonathan Hsu <jreucypoda@gmail.com> wrote:
> Thanks for your quick reply.
>
> I didn't deploy my iterator jar to the directory, so I did that an= d tried to
> run it again. =A0I looked in the .debug.log file, and now I'm gett= ing
> NoClassDefFoundErrors for classes that are not in the same package. = =A0For
> example, I imported and am using the imported Jama Matrix class, and i= t
> threw a ClassNotFoundException on that class. =A0Am I missing somethin= g
> important with regard to importing external classes?
>
> Thanks again,
>
> Jonathan Hsu
>
>
> On Wed, Jun 27, 2012 at 1:16 PM, Jim Klucar <klucar@gmail.com> wrote:
>>
>> My first thought would be that you didn't deploy your iterator= jar to
>> the accumulo lib/ext directory. That wouldn't be causing the >> "built-in" iterators to produce an error also. Unfortuna= tely this
>> error message isn't very informative because all the client kn= ows is
>> that a Thrift service call somehow failed. The error message you w= ant
>> to look at will be in the tserver_<hostname>.log file (or .d= ebug.log
>> file) on the tserver. This is in $ACCUMULO_HOME/logs by default. >>
>> On Wed, Jun 27, 2012 at 1:03 PM, Jonathan Hsu <jreucypoda@gmail.com>
>> wrote:
>> > I'm trying to use iterators to access a table that I crea= ted. =A0When I
>> > run a
>> > MockInstance, the iterators work perfectly, but they don'= t work when I
>> > try
>> > to use them on an actual table. =A0Also, if I don't add t= he iterators, the
>> > scanner works fine as well. =A0I wrote both iterators, but th= e error
>> > doesn't
>> > seem to be with my iterators, because I also attempted the sa= me thing
>> > with
>> > one of accumulo's iterators, and got the same result. =A0= I know that the
>> > table
>> > is created correctly (by checking the accumulo shell).
>> >
>> > Here is my code :
>> >
>> > /* created the table first */
>> >
>> > Scanner scann =3D conn.createScanner(table, new Authorization= s());
>> > scann.setRange(new Range("term1"));
>> >
>> > IteratorSetting cfg =3D new IteratorSetting(10, MyIterator1.c= lass,
>> > properties);
>> > scann.addScanIterator(cfg);
>> >
>> > IteratorSetting cfg2 =3D new IteratorSetting(11, MyIterator2.= class,
>> > properties);
>> > scann.addScanIterator(cfg2);
>> >
>> >
>> >
>> > I'm getting this error :
>> >
>> > Exception in thread "main" java.lang.RuntimeExcepti= on:
>> > org.apache.accumulo.core.client.impl.AccumuloServerException:= Error on
>> > server127.0.0.1:9997
>> > at
>> >
>> > org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(= ScannerIterator.java:186)
>> > at
>> >
>> > com.bah.applefox.test.plugins.fulltextindex.DebugTest.main(De= bugTest.java:151)
>> >
>> > Caused by: org.apache.accumulo.core.client.impl.AccumuloServe= rException:
>> > Error on server=A0127.0.0.1:9997
>> > at
>> >
>> > org.apache.accumulo.core.client.impl.ThriftScanner.scan(Thrif= tScanner.java:302)
>> > at
>> >
>> > org.apache.accumulo.core.client.impl.ScannerIterator$Reader.r= un(ScannerIterator.java:94)
>> > at
>> >
>> > org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(= ScannerIterator.java:176)
>> > ... 1 more
>> >
>> > Caused by: org.apache.thrift.TApplicationException: Internal = error
>> > processing startScan
>> > at
>> >
>> > org.apache.thrift.TApplicationException.read(TApplicationExce= ption.java:108)
>> > at
>> >
>> > org.apache.accumulo.core.tabletserver.thrift.TabletClientServ= ice$Client.recv_startScan(TabletClientService.java:184)
>> > at
>> >
>> > org.apache.accumulo.core.tabletserver.thrift.TabletClientServ= ice$Client.startScan(TabletClientService.java:157)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method= )
>> > at
>> >
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcces= sorImpl.java:39)
>> > at
>> >
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMet= hodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> >
>> > org.apache.accumulo.cloudtrace.instrument.thrift.TraceWrap$2.= invoke(TraceWrap.java:84)
>> > at $Proxy2.startScan(Unknown Source)
>> > at
>> >
>> > org.apache.accumulo.core.client.impl.ThriftScanner.scan(Thrif= tScanner.java:415)
>> > at
>> >
>> > org.apache.accumulo.core.client.impl.ThriftScanner.scan(Thrif= tScanner.java:295)
>> > ... 3 more
>> >
>> >
>> > Thanks for your time,
>> >
>> > Jonathan Hsu
>
>
>
>
> --
> - Jonathan Hsu



--
= - Jonathan Hsu
--e89a8f13ea88f5e43f04c379ea44--