hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Custom WritableByteArrayComparable crashing. Need help please (UNCLASSIFIED)
Date Fri, 11 Jun 2010 17:55:52 GMT
On Fri, Jun 11, 2010 at 10:30 AM, Jonathan Gray <jgray@facebook.com> wrote:

> Hey Chris,
>
> Did you check out the logs on the server side?  This should give some more
> insight into what's going on.


> One thing to keep in mind is that the server needs to know about the filter
> you are using (it needs to be in a jar in the classpath).  You also may need
> to add it to HbaseObjectWritable.java so it can be sent across the RPC
> layer.
>
>
That should only be an efficiency thing, right?


> There should be some previous emails on the mailing list about this.
>
> JG
>
> > -----Original Message-----
> > From: Schlesiger, Chris (Civ, ARL/CISD)
> > [mailto:chris.schlesiger@us.army.mil]
> > Sent: Friday, June 11, 2010 10:13 AM
> > To: user@hbase.apache.org
> > Subject: Custom WritableByteArrayComparable crashing. Need help please
> > (UNCLASSIFIED)
> >
> > Classification:  UNCLASSIFIED
> > Caveats: NONE
> >
> > Our HBase contains people names for RowIDs, and we need our application
> > to be able to try other types of string matching to find rows other
> > than
> > than the straight Substring and Regex matching that comes with the
> > HBase
> > API in the filters package.
> >
> > We were first working with HBase 0.20.1 where
> > WritableByteArrayComparable was an interface. I tried implementing the
> > interface and had no luck. I figured I'd better upgrade to the newest
> > HBase, which I just did. Now in 0.20.4, WritableByteArrayComparable is
> > an abstract class. Okay, so I changed my code to create a new child
> > class and I still ran into the same problem with crashing and no rows
> > returned by the scanner.
> >
> > So I decided to just try out the mere idea of creating a new child
> > comparator to see if the same problem arose. I went to the HBase source
> > code for the 0.20.4 tag, and looked at the SubstringComparator class in
> > the filters package. I re-created it line by line in my own java
> > project. The class I created was the same except that since I wasn't in
> > the same package as the super class, I couldn't access the .value
> > property as on line 74. But it looked to me that it wasn't needed since
> > the getValue() method was overridden and ignoring .value anyway.
> >
> > I created a scanner, created a RowFilter and passed in my
> > SubstringComparatorDuplicate, and ran the program. Same exact crash. If
> > I change my code only to create a SubstringComparator from the hbase
> > jar
> > instead, it works perfectly and I get rows returned by the scanner.
> >
> > Here are my symptoms:
> >
> > The program halts for almost 30 seconds, then I get the following
> > exception:
> >
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> > contact region server 127.0.0.1:60062 for region People,,1276369002801,
> > row '', but failed after 10 attempts.
> > Exceptions:
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> > java.io.IOException: Call to /127.0.0.1:60062 failed on local
> > exception:
> > java.io.EOFException
> >
> >       at
> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegio
> > n
> > ServerWithRetries(HConnectionManager.java:1055)
> >       at
> > org.apache.hadoop.hbase.client.Htable$ClientScanner.nextScanner(HTable.
> > j
> > ava:2003)
> >       at
> > org.apache.hadoop.hbase.client.Htable$ClientScanner.initialize(HTable.j
> > a
> > va:1923)
> >       at
> > org.apache.hadoop.hbase.client.Htable.getScanner(HTable.java:403)
> >       at org.mycompany.test.search(Search.java:209)
> >       at org.mycompany.test.main(Main.java:40)
> >
> > I tried putting break points on each of the methods in my
> > SubstringComparatorDuplicate, and the breakpoint in the write method
> > gets hit 10 times and then I get the exception.
> >
> > I'm running HBase in pseudo-distributed mode on my development computer
> > until I get the code working. The database is small. I'm running Hadoop
> > 0.20.2 in dfs mode only for the same reason.
> >
> > I can't see any reason why my SubstringComparatorDuplicate class
> > shouldn't work. It is identical to the code found here
> >
> > http://svn.apache.org/viewvc/hbase/tags/0.20.4/src/java/org/apache/hado
> > o
> > p/hbase/filter/SubstringComparator.java?view=markup
> >
> > With only 2 exceptions.
> > 1) I'm in a different package, project, and jar file.
> > 2) Line 74 is commented out
> >
> > Has anyone got any great ideas? I'm pretty much stuck now, and it's
> > looking like there's a bug in the HBase code as near as I can tell.
> >
> > I'd appreciate the help. Thanks in advance.
> >
> > Chris Schlesiger
> > cschles@arl.army.mil
> > Electronics/Software Engineer
> > U.S. Army Research Lab
> > Voc: 301-394-2473
> > Fax: 301-394-3591
> > DSN 290-2473
> > Classification:  UNCLASSIFIED
> > Caveats: NONE
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message