hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gray <jg...@facebook.com>
Subject RE: Custom WritableByteArrayComparable crashing. Need help please (UNCLASSIFIED)
Date Fri, 11 Jun 2010 17:30:50 GMT
Hey Chris,

Did you check out the logs on the server side?  This should give some more insight into what's
going on.

One thing to keep in mind is that the server needs to know about the filter you are using
(it needs to be in a jar in the classpath).  You also may need to add it to HbaseObjectWritable.java
so it can be sent across the RPC layer.

There should be some previous emails on the mailing list about this.

JG

> -----Original Message-----
> From: Schlesiger, Chris (Civ, ARL/CISD)
> [mailto:chris.schlesiger@us.army.mil]
> Sent: Friday, June 11, 2010 10:13 AM
> To: user@hbase.apache.org
> Subject: Custom WritableByteArrayComparable crashing. Need help please
> (UNCLASSIFIED)
> 
> Classification:  UNCLASSIFIED
> Caveats: NONE
> 
> Our HBase contains people names for RowIDs, and we need our application
> to be able to try other types of string matching to find rows other
> than
> than the straight Substring and Regex matching that comes with the
> HBase
> API in the filters package.
> 
> We were first working with HBase 0.20.1 where
> WritableByteArrayComparable was an interface. I tried implementing the
> interface and had no luck. I figured I'd better upgrade to the newest
> HBase, which I just did. Now in 0.20.4, WritableByteArrayComparable is
> an abstract class. Okay, so I changed my code to create a new child
> class and I still ran into the same problem with crashing and no rows
> returned by the scanner.
> 
> So I decided to just try out the mere idea of creating a new child
> comparator to see if the same problem arose. I went to the HBase source
> code for the 0.20.4 tag, and looked at the SubstringComparator class in
> the filters package. I re-created it line by line in my own java
> project. The class I created was the same except that since I wasn't in
> the same package as the super class, I couldn't access the .value
> property as on line 74. But it looked to me that it wasn't needed since
> the getValue() method was overridden and ignoring .value anyway.
> 
> I created a scanner, created a RowFilter and passed in my
> SubstringComparatorDuplicate, and ran the program. Same exact crash. If
> I change my code only to create a SubstringComparator from the hbase
> jar
> instead, it works perfectly and I get rows returned by the scanner.
> 
> Here are my symptoms:
> 
> The program halts for almost 30 seconds, then I get the following
> exception:
> 
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> contact region server 127.0.0.1:60062 for region People,,1276369002801,
> row '', but failed after 10 attempts.
> Exceptions:
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> java.io.IOException: Call to /127.0.0.1:60062 failed on local
> exception:
> java.io.EOFException
> 
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegio
> n
> ServerWithRetries(HConnectionManager.java:1055)
> 	at
> org.apache.hadoop.hbase.client.Htable$ClientScanner.nextScanner(HTable.
> j
> ava:2003)
> 	at
> org.apache.hadoop.hbase.client.Htable$ClientScanner.initialize(HTable.j
> a
> va:1923)
> 	at
> org.apache.hadoop.hbase.client.Htable.getScanner(HTable.java:403)
> 	at org.mycompany.test.search(Search.java:209)
> 	at org.mycompany.test.main(Main.java:40)
> 
> I tried putting break points on each of the methods in my
> SubstringComparatorDuplicate, and the breakpoint in the write method
> gets hit 10 times and then I get the exception.
> 
> I'm running HBase in pseudo-distributed mode on my development computer
> until I get the code working. The database is small. I'm running Hadoop
> 0.20.2 in dfs mode only for the same reason.
> 
> I can't see any reason why my SubstringComparatorDuplicate class
> shouldn't work. It is identical to the code found here
> 
> http://svn.apache.org/viewvc/hbase/tags/0.20.4/src/java/org/apache/hado
> o
> p/hbase/filter/SubstringComparator.java?view=markup
> 
> With only 2 exceptions.
> 1) I'm in a different package, project, and jar file.
> 2) Line 74 is commented out
> 
> Has anyone got any great ideas? I'm pretty much stuck now, and it's
> looking like there's a bug in the HBase code as near as I can tell.
> 
> I'd appreciate the help. Thanks in advance.
> 
> Chris Schlesiger
> cschles@arl.army.mil
> Electronics/Software Engineer
> U.S. Army Research Lab
> Voc: 301-394-2473
> Fax: 301-394-3591
> DSN 290-2473
> Classification:  UNCLASSIFIED
> Caveats: NONE


Mime
View raw message