hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pig user <pigu...@gmail.com>
Subject Re: HBase 0.90.4: YCSB gets NPE with large loads when timeseries is set
Date Tue, 21 Feb 2012 02:32:19 GMT
It looks like HBASE-2909, but it was fixed in 0.90.0.

On Sat, Feb 18, 2012 at 6:48 PM, Jean T. Anderson <jta@bristowhill.com>wrote:

> On 2/18/12 12:02 PM, Stack wrote:
>
>> What version are you running?
>>
> HBase 0.90.4 / hadoop 0.20.2
>
>
>  I don't see any explicit NPE mentions in the commit log on
>> HConnectionManager.  Maybe its still an issue?
>>
>> St.Ack
>>
>
> ok, thanks for the sanity check. I'll dig a little deeper and also see if
> the problem reproduces on 0.92.
>
> regards,
>
>  -jean
>
>
>  On Fri, Feb 17, 2012 at 4:24 PM, Jean T. Anderson<jta@bristowhill.com>
>>  wrote:
>>
>>> Hi, everyone,
>>>
>>> I often get an NPE during the YCSB load process if timeseries is set and
>>> I'm
>>> loading lots of records. Here's the syntax I'm using:
>>>
>>>   YCSB Client 0.1
>>>   Command line: -load -db com.yahoo.ycsb.db.HBaseClient -P
>>> workloads/workloada -p columnfamily=family -p recordcount=10000000
>>> -threads
>>> 32 -s -p measurementtype=timeseries -p timeseries.granularity=5000
>>>
>>> 100 million records load successfully with 32 threads if timeseries is
>>> not
>>> set. But when I set timeseries, I often get NPEs with stacks that move
>>> around depending on the number of records and the timeseries granularity.
>>>
>>> For example:
>>>
>>> 1) 10 million records gets an NPE when timeseries is enabled with a
>>> granularity of 5000 ms (but 4000 ms loads successfully):
>>>
>>> java.lang.NullPointerException
>>>        at org.apache.hadoop.hbase.util.**Bytes.compareTo(Bytes.java:**
>>> 916)
>>>        at
>>> org.apache.hadoop.hbase.util.**Bytes$ByteArrayComparator.**
>>> compare(Bytes.java:109)
>>>        at
>>> org.apache.hadoop.hbase.util.**Bytes$ByteArrayComparator.**
>>> compare(Bytes.java:101)
>>>        at
>>> java.util.TreeMap$**NavigableSubMap.**checkUpperBound(TreeMap.java:**
>>> 2666)
>>>        at java.util.TreeMap$**NavigableSubMap.isInRange(**
>>> TreeMap.java:2688)
>>>        at java.util.TreeMap$**NavigableSubMap.get(TreeMap.**java:2502)
>>>        at
>>> org.apache.hadoop.hbase.util.**SoftValueSortedMap.get(**
>>> SoftValueSortedMap.java:96)
>>>        at
>>> org.apache.hadoop.hbase.**client.HConnectionManager$**
>>> HConnectionImplementation.**getCachedLocation(**
>>> HConnectionManager.java:826)
>>>
>>> 2) 100 million records gets an NPE with 4000 ms (note the changed stack):
>>>
>>> java.lang.NullPointerException
>>>        at org.apache.hadoop.hbase.util.**Bytes.compareTo(Bytes.java:**
>>> 916)
>>>        at
>>> org.apache.hadoop.hbase.util.**Bytes$ByteArrayComparator.**
>>> compare(Bytes.java:109)
>>>        at
>>> org.apache.hadoop.hbase.util.**Bytes$ByteArrayComparator.**
>>> compare(Bytes.java:101)
>>>        at java.util.TreeMap$**NavigableSubMap.cmp(TreeMap.**java:2748)
>>>        at
>>> java.util.TreeMap$**NavigableSubMap.**findLowerEntryImpl(TreeMap.**
>>> java:2881)
>>>        at
>>> java.util.TreeMap$**NavigableSubMap.**findLowerEntry(TreeMap.java:**
>>> 3000)
>>>        at java.util.TreeMap$**NavigableSubMap.smallerEntry(**
>>> TreeMap.java:2994)
>>>        at
>>> java.util.TreeMap$**AscendingSubMapIterator.**
>>> getBoundaryNode(TreeMap.java:**281)
>>>        at java.util.TreeMap$**AbstractSubMapIterator.<init>(**
>>> TreeMap.java:186)
>>>        at
>>> java.util.TreeMap$**AscendingSubMapKeySet.**iterator(TreeMap.java:1968)
>>>        at java.util.TreeMap$**NavigableSubMap.isEmpty(**
>>> TreeMap.java:2477)
>>>        at
>>> org.apache.hadoop.hbase.util.**SoftValueSortedMap.isEmpty(**
>>> SoftValueSortedMap.java:151)
>>>        at
>>> org.apache.hadoop.hbase.**client.HConnectionManager$**
>>> HConnectionImplementation.**getCachedLocation(**
>>> HConnectionManager.java:826)
>>>
>>> 3) 100 million records gets an NPE with 3000 ms (and the stack changes):
>>>
>>> java.lang.NullPointerException
>>>        at org.apache.hadoop.hbase.util.**Bytes.compareTo(Bytes.java:**
>>> 916)
>>>        at
>>> org.apache.hadoop.hbase.util.**Bytes$ByteArrayComparator.**
>>> compare(Bytes.java:109)
>>>        at
>>> org.apache.hadoop.hbase.util.**Bytes$ByteArrayComparator.**
>>> compare(Bytes.java:101)
>>>        at java.util.TreeMap.cmp(TreeMap.**java:4514)
>>>        at java.util.TreeMap.get(TreeMap.**java:4386)
>>>        at
>>> org.apache.hadoop.hbase.util.**SoftValueSortedMap.get(**
>>> SoftValueSortedMap.java:96)
>>>        at
>>> org.apache.hadoop.hbase.**client.HConnectionManager$**
>>> HConnectionImplementation.**getCachedLocation(**
>>> HConnectionManager.java:830)
>>>
>>> I found HBASE-3878, but it looks like that was fixed in 0.90.4.
>>> I also found HBASE-5088, which is marked fixed in 0.90.6 -- does anyone
>>> recognize if this problem fits that profile?
>>>
>>> many thanks,
>>>
>>>  -jean
>>>
>>>
>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message