hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-750) NPE caused by StoreFileScanner.updateReaders
Date Thu, 17 Jul 2008 19:15:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614498#action_12614498
] 

Andrew Purtell commented on HBASE-750:
--------------------------------------

Regarding the performance test itself, these were my findings:

I created a test table with a single column family and a single row. The test added 1000 cells
at a time, using unique qualified column family addresses, into the single column family in
that single row. The test attempted to add eventually 1M cells to the family/row and hence
1M unique qualified column family members. During the test both the time required for insert
of 1000 cells in batch, the time required to read back one cell from the row, and the time
required to read back all cells from the row were measured.

I executed the test using a test cluster running Hadoop 0.17.1 and HBase trunk (0.2.0-dev)
revision 675659. The test client and regionserver ran on different hosts linked by a Gigabit
Ethernet network. 

The time required to insert 1000 cells remained within a narrow range over the duration of
the test, ranging from 6 to 85 milliseconds.

The time required to retrieve a single cell from the column/row remained within a narrower
range over the duration of the test, ranging from 1 to 42 milliseconds.

The time required to read back all values in the column/row increased in essentially a linear
manner over the duration of the test, rising from 18 milliseconds to 6,064 milliseconds from
1,000 cells to 865,000 cells. See attached chart. I don't find this surprising and I do not
think this behavior indicates a bug. Maybe the slope and possibly also some of the variability
can be decreased through profile driven changes, but there will always be this kind of linear
increase as cells are added to a column family/row in this manner. 

> NPE caused by StoreFileScanner.updateReaders
> --------------------------------------------
>
>                 Key: HBASE-750
>                 URL: https://issues.apache.org/jira/browse/HBASE-750
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.2.0
>         Environment: CentOS 5.1 x86_64 / 64-bit JDK build 1.6.0_03-b05
>            Reporter: Andrew Purtell
>         Attachments: chart.png, columntest.java, HRegionServer-sjdc-atr-test-4.log, output.txt
>
>
> Running a test to determine performance during inserts of many 100,000s of cells into
a single column family in a single row, the region server involved went down after taking
a NPE:
> 2008-07-17 18:12:18,051 FATAL org.apache.hadoop.hbase.regionserver.Flusher: Replay of
hlog required. Forcing server restart
> org.apache.hadoop.hbase.DroppedSnapshotException
>         at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>         at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>         at org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>         at org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
> Caused by: java.lang.NullPointerException
>         at java.lang.String.<init>(String.java:516)
>         at org.apache.hadoop.hbase.util.Bytes.toString(Bytes.java:71)
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.updateReaders(StoreFileScanner.java:374)
>         at org.apache.hadoop.hbase.regionserver.HStore.notifyChangedReadersObservers(HStore.java:797)
>         at org.apache.hadoop.hbase.regionserver.HStore.updateReaders(HStore.java:784)
>         at org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:755)
>         at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>         at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>         ... 3 more
> Any ideas about this one?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message