hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3514) Speedup HFile.Writer append
Date Fri, 25 Feb 2011 21:31:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12999582#comment-12999582
] 

Ted Yu commented on HBASE-3514:
-------------------------------

v2 of the patches produced the following in TestMultipleTimestamps:
{code}
2011-02-25 13:23:33,208 ERROR [RS_CLOSE_REGION-10.32.41.42,59119,1298668976212-2] handler.CloseRegionHandler(127):
Unrecoverable exception while closing region testReseeksWithOneColumnMiltipleTimestamps,,1298668987716.1f43d31fd29ab39c0ae0e4c176bc0766.,
still finishing close
org.apache.hadoop.hbase.DroppedSnapshotException: region: testReseeksWithOneColumnMiltipleTimestamps,,1298668987716.1f43d31fd29ab39c0ae0e4c176bc0766.
        at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:989)
        at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:894)
        at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:540)
        at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:493)
        at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:118)
        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:151)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.OutOfMemoryError: Java heap space
{code}


> Speedup HFile.Writer append
> ---------------------------
>
>                 Key: HBASE-3514
>                 URL: https://issues.apache.org/jira/browse/HBASE-3514
>             Project: HBase
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.90.0
>            Reporter: Matteo Bertozzi
>            Priority: Minor
>         Attachments: HBASE-3514-append-0.90-v2.patch, HBASE-3514-append-0.90.patch, HBASE-3514-append-trunk-v2.patch,
HBASE-3514-append.patch, HBASE-3514-metaBlock-bsearch.patch
>
>
> Remove double writes when block cache is specified, by using, only, the ByteArrayDataStream.
> baos is flushed with the compress stream on finishBlock.
> On my machines HFilePerformanceEvaluation SequentialWriteBenchmark passes from 4000ms
to 2500ms.
> Running SequentialWriteBenchmark for 1000000 rows took 4247ms.
> Running SequentialWriteBenchmark for 1000000 rows took 4512ms.
> Running SequentialWriteBenchmark for 1000000 rows took 4498ms.
> Running SequentialWriteBenchmark for 1000000 rows took 2697ms.
> Running SequentialWriteBenchmark for 1000000 rows took 2770ms.
> Running SequentialWriteBenchmark for 1000000 rows took 2721ms.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message