hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
Date Wed, 08 Apr 2015 21:44:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14486121#comment-14486121
] 

Hudson commented on HBASE-11544:
--------------------------------

FAILURE: Integrated in HBase-TRUNK #6359 (See [https://builds.apache.org/job/HBase-TRUNK/6359/])
HBASE-13421 Reduce the number of object creations introduced by HBASE-11544 in scan RPC hot
code paths (stack: rev 62d47e175c7c36dc2bd6b225d03978cd6303fc59)
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilter.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpoint.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointWithErrors.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/ColumnAggregationEndpointNullResponse.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicasClient.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueHeap.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestPrefixTree.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestKeepDeletes.java
* hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RowCountEndpoint.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestMultipleColumnPrefixFilter.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/InternalScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/AggregateImplementation.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeCompactor.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
* hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/BulkDeleteEndpoint.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSeekOptimizations.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransaction.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/ClientSideRegionScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoLimitScannerContext.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIntraRowPagination.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/ScannerCallableWithReplicas.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFlusher.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingKeyRange.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestDependentColumnFilter.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestInvocationRecordFilter.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWithBloomError.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestPartialResultsFromClientSide.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java


> [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even
if it means OOME
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11544
>                 URL: https://issues.apache.org/jira/browse/HBASE-11544
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Jonathan Lawlor
>            Priority: Critical
>             Fix For: 2.0.0, 1.1.0
>
>         Attachments: Allocation_Hot_Spots.html, HBASE-11544-addendum-v1.patch, HBASE-11544-addendum-v2.patch,
HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch,
HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch,
HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch,
gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png
>
>
> Running some tests, I set hbase.client.scanner.caching=1000.  Dataset has large cells.
 I kept OOME'ing.
> Serverside, we should measure how much we've accumulated and return to the client whatever
we've gathered once we pass out a certain size threshold rather than keep accumulating till
we OOME.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message