phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kiran Kumar Maturi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-5137) Index Rebuilder scan increases data table region split time
Date Mon, 18 Feb 2019 08:19:00 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kiran Kumar Maturi updated PHOENIX-5137:
----------------------------------------
    Description: 
[~lhofhansl] [~vincentpoon] [~tdsilva] please review

In order to differentiate between the index rebuilder retries  (UngroupedAggregateRegionObserver.rebuildIndices())
and commits that happen in the loop of UngroupedAggregateRegionObserver.doPostScannerOpen()
as part of  PHOENIX-4600 blockingMemstoreSize was set to -1 for rebuildIndices;
{code:java}
commitBatchWithRetries(region, mutations, -1);{code}
blocks the region split as the check for region closing does not happen  blockingMemstoreSize
> 0
{code:java}
for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize
&& i < 30; i++) {
  try{
   checkForRegionClosing();
   ....
{code}
Plan is to have the check for region closing at least once before committing the batch
{code:java}
checkForRegionClosing();
for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize
&& i < 30; i++) {
  try{
   checkForRegionClosing();
   ....
{code}


  was:
[~lhofhansl] [~vincentpoon] [~tdsilva] please review

In order to differentiate between the index rebuilder retries  (UngroupedAggregateRegionObserver.rebuildIndices())
and commits that happen in the loop of UngroupedAggregateRegionObserver.doPostScannerOpen()
as part of  PHOENIX-4600 blockingMemstoreSize was set to -1 for rebuildIndices;
{code:java}
commitBatchWithRetries(region, mutations, -1);{code}
blocks the region split as the check for region closing does not happen  blockingMemstoreSize
> 0
{code:java}
for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize
&& i < 30; i++) {
  try{
   checkForRegionClosing();
   ....
{code}
Plan is to have the check for region closing at least once before committing the batch
{code:java}
int i = 0;
do {
   try {
     if (i > 0) {
         Thread.sleep(100); 
     }
     checkForRegionClosing();   
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        throw new IOException(e);
    }
}while (blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize
&& i++ < 30);
{code}



> Index Rebuilder scan increases data table region split time
> -----------------------------------------------------------
>
>                 Key: PHOENIX-5137
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5137
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.14.1
>            Reporter: Kiran Kumar Maturi
>            Assignee: Kiran Kumar Maturi
>            Priority: Major
>         Attachments: PHOENIX-5137-4.14-Hbase-1.3.01.patch, PHOENIX-5137-4.14-Hbase-1.3.01.patch
>
>
> [~lhofhansl] [~vincentpoon] [~tdsilva] please review
> In order to differentiate between the index rebuilder retries  (UngroupedAggregateRegionObserver.rebuildIndices())
and commits that happen in the loop of UngroupedAggregateRegionObserver.doPostScannerOpen()
as part of  PHOENIX-4600 blockingMemstoreSize was set to -1 for rebuildIndices;
> {code:java}
> commitBatchWithRetries(region, mutations, -1);{code}
> blocks the region split as the check for region closing does not happen  blockingMemstoreSize
> 0
> {code:java}
> for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() >
blockingMemstoreSize && i < 30; i++) {
>   try{
>    checkForRegionClosing();
>    ....
> {code}
> Plan is to have the check for region closing at least once before committing the batch
> {code:java}
> checkForRegionClosing();
> for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() >
blockingMemstoreSize && i < 30; i++) {
>   try{
>    checkForRegionClosing();
>    ....
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message