phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3797) Local Index - Compaction fails on table with local index due to non-increasing bloom keys
Date Sat, 03 Jun 2017 00:23:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035680#comment-16035680
] 

James Taylor edited comment on PHOENIX-3797 at 6/3/17 12:22 AM:
----------------------------------------------------------------

The disable index could be plugged in as the else block here:
{code}
+                if (c.getEnvironment().getConfiguration().getBoolean(LOCAL_INDEX_AUTOMATIC_REPAIR,
true)) {
+                    LOG.info("Starting automatic repair of local Index for region:"
+                            + c.getEnvironment().getRegion().getRegionInfo());
+                    repairScanner = getRepairScanner(c.getEnvironment(), store);
+                }
{code}
You'd need to run a scan against SYSTEM.CATALOG and find all tables with a link to the local
index physical table and invoke the endpoint coprocessor API to disable the index. You'd also
want to truncate all the column families for the local index. Not sure if there's any trickiness
in the disable to prevent the partial index rebuild to kick in (as in this case you'd want
to ensure that the index is rebuilt from scratch). See PHOENIX-3909 for some potential future
work.

Please let us know, [~ankit@apache.org], if you think the above is feasible for 4.11.0. If
not, please commit your v3 patch. 


was (Author: jamestaylor):
The disable index could be plugged in as the else block here:
{code}
+                if (c.getEnvironment().getConfiguration().getBoolean(LOCAL_INDEX_AUTOMATIC_REPAIR,
true)) {
+                    LOG.info("Starting automatic repair of local Index for region:"
+                            + c.getEnvironment().getRegion().getRegionInfo());
+                    repairScanner = getRepairScanner(c.getEnvironment(), store);
+                }
{code}
You'd need to run a scan against SYSTEM.CATALOG and find all tables with a link to the local
index physical table and invoke the endpoint coprocessor API to disable the index. You'd also
want to truncate all the column families for the local index. Not sure if there's any trickiness
in the disable to prevent the partial index rebuild to kick in (as in this case you'd want
to ensure that the index is rebuilt from scratch). See PHOENIX-3909 for some potential future
work.

> Local Index - Compaction fails on table with local index due to non-increasing bloom
keys
> -----------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3797
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3797
>             Project: Phoenix
>          Issue Type: Bug
>         Environment: Head of 4.x-HBase-0.98 with PHOENIX-3796 patch applied. HBase 0.98.23-hadoop2
>            Reporter: Mujtaba Chohan
>            Assignee: Ankit Singhal
>            Priority: Blocker
>             Fix For: 4.11.0
>
>         Attachments: PHOENIX-3797.patch, PHOENIX-3797_v2.patch, PHOENIX-3797_v3.patch
>
>
> Compaction fails on table with local index.
> {noformat}
> 2017-04-19 16:37:56,521 ERROR [RS:0;host:59455-smallCompactions-1492644947594] regionserver.CompactSplitThread:
Compaction failed Request = regionName=FHA,00Dxx0000001gES005001xx000003DGPd,1492644985470.92ec6436984981cdc8ef02388005a957.,
storeName=L#0, fileCount=3, fileSize=44.4 M (23.0 M, 10.7 M, 10.8 M), priority=7, time=7442973347247614
> java.io.IOException: Non-increasing Bloom keys: 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01H+&\xA1(00Dxx0000001gER001001xx000003DGPb01739544DCtf
   after 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01I+\xF4\x9Ax00Dxx0000001gER001001xx000003DGPa017115434KTM
  
> 	at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.appendGeneralBloomfilter(StoreFile.java:960)
> 	at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:996)
> 	at org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:428)
> 	at org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:276)
> 	at org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> 	at org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:121)
> 	at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1154)
> 	at org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1559)
> 	at org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:502)
> 	at org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:540)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:722)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message