phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankit Singhal (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3797) Local Index - Compaction fails on table with local index due to non-increasing bloom keys
Date Mon, 05 Jun 2017 18:28:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037323#comment-16037323
] 

Ankit Singhal edited comment on PHOENIX-3797 at 6/5/17 6:27 PM:
----------------------------------------------------------------

bq. Can we optionally disable the index? I'd be somewhat concerned about performance during
compactions.
We can but if the local index spans in multiple column family/stores then I'll not be sure
when I can enable the indexes as compaction runs independently for each store.

bq. You'd also want to truncate all the column families for the local index.
Column families will be truncated(with an empty file) and re-written(from region path) as
a part of compaction itself. so, we may not need to do it explicitly.

bq. Please let us know, Ankit Singhal, if you think the above is feasible for 4.11.0. If not,
please commit your v3 patch.
As said above, without state management or client executing the repair operation, it will
be difficult to change the state of index accurately. Let me commit this for 4.11.0 and we
can work on adding "CHECK/REPAIR TABLE" functionality(with PHOENIX-3909) in a repair tool(ping
[~sergey.soldatov]) which can be run after hbck to fix this 






was (Author: ankit@apache.org):
bq. Can we optionally disable the index? I'd be somewhat concerned about performance during
compactions.
We can but if the local index spans in multiple column family/stores then I'll not be sure
when I can enable the indexes as compaction runs independently for each store.

bq. You'd also want to truncate all the column families for the local index.
Column families will be truncated(with an empty file) and re-written(from region path) as
a part of compaction itself. so, we may not need to do it explicitly.

bq. Please let us know, Ankit Singhal, if you think the above is feasible for 4.11.0. If not,
please commit your v3 patch.
As said above, without state management or client executing the repair operation, it will
be difficult to change the state of index accurately. Let me commit this for 4.11.0 and we
can work on adding "CHECK/REPAIR TABLE" functionality in a repair tool(ping [~sergey.soldatov])
which can be run after hbck to fix this.





> Local Index - Compaction fails on table with local index due to non-increasing bloom
keys
> -----------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3797
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3797
>             Project: Phoenix
>          Issue Type: Bug
>         Environment: Head of 4.x-HBase-0.98 with PHOENIX-3796 patch applied. HBase 0.98.23-hadoop2
>            Reporter: Mujtaba Chohan
>            Assignee: Ankit Singhal
>            Priority: Blocker
>             Fix For: 4.11.0
>
>         Attachments: PHOENIX-3797.patch, PHOENIX-3797_v2.patch, PHOENIX-3797_v3.patch
>
>
> Compaction fails on table with local index.
> {noformat}
> 2017-04-19 16:37:56,521 ERROR [RS:0;host:59455-smallCompactions-1492644947594] regionserver.CompactSplitThread:
Compaction failed Request = regionName=FHA,00Dxx0000001gES005001xx000003DGPd,1492644985470.92ec6436984981cdc8ef02388005a957.,
storeName=L#0, fileCount=3, fileSize=44.4 M (23.0 M, 10.7 M, 10.8 M), priority=7, time=7442973347247614
> java.io.IOException: Non-increasing Bloom keys: 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01H+&\xA1(00Dxx0000001gER001001xx000003DGPb01739544DCtf
   after 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01I+\xF4\x9Ax00Dxx0000001gER001001xx000003DGPa017115434KTM
  
> 	at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.appendGeneralBloomfilter(StoreFile.java:960)
> 	at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:996)
> 	at org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:428)
> 	at org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:276)
> 	at org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> 	at org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:121)
> 	at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1154)
> 	at org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1559)
> 	at org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:502)
> 	at org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:540)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:722)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message