phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3797) Local Index - Compaction fails on table with local index due to non-increasing bloom keys
Date Tue, 30 May 2017 23:58:04 GMT


Lars Hofhansl commented on PHOENIX-3797:

Thanks [].
So if understand this correctly the issue happens when HBCK merges two regions. Then we'll
two sets of local index rows in the L#0 column family right, so as we read and rewrite we
might find a previous key following the later start key of the 2nd region and hence get the
out of order problem. Your solution is to not have the compaction write write the local index
stuff, but instead write it in smaller batches that themselves will not overlap, directly
through the region.

Is that about right?

If would certainly be nice if we could somehow leave the compaction writer do the work, but
I do not see how we can do the scanning then. Perhaps one solution is to wrap two scanner
inside a scanner wrapper and scan from both region start keys. In that way we could get the
re-written keys in the correct order and leave the writing to the compation writer - just
thinking aloud here.

> Local Index - Compaction fails on table with local index due to non-increasing bloom
> -----------------------------------------------------------------------------------------
>                 Key: PHOENIX-3797
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>         Environment: Head of 4.x-HBase-0.98 with PHOENIX-3796 patch applied. HBase 0.98.23-hadoop2
>            Reporter: Mujtaba Chohan
>            Assignee: Ankit Singhal
>            Priority: Blocker
>             Fix For: 4.11.0
>         Attachments: PHOENIX-3797.patch, PHOENIX-3797_v2.patch
> Compaction fails on table with local index.
> {noformat}
> 2017-04-19 16:37:56,521 ERROR [RS:0;host:59455-smallCompactions-1492644947594] regionserver.CompactSplitThread:
Compaction failed Request = regionName=FHA,00Dxx0000001gES005001xx000003DGPd,1492644985470.92ec6436984981cdc8ef02388005a957.,
storeName=L#0, fileCount=3, fileSize=44.4 M (23.0 M, 10.7 M, 10.8 M), priority=7, time=7442973347247614
> Non-increasing Bloom keys: 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01H+&\xA1(00Dxx0000001gER001001xx000003DGPb01739544DCtf
   after 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01I+\xF4\x9Ax00Dxx0000001gER001001xx000003DGPa017115434KTM
> 	at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.appendGeneralBloomfilter(
> 	at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(
> 	at org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(
> 	at org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(
> 	at org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(
> 	at org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(
> 	at org.apache.hadoop.hbase.regionserver.HStore.compact(
> 	at org.apache.hadoop.hbase.regionserver.HRegion.compact(
> 	at org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(
> 	at org.apache.hadoop.hbase.regionserver.CompactSplitThread$
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(
> 	at java.util.concurrent.ThreadPoolExecutor$
> 	at
> {noformat}

This message was sent by Atlassian JIRA

View raw message