accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russ Weeks <rwe...@newbrightidea.com>
Subject Re: Seeing lots of "Internal error processing waitForTableOperation"
Date Thu, 27 Feb 2014 04:49:24 GMT
Hi, Josh,

Thanks for responding so quickly... sorry, I should've mentioned that I'm
running Accumulo 1.5.0.

Nothing really helpful that I can see in the monitor or debug monitor logs.
It just interleaves the 2 errors above from the state.MetaDataTableScanner
and util.TabletIterator loggers.

-Russ


On Wed, Feb 26, 2014 at 8:37 PM, Josh Elser <josh.elser@gmail.com> wrote:

> Hi Russ,
>
> What version of Accumulo are you running?
>
> Any other exceptions in the Monitor?
>
> - Josh
>
>
> On 2/26/14, 11:10 PM, Russ Weeks wrote:
>
>> Hi, All,
>>
>> I can't delete any tables through the accumulo shell, I keep getting the
>> message: "Internal error processing waitForTableOperation".
>>
>> Recently my accumulo cluster has started showing this error repeatedly
>> in the master log:
>> [state.MetaDataTableScanner] WARN : No prev-row for key extent: {z< : []
>> 9223372036854775807
>> false=  loc243f8e8ad6f5875>. 10.128.1.17:9997 ~tab ~pr-�  �}
>>
>> And this error in the monitor log:
>> [util.TabletIterator] WARN : gc:agile01 Metadata problem : tablet !0;!0<
>> has no prev end row
>>
>> And this in the GC log:
>> [util.TabletIterator] WARN : Metadata problem : tablet !0;!0< has no
>> prev end row
>> [util.TabletIterator] INFO : Resetting !METADATA scanner to [!0 : []
>> 9223372036854775807 false,~ : [] 9223372036854775807 false)
>>
>> Maybe coincidence, but it seems like things started getting weird when I
>> added a WholeRowIterator through the accumulo shell (which is nonsense,
>> I know). The iterators on one of my tables (36M entries) look like this:
>>
>> -    Iterator WholeRowIterator, scan scope options:
>> -        iteratorPriority = 50
>> -        iteratorClassName =
>> org.apache.accumulo.core.iterators.user.WholeRowIterator
>> -    Iterator rowdel, majc scope options:
>> -        iteratorPriority = 30
>> -        iteratorClassName =
>> org.apache.accumulo.core.iterators.user.RowDeletingIterator
>> -    Iterator transformer, majc scope options:
>> -        iteratorPriority = 15
>> -        iteratorClassName =
>> com.phemi.agile.datapolicy.DataColumnTransformer
>> -        maxBufferSize = 256M
>> -    Iterator vers, majc scope options:
>> -        iteratorPriority = 20
>> -        iteratorClassName =
>> org.apache.accumulo.core.iterators.user.VersioningIterator
>> -        maxVersions = 1
>> -    Iterator vers, minc scope options:
>> -        iteratorPriority = 20
>> -        iteratorClassName =
>> org.apache.accumulo.core.iterators.user.VersioningIterator
>> -        maxVersions = 1
>> -    Iterator vers, scan scope options:
>> -        iteratorPriority = 20
>> -        iteratorClassName =
>> org.apache.accumulo.core.iterators.user.VersioningIterator
>> -        maxVersions = 1
>>
>> I tried deleting DataColumnTransformer, RowDeletingIterator and
>> WholeRowIterator. No errors were returned from the deleteiter command in
>> the shell but after running listiter, the WholeRowIterator and
>> RowDeletingIterator are still there.
>>
>> I *am* still able to create tables, but tables that I create can't then
>> be deleted.
>>
>> Any advice appreciated,
>> -Russ
>>
>>

Mime
View raw message