accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: Adding empty string split causes consistency check to failed for merge command
Date Mon, 05 Mar 2018 17:34:54 GMT
I omitted something in with my previous email. I assumed the the
tablet with an empty row could have no data.  However, it can have
data.  If the tablet has files, they should not be deleted without
inserting those files into the next tablet.

  root@uno accumulo.metadata> addsplits -t foo '' abc
  root@uno foo> insert '' f q v3
  root@uno foo> scan
   f:q []    v3
  a f:q []    v1
  b f:q []    v2

Now we see the empty end row tablet has a file.  This file would need
to be inserted into the next tablet before deleting.

  root@uno foo> flush -t foo
  root@uno foo> scan -t accumulo.metadata -c file -r 2;
  2; file:hdfs://localhost:8020/accumulo/tables/2/t-0000015/F0000016.rf
[]    185,1


On Mon, Mar 5, 2018 at 12:27 PM, Keith Turner <keith@deenlo.com> wrote:
> I replicated this locally and then did the following surgery to fix it.
>
> Create a table and determine its table id.  The table foo is table id
> 2.  This tableid is used for rows in metadata table.
>
>   root@uno> createtable foo
>   root@uno foo> tables -l
>   accumulo.metadata    =>        !0
>   accumulo.replication =>      +rep
>   accumulo.root        =>        +r
>   foo                  =>         2
>   trace                =>         1
>
> Add the empty split and another split.
>
>   root@uno foo> addsplits -t foo '' abc
>
> Inpect the metdata table.
>
>   root@uno foo> scan -t accumulo.metadata -b 2;
>   2; loc:1000174d2400003 []    localhost:9997
>   2; srv:dir []    hdfs://localhost:8020/accumulo/tables/2/t-0000005
>   2; srv:lock []    tservers/localhost:9997/zlock-0000000000$1000174d2400003
>   2; srv:time []    M0
>   2; ~tab:~pr []    \x00
>   2;abc loc:1000174d2400003 []    localhost:9997
>   2;abc srv:dir []    hdfs://localhost:8020/accumulo/tables/2/t-0000006
>   2;abc srv:lock []    tservers/localhost:9997/zlock-0000000000$1000174d2400003
>   2;abc srv:time []    M0
>   2;abc ~tab:~pr []    \x01
>   2< loc:1000174d2400003 []    localhost:9997
>   2< srv:dir []    hdfs://localhost:8020/accumulo/tables/2/default_tablet
>   2< srv:lock []    tservers/localhost:9997/zlock-0000000000$1000174d2400003
>   2< srv:time []    M0
>   2< ~tab:~pr []    \x01abc
>
> Insert some data
>
>   root@uno foo> insert a f q v1
>   root@uno foo> insert b f q v2
>
> Offline table and grant write permission to metadata.
>
>   root@uno foo> offline -t foo -w
>   root@uno foo> grant Table.WRITE -u root -t accumulo.metadata
>
> Delete the tablet with an empty end row
>
>   root@uno foo> deletemany -r 2; -t accumulo.metadata
>   Delete { 2; srv:dir [] } ? y
>   [DELETED] 2; srv:dir []
>   Delete { 2; srv:lock [] } ? y
>   [DELETED] 2; srv:lock []
>   Delete { 2; srv:time [] } ? y
>   [DELETED] 2; srv:time []
>   Delete { 2; ~tab:~pr [] } ? y
>   [DELETED] 2; ~tab:~pr []
>
> Fix the tablet with end row "abc" to indicate no previous tablet
>
>   root@uno foo> table accumulo.metadata
>   root@uno accumulo.metadata> insert 2;abc ~tab ~pr \x00
>
> Bring table online and scan it
>
>   root@uno accumulo.metadata> online foo
>   root@uno accumulo.metadata> scan -t foo
>   a f:q []    v1
>   b f:q []    v2
>
> Remove write permission to metadata table.
>
>   root@uno accumulo.metadata> revoke Table.WRITE -u root -t accumulo.metadata
>
> On Thu, Mar 1, 2018 at 8:46 PM, Dong Zhou <dzhou26@gmail.com> wrote:
>> Hi all,
>>
>> I noticed that adding an empty string to as a split will cause merge command
>> failed the consistency check.
>>
>> 2018-03-01 19:58:46,724 [state.MergeStats] DEBUG: chopped 442 v.chopped 441
>> unassigned 442 v.unassigned 441 verify.total 441
>> 2018-03-01 19:58:46,724 [state.MergeStats] INFO : Merge consistency check
>> failed 1b<<
>>
>> Also, is there a way to remove an unwanted split from table. Merge is one
>> way, but with empty string does not work.
>>
>> Cheers,
>>
>> -Dong Zhou

Mime
View raw message