From user-return-7248-archive-asf-public=cust-asf.ponee.io@accumulo.apache.org Mon Mar 5 23:22:44 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 8D6E5180608 for ; Mon, 5 Mar 2018 23:22:43 +0100 (CET) Received: (qmail 65884 invoked by uid 500); 5 Mar 2018 22:22:42 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 65874 invoked by uid 99); 5 Mar 2018 22:22:42 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Mar 2018 22:22:42 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 074131A10C4 for ; Mon, 5 Mar 2018 22:22:42 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.13 X-Spam-Level: ** X-Spam-Status: No, score=2.13 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, LOTS_OF_MONEY=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id L01mmTcJz-k5 for ; Mon, 5 Mar 2018 22:22:39 +0000 (UTC) Received: from mail-lf0-f46.google.com (mail-lf0-f46.google.com [209.85.215.46]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 315B95F169 for ; Mon, 5 Mar 2018 22:22:39 +0000 (UTC) Received: by mail-lf0-f46.google.com with SMTP id 70so25544335lfw.2 for ; Mon, 05 Mar 2018 14:22:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=s/PdusWfEf3KY9JOVcBayTAVcZmBIRVYRz2OFyKXAXQ=; b=NHoasCFYesi+hofvjOLdwPANHRMIBcG2OthnF8vOGmbLwwEClBUUZ+hyptnXaRbLOd 6Hnf8ms+PX8w/08EtpETMWpPX4l8l3PQAZYFoXN1QTPqnudCN3xbqVdZCTQTZ9ZAHqv1 DYc90YL6byqZoFR6vg58AC60X2m2XmXIi1cNVJCgUBeKdFW1WPiYujIAOprbAHpJ6hj4 Bgt3UbRL05bBA+qIGXIOQtlnW+iAfXjSODdq/u+DZskTzpbLk8LwPiEwXWhjX2A8mP3/ QMxECMbMZI6VZJiHfAdL3cwD1qMsPblKcSwpKpRRxhFHqixeKRb9pCL6EfIqLJuqJLLA sVBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=s/PdusWfEf3KY9JOVcBayTAVcZmBIRVYRz2OFyKXAXQ=; b=XTHEKcHIinC7Hp5GcTlqPY/MW/wFhL3J2SjFduDCSk/hbLUOfU3skXPEnn964IkPWh bwLPTPWJw0d1uzsbOkIQpr9HEdIOnLe8CAhkwUBvCpVizFmLdzMNs8aeJZv18lJDmIwn pMhweNEvDeO7ZTiQMOTInIOAgvpvRa7EU91TPuMDOQhQzPKwnKapuP2PukeBLP4rubZu 3qejUOjoKbbbEwrvmFzZxtnD67pNee31rIKtmHIANoAj5ix3n3wt6zek3Bh3TLBNwKsl upc85mkAigESE5d5GUOeXayyYOmnC+vz4bxppktkZOI6PK7cge4pKDMp+KNLIlXAzSJA zFQw== X-Gm-Message-State: AElRT7EZFOTX5KkOpUUfc8z5yAgAcJ0FKWpnQL1EJzgiYMIuF0c1p60J 5uV9NmzJ2KyvcZ4T9+g/862kcygOSFxG79MO9kE= X-Google-Smtp-Source: AG47ELvcx2v41ynmS18zKyGQttdCXKcI88GwZM9YMKzdLhYZIO6qr8DP/CLmrSkPJDh9OlAHkP5B/wzw2YFb5c8CXEQ= X-Received: by 10.25.252.4 with SMTP id a4mr11809052lfi.43.1520288558187; Mon, 05 Mar 2018 14:22:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.179.68.217 with HTTP; Mon, 5 Mar 2018 14:22:37 -0800 (PST) In-Reply-To: References: From: Dong Zhou Date: Mon, 5 Mar 2018 14:22:37 -0800 Message-ID: Subject: Re: Adding empty string split causes consistency check to failed for merge command To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary="f4f5e80b8e407788790566b1c373" --f4f5e80b8e407788790566b1c373 Content-Type: text/plain; charset="UTF-8" right, will scan through the metadata with column "file" to be sure. Thanks a lot for the help. :) -Dong On Mon, Mar 5, 2018 at 1:50 PM, Keith Turner wrote: > On Mon, Mar 5, 2018 at 2:16 PM, Dong Zhou wrote: > > Hi Keith, > > > > Thanks for the remedy steps, I tried the instruction and it worked. Yay. > > > > I was trying to fix the issue using similar steps, but I missed out > offline > > and online the table part, which leads me to not able to run merge > command > > with following error: > >> 2018-03-05 19:10:06,037 [master.Master] INFO : Asking > >> tserver1:10011[161d36db90563c3] to chop 3;a< > >> 2018-03-05 19:10:06,038 [state.MergeStats] INFO : Computing next merge > >> state for 3<< which is presently WAITING_FOR_CHOPPED isDelete : false > >> 2018-03-05 19:10:06,038 [state.MergeStats] INFO : 21 tablets are chopped > >> 3<< > >> 2018-03-05 19:10:06,038 [state.MergeStats] INFO : Waiting for 21 chopped > >> tablets to be 22 3<< > >> 2018-03-05 19:10:06,151 [master.Master] INFO : Asking > >> tserver1:10011[161d36db90563c3] to chop 3;a< > >> 2018-03-05 19:10:06,151 [state.MergeStats] INFO : Computing next merge > >> state for 3<< which is presently WAITING_FOR_CHOPPED isDelete : false > >> 2018-03-05 19:10:06,151 [state.MergeStats] INFO : 21 tablets are chopped > >> 3<< > >> 2018-03-05 19:10:06,151 [state.MergeStats] INFO : Waiting for 21 chopped > >> tablets to be 22 3<< > > > > One more question for the supplementary part that the "" split has files. > > Would it be a good idea to run hdfs command to physically move the file > into > > That may be helpful because I am not sure if the Accumulo Garbage > Collector will ever delete the directory when it becomes empty. You > could compact the tablets referencing the file and then delete the dir > when its empty. > > If moving the file, then you need to be careful because multiple files > may reference the tablet. You can grep the metadata table for the > file to be sure. > > > the next tablet folder, and then insert an entry in metadata for it. > > Or it does not matter, I can just reuse the folder that was created for > "" > > split. > > > > Thanks, > > > > -Dong Zhou > > > > > > > > On Mon, Mar 5, 2018 at 9:34 AM, Keith Turner wrote: > >> > >> I omitted something in with my previous email. I assumed the the > >> tablet with an empty row could have no data. However, it can have > >> data. If the tablet has files, they should not be deleted without > >> inserting those files into the next tablet. > >> > >> root@uno accumulo.metadata> addsplits -t foo '' abc > >> root@uno foo> insert '' f q v3 > >> root@uno foo> scan > >> f:q [] v3 > >> a f:q [] v1 > >> b f:q [] v2 > >> > >> Now we see the empty end row tablet has a file. This file would need > >> to be inserted into the next tablet before deleting. > >> > >> root@uno foo> flush -t foo > >> root@uno foo> scan -t accumulo.metadata -c file -r 2; > >> 2; file:hdfs://localhost:8020/accumulo/tables/2/t-0000015/F0000016.rf > >> [] 185,1 > >> > >> > >> On Mon, Mar 5, 2018 at 12:27 PM, Keith Turner wrote: > >> > I replicated this locally and then did the following surgery to fix > it. > >> > > >> > Create a table and determine its table id. The table foo is table id > >> > 2. This tableid is used for rows in metadata table. > >> > > >> > root@uno> createtable foo > >> > root@uno foo> tables -l > >> > accumulo.metadata => !0 > >> > accumulo.replication => +rep > >> > accumulo.root => +r > >> > foo => 2 > >> > trace => 1 > >> > > >> > Add the empty split and another split. > >> > > >> > root@uno foo> addsplits -t foo '' abc > >> > > >> > Inpect the metdata table. > >> > > >> > root@uno foo> scan -t accumulo.metadata -b 2; > >> > 2; loc:1000174d2400003 [] localhost:9997 > >> > 2; srv:dir [] hdfs://localhost:8020/accumulo/tables/2/t-0000005 > >> > 2; srv:lock [] > >> > tservers/localhost:9997/zlock-0000000000$1000174d2400003 > >> > 2; srv:time [] M0 > >> > 2; ~tab:~pr [] \x00 > >> > 2;abc loc:1000174d2400003 [] localhost:9997 > >> > 2;abc srv:dir [] hdfs://localhost:8020/ > accumulo/tables/2/t-0000006 > >> > 2;abc srv:lock [] > >> > tservers/localhost:9997/zlock-0000000000$1000174d2400003 > >> > 2;abc srv:time [] M0 > >> > 2;abc ~tab:~pr [] \x01 > >> > 2< loc:1000174d2400003 [] localhost:9997 > >> > 2< srv:dir [] > >> > hdfs://localhost:8020/accumulo/tables/2/default_tablet > >> > 2< srv:lock [] > >> > tservers/localhost:9997/zlock-0000000000$1000174d2400003 > >> > 2< srv:time [] M0 > >> > 2< ~tab:~pr [] \x01abc > >> > > >> > Insert some data > >> > > >> > root@uno foo> insert a f q v1 > >> > root@uno foo> insert b f q v2 > >> > > >> > Offline table and grant write permission to metadata. > >> > > >> > root@uno foo> offline -t foo -w > >> > root@uno foo> grant Table.WRITE -u root -t accumulo.metadata > >> > > >> > Delete the tablet with an empty end row > >> > > >> > root@uno foo> deletemany -r 2; -t accumulo.metadata > >> > Delete { 2; srv:dir [] } ? y > >> > [DELETED] 2; srv:dir [] > >> > Delete { 2; srv:lock [] } ? y > >> > [DELETED] 2; srv:lock [] > >> > Delete { 2; srv:time [] } ? y > >> > [DELETED] 2; srv:time [] > >> > Delete { 2; ~tab:~pr [] } ? y > >> > [DELETED] 2; ~tab:~pr [] > >> > > >> > Fix the tablet with end row "abc" to indicate no previous tablet > >> > > >> > root@uno foo> table accumulo.metadata > >> > root@uno accumulo.metadata> insert 2;abc ~tab ~pr \x00 > >> > > >> > Bring table online and scan it > >> > > >> > root@uno accumulo.metadata> online foo > >> > root@uno accumulo.metadata> scan -t foo > >> > a f:q [] v1 > >> > b f:q [] v2 > >> > > >> > Remove write permission to metadata table. > >> > > >> > root@uno accumulo.metadata> revoke Table.WRITE -u root -t > >> > accumulo.metadata > >> > > >> > On Thu, Mar 1, 2018 at 8:46 PM, Dong Zhou wrote: > >> >> Hi all, > >> >> > >> >> I noticed that adding an empty string to as a split will cause merge > >> >> command > >> >> failed the consistency check. > >> >> > >> >> 2018-03-01 19:58:46,724 [state.MergeStats] DEBUG: chopped 442 > v.chopped > >> >> 441 > >> >> unassigned 442 v.unassigned 441 verify.total 441 > >> >> 2018-03-01 19:58:46,724 [state.MergeStats] INFO : Merge consistency > >> >> check > >> >> failed 1b<< > >> >> > >> >> Also, is there a way to remove an unwanted split from table. Merge is > >> >> one > >> >> way, but with empty string does not work. > >> >> > >> >> Cheers, > >> >> > >> >> -Dong Zhou > > > > > --f4f5e80b8e407788790566b1c373 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
right, will scan through the metadata with column "fi= le" to be sure.
Thanks a lot for the help. :)

=
-Dong

On Mon, Mar 5, 2018 at 1:50 PM, Keith Turner <keith@deenlo.com>= wrote:
On Mon, M= ar 5, 2018 at 2:16 PM, Dong Zhou <d= zhou26@gmail.com> wrote:
> Hi Keith,
>
> Thanks for the remedy steps, I tried the instruction and it worked. Ya= y.
>
> I was trying to fix the issue using similar steps, but I missed out of= fline
> and online the table part, which leads me to not able to run merge com= mand
> with following error:
>> 2018-03-05 19:10:06,037 [master.Master] INFO : Asking
>> tserver1:10011[161d36db90563c3] to chop 3;a<
>> 2018-03-05 19:10:06,038 [state.MergeStats] INFO : Computing next m= erge
>> state for 3<< which is presently WAITING_FOR_CHOPPED isDelet= e : false
>> 2018-03-05 19:10:06,038 [state.MergeStats] INFO : 21 tablets are c= hopped
>> 3<<
>> 2018-03-05 19:10:06,038 [state.MergeStats] INFO : Waiting for 21 c= hopped
>> tablets to be 22 3<<
>> 2018-03-05 19:10:06,151 [master.Master] INFO : Asking
>> tserver1:10011[161d36db90563c3] to chop 3;a<
>> 2018-03-05 19:10:06,151 [state.MergeStats] INFO : Computing next m= erge
>> state for 3<< which is presently WAITING_FOR_CHOPPED isDelet= e : false
>> 2018-03-05 19:10:06,151 [state.MergeStats] INFO : 21 tablets are c= hopped
>> 3<<
>> 2018-03-05 19:10:06,151 [state.MergeStats] INFO : Waiting for 21 c= hopped
>> tablets to be 22 3<<
>
> One more question for the supplementary part that the "" spl= it has files.
> Would it be a good idea to run hdfs command to physically move the fil= e into

That may be helpful because I am not sure if the Accumulo Garbage Collector will ever delete the directory when it becomes empty.=C2=A0 You could compact the tablets referencing the file and then delete the dir
when its empty.

If moving the file, then you need to be careful because multiple files
may reference the tablet.=C2=A0 You can grep the metadata table for the
file to be sure.

> the next tablet folder, and then insert an entry in metadata for it. > Or it does not matter, I can just reuse the folder that was created fo= r ""
> split.
>
> Thanks,
>
> -Dong Zhou
>
>
>
> On Mon, Mar 5, 2018 at 9:34 AM, Keith Turner <keith@deenlo.com> wrote:
>>
>> I omitted something in with my previous email. I assumed the the >> tablet with an empty row could have no data.=C2=A0 However, it can= have
>> data.=C2=A0 If the tablet has files, they should not be deleted wi= thout
>> inserting those files into the next tablet.
>>
>>=C2=A0 =C2=A0root@uno accumulo.metadata> addsplits -t foo '&= #39; abc
>>=C2=A0 =C2=A0root@uno foo> insert '' f q v3
>>=C2=A0 =C2=A0root@uno foo> scan
>>=C2=A0 =C2=A0 f:q []=C2=A0 =C2=A0 v3
>>=C2=A0 =C2=A0a f:q []=C2=A0 =C2=A0 v1
>>=C2=A0 =C2=A0b f:q []=C2=A0 =C2=A0 v2
>>
>> Now we see the empty end row tablet has a file.=C2=A0 This file wo= uld need
>> to be inserted into the next tablet before deleting.
>>
>>=C2=A0 =C2=A0root@uno foo> flush -t foo
>>=C2=A0 =C2=A0root@uno foo> scan -t accumulo.metadata -c file -r = 2;
>>=C2=A0 =C2=A02; file:hdfs://localhost:8020/accumulo/tables/2/t= -0000015/F0000016.rf
>> []=C2=A0 =C2=A0 185,1
>>
>>
>> On Mon, Mar 5, 2018 at 12:27 PM, Keith Turner <keith@deenlo.com> wrote:
>> > I replicated this locally and then did the following surgery = to fix it.
>> >
>> > Create a table and determine its table id.=C2=A0 The table fo= o is table id
>> > 2.=C2=A0 This tableid is used for rows in metadata table.
>> >
>> >=C2=A0 =C2=A0root@uno> createtable foo
>> >=C2=A0 =C2=A0root@uno foo> tables -l
>> >=C2=A0 =C2=A0accumulo.metadata=C2=A0 =C2=A0 =3D>=C2=A0 =C2= =A0 =C2=A0 =C2=A0 !0
>> >=C2=A0 =C2=A0accumulo.replication =3D>=C2=A0 =C2=A0 =C2=A0 = +rep
>> >=C2=A0 =C2=A0accumulo.root=C2=A0 =C2=A0 =C2=A0 =C2=A0 =3D>= =C2=A0 =C2=A0 =C2=A0 =C2=A0 +r
>> >=C2=A0 =C2=A0foo=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =3D>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A02
>> >=C2=A0 =C2=A0trace=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =3D>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01
>> >
>> > Add the empty split and another split.
>> >
>> >=C2=A0 =C2=A0root@uno foo> addsplits -t foo '' abc<= br> >> >
>> > Inpect the metdata table.
>> >
>> >=C2=A0 =C2=A0root@uno foo> scan -t accumulo.metadata -b 2;<= br> >> >=C2=A0 =C2=A02; loc:1000174d2400003 []=C2=A0 =C2=A0 localhost:= 9997
>> >=C2=A0 =C2=A02; srv:dir []=C2=A0 =C2=A0 hdfs://localhost:8020/= accumulo/tables/2/t-0000005
>> >=C2=A0 =C2=A02; srv:lock []
>> > tservers/localhost:9997/zlock-0000000000$1000174d2400003=
>> >=C2=A0 =C2=A02; srv:time []=C2=A0 =C2=A0 M0
>> >=C2=A0 =C2=A02; ~tab:~pr []=C2=A0 =C2=A0 \x00
>> >=C2=A0 =C2=A02;abc loc:1000174d2400003 []=C2=A0 =C2=A0 localho= st:9997
>> >=C2=A0 =C2=A02;abc srv:dir []=C2=A0 =C2=A0 hdfs://localhost:80= 20/accumulo/tables/2/t-0000006
>> >=C2=A0 =C2=A02;abc srv:lock []
>> > tservers/localhost:9997/zlock-0000000000$1000174d2400003=
>> >=C2=A0 =C2=A02;abc srv:time []=C2=A0 =C2=A0 M0
>> >=C2=A0 =C2=A02;abc ~tab:~pr []=C2=A0 =C2=A0 \x01
>> >=C2=A0 =C2=A02< loc:1000174d2400003 []=C2=A0 =C2=A0 localho= st:9997
>> >=C2=A0 =C2=A02< srv:dir []
>> > hdfs://localhost:8020/accumulo/tables/2/default_tab= let
>> >=C2=A0 =C2=A02< srv:lock []
>> > tservers/localhost:9997/zlock-0000000000$1000174d2400003=
>> >=C2=A0 =C2=A02< srv:time []=C2=A0 =C2=A0 M0
>> >=C2=A0 =C2=A02< ~tab:~pr []=C2=A0 =C2=A0 \x01abc
>> >
>> > Insert some data
>> >
>> >=C2=A0 =C2=A0root@uno foo> insert a f q v1
>> >=C2=A0 =C2=A0root@uno foo> insert b f q v2
>> >
>> > Offline table and grant write permission to metadata.
>> >
>> >=C2=A0 =C2=A0root@uno foo> offline -t foo -w
>> >=C2=A0 =C2=A0root@uno foo> grant Table.WRITE -u root -t acc= umulo.metadata
>> >
>> > Delete the tablet with an empty end row
>> >
>> >=C2=A0 =C2=A0root@uno foo> deletemany -r 2; -t accumulo.met= adata
>> >=C2=A0 =C2=A0Delete { 2; srv:dir [] } ? y
>> >=C2=A0 =C2=A0[DELETED] 2; srv:dir []
>> >=C2=A0 =C2=A0Delete { 2; srv:lock [] } ? y
>> >=C2=A0 =C2=A0[DELETED] 2; srv:lock []
>> >=C2=A0 =C2=A0Delete { 2; srv:time [] } ? y
>> >=C2=A0 =C2=A0[DELETED] 2; srv:time []
>> >=C2=A0 =C2=A0Delete { 2; ~tab:~pr [] } ? y
>> >=C2=A0 =C2=A0[DELETED] 2; ~tab:~pr []
>> >
>> > Fix the tablet with end row "abc" to indicate no pr= evious tablet
>> >
>> >=C2=A0 =C2=A0root@uno foo> table accumulo.metadata
>> >=C2=A0 =C2=A0root@uno accumulo.metadata> insert 2;abc ~tab = ~pr \x00
>> >
>> > Bring table online and scan it
>> >
>> >=C2=A0 =C2=A0root@uno accumulo.metadata> online foo
>> >=C2=A0 =C2=A0root@uno accumulo.metadata> scan -t foo
>> >=C2=A0 =C2=A0a f:q []=C2=A0 =C2=A0 v1
>> >=C2=A0 =C2=A0b f:q []=C2=A0 =C2=A0 v2
>> >
>> > Remove write permission to metadata table.
>> >
>> >=C2=A0 =C2=A0root@uno accumulo.metadata> revoke Table.WRITE= -u root -t
>> > accumulo.metadata
>> >
>> > On Thu, Mar 1, 2018 at 8:46 PM, Dong Zhou <dzhou26@gmail.com> wrote:
>> >> Hi all,
>> >>
>> >> I noticed that adding an empty string to as a split will = cause merge
>> >> command
>> >> failed the consistency check.
>> >>
>> >> 2018-03-01 19:58:46,724 [state.MergeStats] DEBUG: chopped= 442 v.chopped
>> >> 441
>> >> unassigned 442 v.unassigned 441 verify.total 441
>> >> 2018-03-01 19:58:46,724 [state.MergeStats] INFO : Merge c= onsistency
>> >> check
>> >> failed 1b<<
>> >>
>> >> Also, is there a way to remove an unwanted split from tab= le. Merge is
>> >> one
>> >> way, but with empty string does not work.
>> >>
>> >> Cheers,
>> >>
>> >> -Dong Zhou
>
>

--f4f5e80b8e407788790566b1c373--