Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Subject: Re: lots of DigestMismatchException in cassandra3
To: <user@cassandra.apache.org>
References: <2E2FD4ACA211C145911599AE3E6C67860C904C94@SG1P-ERFMMBX03.ERF.thomson.com>
From: Tommy Stendahl <tommy.stendahl@ericsson.com>
Organization: Ericsson AB
Message-ID: <7e512db0-d5d4-7c95-69d8-4fbb5099ab33@ericsson.com>
Date: Thu, 17 Nov 2016 10:16:10 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <2E2FD4ACA211C145911599AE3E6C67860C904C94@SG1P-ERFMMBX03.ERF.thomson.com>
Content-Type: multipart/alternative;
	boundary="------------E85CF69481E71E00B13D6326"
Received-SPF: None (protection.outlook.com: ericsson.com does not designate
 permitted sender hosts)
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2016 09:16:13.2005 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB1437
X-OriginatorOrg: ericsson.com
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprKKsWRmVeSWpSXmKPExsUyM2K7um5vqW6EwfTXghbn3v1jdmD0+Hbt
	G3sAYxSXTUpqTmZZapG+XQJXxtUZa1kLrs5jrHi7+z9rA+PSBsYuRk4OCQETiV/H9jF3MXJx
	CAmsY5ToevKSFcI5wSjxse03E4jDItDLLPFt7mOgDAcHo0CcROtFD4iiv4wSLd1HmUFGCQtY
	Sbw8+4YFxBYRkJfYeG4OK4gtJBAqsfDMJqA4OwebgJHEVyeQKL+ApMSGht1gnbwC9hIT/01n
	A7FZBFQlJm++CxYXFYiQ2PR1DgtEjaDEyZlPwGxOgTCJdYdus4PYzED2nDP3oJ5RkPiz/igL
	yGkSAu2MEj2X37JDnKAhcWPDT3aIRD+zxNIZ15kgOnwlzm9uZ4exDzXuZIGwT3BLLPjjCNHw
	lE3i5snvUIlsiSVnZrJC2NYSvw5MYoQouswo8X3PalYIZwuLxPLXa6GqZCQOzZzNBJG4xyrx
	Z/kbZoijUiW23Ghhm8CoOQvJg7OQPDULGNzMwMB5sLUMIiwvsf3tHGYIW1/i+p37rBC2tsSy
	ha+ZFzCyrWIULU4tLs5NNzLWSy3KTC4uzs/Ty0st2cQITCwHt/zW3cG4+rXjIUYBDkYlHt6C
	Ep0IIdbEsuLK3EOMEhzMSiK8bZm6EUK8KYmVValF+fFFpTmpxYcYpTlYlMR5zVbeDxcSSE8s
	Sc1OTS1ILYLJMnFwSjUw8r6ZH/Wu+9WGaOP1B7pObngz4fHkU8GMN9Lns0sXzFj0KUrpS/K0
	S9xO39X9nNjZTkx1kZ9x1XbRk5fPP6lvlKwMVDlX1uosp/CbL09TcYb1xDn2Yg0GWmsWu03c
	2PeMc+nNtzkT3IWzJBjNL3WFKxpI2WQ82r5CRv7uov0s629zsKv9PzpFUImlOCPRUIu5qDgR
	AOQgkXYoAwAA
archived-at: Thu, 17 Nov 2016 09:17:14 -0000

--------------E85CF69481E71E00B13D6326
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

Could it be CASSANDRA-12090 
<https://issues.apache.org/jira/browse/CASSANDRA-12090>?

Regards,
/Tommy

On 2016-11-17 09:47, Adeline.Pan@thomsonreuters.com wrote:
>
> Hi,
>
> We are using Cassandra 3.7 and we have a single datacenter of 4 nodes, 
> RF =3, and the consistency level is quorum.
>
> I found *lots* *of* DigestMismatchException in debug.log, as below:
>
> DEBUG [ReadRepairStage:11] 2016-11-17 08:24:57,066 
> ReadCallback.java:235 - Digest mismatch:
>
> org.apache.cassandra.service.DigestMismatchException: Mismatch for key 
> DecoratedKey(-7187709033646443438, 
> 0008000001586f95380000000405f72c7a0000040000000300) 
> (c8d67cfae7be8470ca1786eaa0d8968e vs 3aa974d3eca302682c6f31ca250ea641)
>
> at 
> org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:85) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:226) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
> [na:1.8.0_25]
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
> [na:1.8.0_25]
>
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
>
> INFO [HintsDispatcher:5] 2016-11-17 08:25:06,958 HintsStore.java:126 - 
> Deleted hint file 
> bcd84017-6068-4a76-be89-96c0130599f3-1479371095876-1.hints
>
> INFO [HintsDispatcher:5] 2016-11-17 08:25:06,962 
> HintsDispatchExecutor.java:263 - Finished hinted handoff of file 
> bcd84017-6068-4a76-be89-96c0130599f3-1479371095876-1.hints to endpoint 
> bcd84017-6068-4a76-be89-96c0130599f3
>
> DEBUG [GossipStage:1] 2016-11-17 08:25:32,551 FailureDetector.java:456 
> - Ignoring interval time of 2003165297 for /10.52.153.107
>
> DEBUG [CompactionExecutor:9] 2016-11-17 08:25:33,034 
> CompactionTask.java:223 - Compacted 
> (3b3d7600-ac9f-11e6-8b55-03fd515998e4) 4 sstables to 
> [/data/c878jrn/data/pc_object_data_beta/vsc_data-53935ff0a74811e69ab46393a19c464a/mb-2256-big,] 
> to level=0.  85.011MiB to 53.580MiB (~63% of original) in 75,524ms.  
> Throughput = 726.471KiB/s. 0 total partitions merged to 15,552.  
> Partition merge counts were {1:11623, 2:3543, 3:385, 4:1, }
>
> ERROR [MessagingService-Incoming-/10.52.153.109] 2016-11-17 
> 08:25:42,126 CassandraDaemon.java:217 - Exception in thread 
> Thread[MessagingService-Incoming-/10.52.153.109,5,main]
>
> java.io.IOError: java.io.IOException: Corrupt empty row found in 
> unfiltered partition
>
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:224) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:212) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize30(PartitionUpdate.java:681) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSerializer.deserialize(PartitionUpdate.java:642) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:331) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:350) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:287) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:114) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:190) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> Caused by: java.io.IOException: Corrupt empty row found in unfiltered 
> partition
>
> at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:388) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:219) 
> ~[apache-cassandra-3.7.jar:3.7]
>
> ... 11 common frames omitted
>
> Is this normal?
>
> --
>
> Regards, Adeline
>


--------------E85CF69481E71E00B13D6326
Content-Type: text/html; charset="windows-1252"
Content-Transfer-Encoding: quoted-printable

<html>
  <head>
    <meta content=3D"text/html; charset=3Dwindows-1252"
      http-equiv=3D"Content-Type">
  </head>
  <body bgcolor=3D"#FFFFFF" text=3D"#000000">
    <p>Hi,</p>
    <p>Could it be <a
        href=3D"https://issues.apache.org/jira/browse/CASSANDRA-12090">CA=
SSANDRA-12090</a>?</p>
    <p>Regards,<br>
      /Tommy<br>
    </p>
    <div class=3D"moz-cite-prefix">On 2016-11-17 09:47,
      <a class=3D"moz-txt-link-abbreviated" href=3D"mailto:Adeline.Pan@th=
omsonreuters.com">Adeline.Pan@thomsonreuters.com</a> wrote:<br>
    </div>
    <blockquote
cite=3D"mid:2E2FD4ACA211C145911599AE3E6C67860C904C94@SG1P-ERFMMBX03.ERF.t=
homson.com"
      type=3D"cite">
      <meta http-equiv=3D"Content-Type" content=3D"text/html;
        charset=3Dwindows-1252">
      <meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered
        medium)">
      <style><!--
/* Font Definitions */
@font-face
	{font-family:SimSun;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:SimSun;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@SimSun";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-compose;
	font-family:"Calibri","sans-serif";
	color:windowtext;}
=2EMsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri","sans-serif";}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
      <div class=3D"WordSection1">
        <p class=3D"MsoNormal">Hi,<o:p></o:p></p>
        <p class=3D"MsoNormal"><o:p>=A0</o:p></p>
        <p class=3D"MsoNormal">We are using Cassandra 3.7 and we have a
          single datacenter of 4 nodes, RF =3D3, and the consistency leve=
l
          is quorum.
          <o:p></o:p></p>
        <p class=3D"MsoNormal">I found <b>lots</b> <b>of</b>
          DigestMismatchException in debug.log, as below:<o:p></o:p></p>
        <table class=3D"MsoTableGrid"
          style=3D"border-collapse:collapse;border:none" border=3D"1"
          cellpadding=3D"0" cellspacing=3D"0">
          <tbody>
            <tr>
              <td style=3D"width:6.65in;border:solid windowtext
                1.0pt;padding:0in 5.4pt 0in 5.4pt" valign=3D"top"
                width=3D"638">
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">DEBUG
                    [ReadRepairStage:11] 2016-11-17 08:24:57,066
                    ReadCallback.java:235 - Digest mismatch:<o:p></o:p></=
span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">org.apache.cassandra.service.DigestMismatchExce=
ption:
                    Mismatch for key DecoratedKey(-7187709033646443438,
                    0008000001586f95380000000405f72c7a0000040000000300)
                    (c8d67cfae7be8470ca1786eaa0d8968e vs
                    3aa974d3eca302682c6f31ca250ea641)<o:p></o:p></span></=
p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.service.DigestResolver.resolve(DigestResolver.java:8=
5)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallb=
ack.java:226)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java=
:1142)
                    [na:1.8.0_25]<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.jav=
a:617)
                    [na:1.8.0_25]<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at java.lang.Thread.run(Thread.java:745)
                    [na:1.8.0_25]<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">INFO=A0
                    [HintsDispatcher:5] 2016-11-17 08:25:06,958
                    HintsStore.java:126 - Deleted hint file
                    bcd84017-6068-4a76-be89-96c0130599f3-1479371095876-1.=
hints<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">INFO=A0
                    [HintsDispatcher:5] 2016-11-17 08:25:06,962
                    HintsDispatchExecutor.java:263 - Finished hinted
                    handoff of file
                    bcd84017-6068-4a76-be89-96c0130599f3-1479371095876-1.=
hints
                    to endpoint bcd84017-6068-4a76-be89-96c0130599f3<o:p>=
</o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">DEBUG
                    [GossipStage:1] 2016-11-17 08:25:32,551
                    FailureDetector.java:456 - Ignoring interval time of
                    2003165297 for /10.52.153.107<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">DEBUG
                    [CompactionExecutor:9] 2016-11-17 08:25:33,034
                    CompactionTask.java:223 - Compacted
                    (3b3d7600-ac9f-11e6-8b55-03fd515998e4) 4 sstables to
[/data/c878jrn/data/pc_object_data_beta/vsc_data-53935ff0a74811e69ab46393=
a19c464a/mb-2256-big,]
                    to level=3D0.=A0 85.011MiB to 53.580MiB (~63% of
                    original) in 75,524ms.=A0 Throughput =3D 726.471KiB/s=
=2E=A0
                    0 total partitions merged to 15,552.=A0 Partition
                    merge counts were {1:11623, 2:3543, 3:385, 4:1, }<o:p=
></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">ERROR
                    [MessagingService-Incoming-/10.52.153.109]
                    2016-11-17 08:25:42,126 CassandraDaemon.java:217 -
                    Exception in thread
                    Thread[MessagingService-Incoming-/10.52.153.109,5,mai=
n]<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">java.io.IOError:
                    java.io.IOException: Corrupt empty row found in
                    unfiltered partition<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNex=
t(UnfilteredRowIteratorSerializer.java:224)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNex=
t(UnfilteredRowIteratorSerializer.java:212)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java=
:47)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSeriali=
zer.deserialize30(PartitionUpdate.java:681)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.partitions.PartitionUpdate$PartitionUpdateSeriali=
zer.deserialize(PartitionUpdate.java:642)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.=
java:331)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.=
java:350)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.=
java:287)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
                    org.apache.cassandra.net.MessageIn.read(MessageIn.jav=
a:114)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcp=
Connection.java:190)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTc=
pConnection.java:178)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.=
java:92)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">Caused
                    by: java.io.IOException: Corrupt empty row found in
                    unfiltered partition<o:p></o:p></span></p>
                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredS=
erializer.java:388)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNex=
t(UnfilteredRowIteratorSerializer.java:219)
                    ~[apache-cassandra-3.7.jar:3.7]<o:p></o:p></span></p>=

                <p class=3D"MsoNormal"><span
style=3D"color:#7F7F7F;mso-style-textfill-fill-color:#7F7F7F;mso-style-te=
xtfill-fill-alpha:100.0%">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0
                    ... 11 common frames omitted</span><o:p></o:p></p>
              </td>
            </tr>
          </tbody>
        </table>
        <p class=3D"MsoNormal"><o:p>=A0</o:p></p>
        <p class=3D"MsoNormal">Is this normal?<o:p></o:p></p>
        <p class=3D"MsoNormal"><o:p>=A0</o:p></p>
        <p class=3D"MsoNormal"><o:p>=A0</o:p></p>
        <p class=3D"MsoNormal"><span
            style=3D"color:#1F497D;mso-fareast-language:ZH-CN">--<o:p></o=
:p></span></p>
        <p class=3D"MsoNormal"><span
            style=3D"color:#1F497D;mso-fareast-language:ZH-CN">Regards,
            Adeline
            <o:p></o:p></span></p>
        <p class=3D"MsoNormal"><o:p>=A0</o:p></p>
        <p class=3D"MsoNormal"><o:p>=A0</o:p></p>
      </div>
    </blockquote>
    <br>
  </body>
</html>

--------------E85CF69481E71E00B13D6326--