Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0ACE410676 for ; Tue, 23 Dec 2014 07:58:14 +0000 (UTC) Received: (qmail 34217 invoked by uid 500); 23 Dec 2014 07:58:13 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 34173 invoked by uid 500); 23 Dec 2014 07:58:13 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 34161 invoked by uid 99); 23 Dec 2014 07:58:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Dec 2014 07:58:13 +0000 Date: Tue, 23 Dec 2014 07:58:13 +0000 (UTC) From: "Marcus Eriksson (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (CASSANDRA-8192) Better error logging on corrupt compressed SSTables: currently AssertionError in Memory.java MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256721#comment-14256721 ] Marcus Eriksson edited comment on CASSANDRA-8192 at 12/23/14 7:57 AM: ---------------------------------------------------------------------- I need to create a file like this: {code} DataOutputStream dos = new DataOutputStream(new FileOutputStream("/tmp/comp_info")); dos.writeUTF("LZ4Compressor"); // compressor name dos.writeInt(0);// option count dos.writeInt(0);//chunk length dos.writeLong(0); // data length dos.writeInt(0); // chunk count dos.close(); {code} to reproduce (dunno how common it is for file corruption to create files like that (ie, being zeroed out after a number of bytes)?) Anyway, could we add an assert where we serialize the CompressionInfo to disk to make sure we don't have a bug there? And, if it is corrupt data, we should probably check if chunkCount <= 0? was (Author: krummas): I need to create a file like this: {code} DataOutputStream dos = new DataOutputStream(new FileOutputStream("/tmp/comp_info")); dos.writeUTF("LZ4Compressor"); // compressor name dos.writeInt(0);// option count dos.writeInt(0);//chunk length dos.writeLong(0); // data length dos.writeInt(0); // chunk count dos.close(); {code} to reproduce (dunno how common it is for file corruption to create files like that?) Anyway, could we add an assert where we serialize the CompressionInfo to disk to make sure we don't have a bug there? And, if it is corrupt data, we should probably check if chunkCount <= 0? > Better error logging on corrupt compressed SSTables: currently AssertionError in Memory.java > -------------------------------------------------------------------------------------------- > > Key: CASSANDRA-8192 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8192 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67 > Reporter: Andreas Schnitzerling > Assignee: Joshua McKenzie > Priority: Minor > Fix For: 2.1.3 > > Attachments: 8192_v1.txt, cassandra.bat, cassandra.yaml, logdata-onlinedata-ka-196504-CompressionInfo.zip, printChunkOffsetErrors.txt, system-compactions_in_progress-ka-47594-CompressionInfo.zip, system-sstable_activity-jb-25-Filter.zip, system.log, system_AssertionTest.log > > > Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during start up. > {panel:title=system.log} > ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 - Exception in thread Thread[SSTableBatchOpen:1,5,main] > java.lang.AssertionError: null > at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:135) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302) ~[apache-cassandra-2.1.1.jar:2.1.1] > at org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438) ~[apache-cassandra-2.1.1.jar:2.1.1] > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_55] > at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55] > at java.lang.Thread.run(Unknown Source) [na:1.7.0_55] > {panel} > In the attached log you can still see as well CASSANDRA-8069 and CASSANDRA-6283. -- This message was sent by Atlassian JIRA (v6.3.4#6332)