cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aleksey Yeschenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5930) Offline scrubs can choke on broken files
Date Sat, 04 Jan 2014 00:00:51 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13862052#comment-13862052
] 

Aleksey Yeschenko commented on CASSANDRA-5930:
----------------------------------------------

[~jbellis] There are no options. That said, we should probably allow users to override this
behavior, if they prefer losing some of the counters history to not scrubbing at all.

Also, with CASSANDRA-6504 in this becomes a non-issue (for the newly written 'global' 2.1
shards, at least - we *can* repair those after the scrub).

> Offline scrubs can choke on broken files
> ----------------------------------------
>
>                 Key: CASSANDRA-5930
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5930
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jeremiah Jordan
>            Assignee: Tyler Hobbs
>            Priority: Minor
>         Attachments: 5930-v1.patch
>
>
> There are cases where offline scrub can hit an exception and die, like:
> {noformat}
> WARNING: Non-fatal error reading row (stacktrace follows)
> Exception in thread "main" java.io.IOError: java.io.IOError: java.io.EOFException
> 	at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:242)
> 	at org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:121)
> Caused by: java.io.IOError: java.io.EOFException
> 	at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:116)
> 	at org.apache.cassandra.db.compaction.PrecompactedRow.<init>(PrecompactedRow.java:99)
> 	at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:176)
> 	at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:182)
> 	at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:171)
> 	... 1 more
> Caused by: java.io.EOFException
> 	at java.io.RandomAccessFile.readFully(RandomAccessFile.java:399)
> 	at java.io.RandomAccessFile.readFully(RandomAccessFile.java:377)
> 	at org.apache.cassandra.utils.BytesReadTracker.readFully(BytesReadTracker.java:95)
> 	at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:401)
> 	at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:363)
> 	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:120)
> 	at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:37)
> 	at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:144)
> 	at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:234)
> 	at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:112)
> 	... 5 more
> {noformat}
> Since the purpose of offline scrub is to fix broken stuff, it should be more resilient
to broken stuff...



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message