cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Shuler (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-7199) [dtest] snapshot_test abort logs
Date Sat, 10 May 2014 21:55:30 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael Shuler updated CASSANDRA-7199:
--------------------------------------

    Attachment: jenkins-scratch-2.1_dtest-failed-snapshot_test-dtestdir.tar.gz

dtest DEBUG log (last line created when SIGKILL was sent at job abort):
{noformat}
11:57:44,461 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-HHwzRI
11:57:44,600 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG tmp_commitlog: /tmp/tmp4PKZDL
11:57:49,723 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing first 30,000 rows...
11:57:59,958 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Making snapshot....
11:58:02,171 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Running snapshot cmd: snapshot ks -cf cf -t basic
11:58:03,559 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG snapshot_dir is : /tmp/dtest-HHwzRI/test/node1/data/ks/cf-2494e1a0d77111e393f77ff88ef322c6/snapshots/basic
11:58:03,559 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG snapshot copy is : /tmp/tmpgTsloD
11:58:03,637 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing second 30,000 rows...
11:58:22,562 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing final 5,000 rows...
11:58:25,215 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG node1 commitlog dir: /tmp/dtest-HHwzRI/test/node1/commitlogs
11:58:34,394 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing ccm cluster test at: /tmp/dtest-HHwzRI
11:58:34,404 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-ElgmcU
11:58:39,747 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Restoring snapshot....
11:58:45,70 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Restarting node1..
11:58:59,783 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing snapshot_dir: /tmp/tmpgTsloD
11:58:59,785 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing tmp_commitlog: /tmp/tmp4PKZDL
11:58:59,787 dtest snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing ccm cluster test at: /tmp/dtest-ElgmcU
11:58:59,899 dtest snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-se6EyY
11:59:00,35 dtest snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG tmp_commitlog: /tmp/tmp6c0qNr
11:59:05,146 dtest snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Writing first 30,000 rows...
11:59:15,186 dtest snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Making snapshot....
12:14:15,224 dtest snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Running snapshot cmd: snapshot ks -cf cf -t basic
{noformat}

ccm node1 log (full ccm node tar attached) shows:
{noformat}
INFO  [MemtableFlushWriter:1] 2014-05-09 11:59:05,098 Memtable.java:365 - Completed flushing
/tmp/dtest-se6EyY/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-2-Data.db
(306 bytes) for commitlog position ReplayPosition(segmentId=1399636743311, position=114155)
INFO  [MigrationStage:1] 2014-05-09 11:59:05,121 DefsTables.java:388 - Loading org.apache.cassandra.config.CFMetaData@5341453d[cfId=5189aba0-d771-11e3-ba97-7ff88ef322c6,ksName=ks,cfName=cf,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.1,dcLocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.LongType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.LongType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null}, java.nio.HeapByteBuffer[pos=0 lim=3
cap=3]=ColumnDefinition{name=val, type=org.apache.cassandra.db.marshal.UTF8Type, kind=REGULAR,
componentIndex=0, indexName=null, indexType=null}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
"rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers={}]
INFO  [MigrationStage:1] 2014-05-09 11:59:05,125 ColumnFamilyStore.java:284 - Initializing
ks.cf
WARN  [Thrift:1] 2014-05-09 11:59:05,152 FBUtilities.java:359 - Trigger directory doesn't
exist, please create it and try again.
ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-05-09 11:59:09,844 AbstractCommitLogService.java:125
- Commit log sync failed
org.apache.cassandra.io.FSWriteError: java.lang.IndexOutOfBoundsException
        at org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:337)
~[main/:na]
        at org.apache.cassandra.db.commitlog.CommitLog.sync(CommitLog.java:173) ~[main/:na]
        at org.apache.cassandra.db.commitlog.AbstractCommitLogService$1.run(AbstractCommitLogService.java:81)
~[main/:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
Caused by: java.lang.IndexOutOfBoundsException: null
        at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_55]
        at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:796) ~[na:1.7.0_55]
        at org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:319)
~[main/:na]
        ... 3 common frames omitted
ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-05-09 11:59:12,516 AbstractCommitLogService.java:125
- Commit log sync failed
org.apache.cassandra.io.FSWriteError: java.lang.IndexOutOfBoundsException
        at org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:337)
~[main/:na]
        at org.apache.cassandra.db.commitlog.CommitLog.sync(CommitLog.java:173) ~[main/:na]
        at org.apache.cassandra.db.commitlog.AbstractCommitLogService$1.run(AbstractCommitLogService.java:81)
~[main/:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
Caused by: java.lang.IndexOutOfBoundsException: null
        at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_55]
        at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:796) ~[na:1.7.0_55]
        at org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:319)
~[main/:na]
        ... 3 common frames omitted
ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-05-09 11:59:14,798 AbstractCommitLogService.java:125
- Commit log sync failed
org.apache.cassandra.io.FSWriteError: java.lang.IndexOutOfBoundsException
        at org.apache.cassandra.db.commitlog.CommitLogSegment.sync(CommitLogSegment.java:337)
~[main/:na]
<...>
{noformat}

> [dtest] snapshot_test abort logs
> --------------------------------
>
>                 Key: CASSANDRA-7199
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7199
>             Project: Cassandra
>          Issue Type: Test
>          Components: Tests
>            Reporter: Michael Shuler
>            Assignee: Michael Shuler
>            Priority: Minor
>              Labels: qa-resolved
>             Fix For: 2.1 rc1
>
>         Attachments: 7199.txt, jenkins-scratch-2.1_dtest-failed-snapshot_test-dtestdir.tar.gz
>
>
> Test hung twice on 2.1 in the same manner while trying a new ccm branch as a scratch
jenkins job
> {noformat}
> 11:57:44 dont_test_archive_commitlog (snapshot_test.TestArchiveCommitlog) ... Requested
creating snapshot(s) for [ks] with snapshot name [basic]
> 11:58:03 Snapshot directory: basic
> 11:58:41 Established connection to initial hosts
> 11:58:41 Opening sstables and calculating sections to stream
> 11:58:41 Streaming relevant part of /tmp/tmpgTsloD/ks/cf/ks-cf-ka-1-Data.db to [/127.0.0.1]
> 11:58:41 
> progress: [/127.0.0.1]0:1/1 100% total: 100% 0  MB/s(avg: 0 MB/s)
> progress: [/127.0.0.1]0:1/1 100% total: 100% 0  MB/s(avg: 0 MB/s)
> 11:58:42 Summary statistics: 
> 11:58:42    Connections per host:         : 1         
> 11:58:42    Total files transferred:      : 1         
> 11:58:42    Total bytes transferred:      : 527659    
> 11:58:42    Total duration (ms):          : 2384      
> 11:58:42    Average transfer rate (MB/s): : 0         
> 11:58:42    Peak transfer rate (MB/s):    : 0         
> 11:58:42 
> 11:58:59 ok
> 11:58:59 test_archive_commitlog (snapshot_test.TestArchiveCommitlog) ... rm: cannot remove
`/tmp/tmp6c0qNr/*': No such file or directory
> 12:14:15 Build timed out (after 15 minutes). Marking the build as aborted.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message