cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (Updated) (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-3411) Pre-allocated, Recycled Commitlog Segment Files
Date Thu, 10 Nov 2011 06:01:55 GMT


Jonathan Ellis updated CASSANDRA-3411:

    Attachment: 3411-v5.txt

v5 attached.  minor changes like log lines to debug.  also changed the LogRecordAdder rotate
check to >=.

I think we still need to solve the segment sizing problem though.  If you don't want to actually
record the high water mark, how about recycling the segments we just replayed?

Basically I don't want to inflict bad performance on people who have > 500MB of unflushed
data while CL "extends" itself.  Once is acceptable, but every server restart is not.

I think the logic around the size cap needs some work too.  We should be *strict* about keeping
under the cap -- the idea is we could be given a partition that size, so overflowing is Bad.

If we get stricter there then I think we can simplify the "high water mark" tracking: you
never discard segments (since you never violate the size constraints) so the high water mark
must be exactly the current number of segments.

submitCreateFreshSegment is unused.  Oversight?

Why does recycle bother creating a new Segment object?  (Contrawise, I think I'd prefer creating
a new SequentialWriter object, to using seek here.)

Is the order in CL.shutdownBlocking correct?  Seems to me it would be less dangerous to leave
the allocator active until after we're done processing mutations.
> Pre-allocated, Recycled Commitlog Segment Files
> -----------------------------------------------
>                 Key: CASSANDRA-3411
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Rick Branson
>            Priority: Minor
>         Attachments: 001-pre-allocated-recycled-commitlog-segment-files-CASSANDRA-3411.patch,
003-pre-allocated-recycled-commitlog-segment-files-CASSANDRA-3411.patch, 004-pre-allocated-recycled-commitlog-segment-files-CASSANDRA-3411.patch,
3411-cleaned.txt, 3411-v5.txt
> An approach for improving commitlog performance is to pre-allocate the full 128MB segment
files and reuse them once all the mutations have been flushed. Pre-allocation allows writes
to be performed without modifying the file size metadata, and should (in theory) allow the
filesystem to allocate a contiguous block of space for the file. Recycling the segment files
prevents the overhead of pre-allocation from impacting overall performance.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message