cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-3427) CompressionMetadata is not shared across threads, we create a new one for each read
Date Thu, 03 Nov 2011 10:49:32 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sylvain Lebresne updated CASSANDRA-3427:
----------------------------------------

    Attachment: 3427_v2.patch

Ok, I think using an object cache is ugly (I know that it was my idea).

At first, I tried going with the natural idea, to add the compressionMetadata as a final field
of SSTableReader and use that everywhere, ensuring we use one per sstable. Turned out that
for SSTableWriter you need to have the metadata existing before the SSTableReader is created
and that seemed like a bit of a mess so I backtracked and decided to go with an object cache
in CompressionMetada, but more out of laziness than anything else.

That was wrong of me to be lazy. We don't need that object cache and if its use is going to
leak out of CompressionMetada (like hard coding the addition of the notifier in the DataTracker
constructor; which defeats the purpose of the notifier abstraction in the first place) then
it's not even clean as far as code is concerned.

Attaching a v2 patch that remove the cache and do a slight modification of the initial idea,
that is it just let CompressedSegmentedFile create the CompressionMetada and have the rest
of the code use that. Turns out that once I plug my brain, it's only a few lines of code.

                
> CompressionMetadata is not shared across threads, we create a new one for each read
> -----------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3427
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3427
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Sylvain Lebresne
>            Assignee: Sylvain Lebresne
>              Labels: compression
>             Fix For: 1.0.2
>
>         Attachments: 3427.patch, 3427_v2.patch, CASSANDRA-3427.patch
>
>
> The CompressionMetada holds the compressed block offsets in memory. Without being absolutely
huge, this is still of non-negligible size as soon as you have a bit of data in the DB. Reallocating
this for each read is a very bad idea.
> Note that this only affect range queries, since "normal" queries uses CompressedSegmentedFile
that does reuse a unique CompressionMetadata instance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message