cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2521) Move away from Phantom References for Compaction/Memtable
Date Tue, 28 Jun 2011 09:16:28 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056406#comment-13056406
] 

Sylvain Lebresne commented on CASSANDRA-2521:
---------------------------------------------

bq. but I guess these may be results of references acquired which are not freed as the streaming
fills up the disk and fails.

Yes, until we have CASSANDRA-2433 (rebased with this), we don't detect failing streaming and
thus never delete the files (until restart). Which btw make me say that we better have CASSANDRA-2433
if we have this ticket. But that was the plan anyway.

bq. there are no less but 53 tmp files. A lot of concurrent streams here!

Though it is not related to this ticket, I'll note that CASSANDRA-2816 only stagger the merkle
tree creation, not the streaming. That is, the streaming will be staggered to some extends,
but if the streaming part is much longer than the merkle tree creation one, you will still
have lots of concurrent stream going on. But -tmp files also includes the compaction that
are going on, and failed repair leaves -tmp file around, which could also help explaining
there large number. In any case, this is not related to the issue at hand :)

{quote}
However I noticed this in the log:
INFO [Thread-185] 2011-06-28 05:01:15,390 StorageService.java (line 2083) requesting GC to
free disk space
I guess we can get rid of that?
{quote}

In some cases (mmap with non-sun jvm at least) we are still relying on the GC to free space.


Terje, if you can confirm that you didn't saw something utterly wrong with the last patch
(related to that patch, no repair), I'll commit it. I think having it in trunk quicker will
help with having more testing quicker. And given that we don't want to have bugs in our "force
unmapping" it'll be a good thing. In particular, could be good to have someone try that on
windows.

> Move away from Phantom References for Compaction/Memtable
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-2521
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2521
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Chris Goffinet
>            Assignee: Sylvain Lebresne
>             Fix For: 1.0
>
>         Attachments: 0001-Use-reference-counting-to-decide-when-a-sstable-can-.patch,
0001-Use-reference-counting-to-decide-when-a-sstable-can-v2.patch, 0002-Force-unmapping-files-before-deletion-v2.patch,
2521-v3.txt, 2521-v4.txt
>
>
> http://wiki.apache.org/cassandra/MemtableSSTable
> Let's move to using reference counting instead of relying on GC to be called in StorageService.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message