Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates
 209.85.215.172 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAAnh3_-=vs-74_JX-Aqq7Q0cPKh+P7tfYyyFaYSC_4Ake0Bmqw@mail.gmail.com>
References: 
 <CAAnh3__J-gywGyRpjhA+i30EYOcaS4_Fw8VQNkJSkHY4qoC6CA@mail.gmail.com>
 <CALdd-zgYZ5LnYadBBKN8UZvV9W+1_ytmNhp5TNUzYDZmxnDUQw@mail.gmail.com>
 <CAAnh3_-vJPJ8V8FQ-cniDdqwgvxKmGMaVLTXqRGHw0iU5NtvUg@mail.gmail.com>
 <CAAnh3__v5JehR_CAVe6TQJR_EYCTtK3vnTf92cvENfmJxE0fZg@mail.gmail.com>
 <CALdd-ziaORYL0BRikFt_M2saAQMW1jfcGCGnssG16Cn+6sc-iA@mail.gmail.com>
 <CAAnh3_-=vs-74_JX-Aqq7Q0cPKh+P7tfYyyFaYSC_4Ake0Bmqw@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Thu, 29 Sep 2011 17:10:41 -0500
Message-ID: 
 <CALdd-ziebe9ftZzbJrBA3_QtBciEjZx-xpBxELooLpVqvRfD1g@mail.gmail.com>
Subject: Re: StorageService.requestGC() ?
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

right.  path =3D=3D null means "Cassandra is out of disk space."

On Thu, Sep 29, 2011 at 5:06 PM, Yang <teddyyyy123@gmail.com> wrote:
> I am using mmap:
>
> =A0INFO [main] 2011-09-29 21:35:21,458 DatabaseDescriptor.java (line
> 177) DiskAccessMode is mmap, indexAccessMode is mmap
>
>
>
> =A0I think the cleanerMethod must be OK now,
>
> so the condition for GC
>
> =A0 =A0 =A0 =A0if (path =3D=3D null
> =A0 =A0 =A0 =A0 && (DatabaseDescriptor.getDiskAccessMode() =3D=3D
> Config.DiskAccessMode.mmap || DatabaseDescriptor.getIndexAccessMode()
> =3D=3D Config.DiskAccessMode.mmap)
> =A0 =A0 =A0 =A0 && !MmappedSegmentedFile.isCleanerAvailable())
> =A0 =A0 =A0 =A0{
>
>
> has to be triggered by path=3D=3Dnull,
>
> I have put in more logging to debug it
>
>
>
> On Thu, Sep 29, 2011 at 2:42 PM, Jonathan Ellis <jbellis@gmail.com> wrote=
:
>> you're not using mmap i/o mode?
>>
>> On Thu, Sep 29, 2011 at 2:21 PM, Yang <teddyyyy123@gmail.com> wrote:
>>> I see now :
>>>
>>> https://issues.apache.org/jira/browse/CASSANDRA-2521?focusedCommentId=
=3D13050396&page=3Dcom.atlassian.jira.plugin.system.issuetabpanels:comment-=
tabpanel#comment-13050396
>>>
>>>
>>>
>>> but my logs doesn't show the error from
>>>
>>> MMappedSegmentedFile:
>>>
>>> =A0 =A0public static void initCleaner()
>>> =A0 =A0{
>>> =A0 =A0 =A0 =A0try
>>> =A0 =A0 =A0 =A0{
>>> =A0 =A0 =A0 =A0 =A0 =A0cleanerMethod =3D
>>> Class.forName("sun.nio.ch.DirectBuffer").getMethod("cleaner");
>>> =A0 =A0 =A0 =A0}
>>> =A0 =A0 =A0 =A0catch (Exception e)
>>> =A0 =A0 =A0 =A0{
>>> =A0 =A0 =A0 =A0 =A0 =A0// Perhaps a non-sun-derived JVM - contributions=
 welcome
>>> =A0 =A0 =A0 =A0 =A0 =A0logger.info("Cannot initialize un-mmaper. =A0(Ar=
e you using
>>> a non-SUN JVM?) =A0Compacted data files will not be removed promptly.
>>> Consider using a SUN JVM or using standard disk access mode");
>>> =A0 =A0 =A0 =A0}
>>> =A0 =A0}
>>>
>>>
>>> and indeed I'm using openJDK. I'll switch to sun jdk, but am still
>>> curious why cleanerMethod is null in my case but I don't see the log
>>> message
>>>
>>>
>>> On Thu, Sep 29, 2011 at 12:03 PM, Yang <teddyyyy123@gmail.com> wrote:
>>>> I thought the need for GC was based on the code that sstables
>>>> deletions are tracked through weak/phantom references.
>>>> now I see that sstable is deleted in SStableReader.ReleaseReferences()
>>>> ---> SegmentedFile.cleanup(),
>>>> if this is true, is GC still necessary?
>>>>
>>>> Thanks
>>>> Yang
>>>>
>>>> On Thu, Sep 29, 2011 at 11:50 AM, Jonathan Ellis <jbellis@gmail.com> w=
rote:
>>>>> As the comment says,
>>>>> =A0 =A0 =A0 =A0 =A0 =A0// retry after GCing to force unmap of compact=
ed SSTables
>>>>> so they can be deleted
>>>>>
>>>>> See also http://wiki.apache.org/cassandra/MemtableSSTable
>>>>>
>>>>> Note that this has changed in 1.0:
>>>>> https://issues.apache.org/jira/browse/CASSANDRA-2521
>>>>>
>>>>> On Thu, Sep 29, 2011 at 1:14 PM, Yang <teddyyyy123@gmail.com> wrote:
>>>>>> Table.getDataFileLocation() calls StorageService.requestGC(), which
>>>>>> calls System.gc()
>>>>>>
>>>>>> I think I saw one GC in my log due to this code, and it caused a JVM
>>>>>> freeze for 14 seconds.
>>>>>> why do we need to call explicit GC ?
>>>>>>
>>>>>> thanks
>>>>>> Yang
>>>>>>
>>>>>>
>>>>>>
>>>>>> [Times: user=3D2.81 sys=3D0.00, real=3D0.46 secs]
>>>>>> 3472.943: [GC 3472.943: [ParNew: 21339139K->326984K(23592960K),
>>>>>> 0.4742050 secs] 30764101K->10032251K(49807360K), 0.4743710 secs]
>>>>>> [Times: user=3D2.91 sys=3D0.01, real=3D0.47 secs]
>>>>>> 3519.043: [GC 3519.043: [ParNew: 21298504K->386099K(23592960K),
>>>>>> 0.5201430 secs] 31003771K->10363088K(49807360K), 0.5203120 secs]
>>>>>> [Times: user=3D3.15 sys=3D0.03, real=3D0.52 secs]
>>>>>> 3560.277: [GC 3560.278: [ParNew: 21357619K->434850K(23592960K),
>>>>>> 0.4693880 secs] 31334608K->10676370K(49807360K), 0.4695450 secs]
>>>>>> [Times: user=3D2.92 sys=3D0.00, real=3D0.47 secs]
>>>>>> 3601.343: [Full GC (System) 3601.343: [CMS:
>>>>>> 10241519K->3496676K(26214400K), 14.6823750 secs]
>>>>>> 28763882K->3496676K(49807360K), [CMS Perm : 27683K->27623K(27776K)],
>>>>>> 14.6900120 secs] [Times: user=3D14.73 sys=3D0.00, real=3D14.69 secs]
>>>>>> 3665.223: [GC 3665.224: [ParNew: 20971520K->361802K(23592960K),
>>>>>> 0.2379040 secs] 24468196K->3858479K(49807360K), 0.2380660 secs]
>>>>>> [Times: user=3D1.57 sys=3D0.01, real=3D0.23 secs]
>>>>>> 3704.967: [GC 3704.968: [ParNew: 21333322K->359503K(23592960K),
>>>>>> 0.4433910 secs] 24829999K->4173314K(49807360K), 0.4435500 secs]
>>>>>> [Times: user=3D2.72 sys=3D0.02, real=3D0.44 secs]
>>>>>> 3801.962: [GC 3801.962: [ParNew: 21331023K->388609K(23592960K),
>>>>>> 0.5236180 secs] 25144834K->4479343K(49807360K), 0.5238240 secs]
>>>>>> [Times: user=3D3.42 sys=3D0.00, real=3D0.53 secs]
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jonathan Ellis
>>>>> Project Chair, Apache Cassandra
>>>>> co-founder of DataStax, the source for professional Cassandra support
>>>>> http://www.datastax.com
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>
>


--=20
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com