Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of teddyyyy123@gmail.com
 designates 209.85.218.44 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAAnh3_-qSwmNGmGQ0U8LGTkTB5jUidcTHxLyJbDgaS6GRwB2EA@mail.gmail.com>
References: 
 <CAAnh3_-qSwmNGmGQ0U8LGTkTB5jUidcTHxLyJbDgaS6GRwB2EA@mail.gmail.com>
Date: Thu, 8 Sep 2011 23:02:24 -0700
Message-ID: 
 <CAAnh3_9ue0d-_eA3xJsNecjrVBAbLvjqMf8dvFjG2wy6ZQbUUA@mail.gmail.com>
Subject: Re: disk full and COMMIT-LOG-WRITER ?
From: Yang <teddyyyy123@gmail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1

ok, found past discussions:

http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/cassandra-server-disk-full-td6560725.html

On Thu, Sep 8, 2011 at 11:00 PM, Yang <teddyyyy123@gmail.com> wrote:
> I found the reason of my server freeze:
>
> COMMIT-LOG-WRITER thread is gone, dead, so the blocking queue in
> PeriodicCommitLogExecutorService is full, then all mutationStage jobs
> are stuck on the mutations flushing.
>
> the COMMIT-LOG-WRITER thread died because at one time the disk was full,
> I cleaned up the disk space (not deleting cassandra files, but other files),
> but then since the thread is gone, system is still stuck.
> so I had to restart the server.
>
>
> is it better to let the WRITER thread handle file system exceptions or
> let it die? granted letting disk go full is not
> a good practice, but letting the system proceed after disk is freed
> seems a more natural expectation.
>
> Thanks
> Yang
>