incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tupshin Harper <tups...@tupshin.com>
Subject Re: huge commitlog
Date Mon, 19 Nov 2012 03:14:52 GMT
What consistency level are you writing with? If you were writing with ANY,
try writing with a higher consistency level.

-Tupshin
On Nov 18, 2012 9:05 PM, "Chuan-Heng Hsiao" <hsiao.chuanheng@gmail.com>
wrote:

> Hi Aaron,
>
> Thank you very much for the replying.
>
> The 700 CFs were created in the beginning (before any insertion.)
>
> I did not do anything with commitlog_archiving.properties, so I guess
> I was not using commit log archiving.
>
> What I did was doing a lot of insertions (and some deletions)
> using another 4 machines with 32 processes in total.
> (There are 4 nodes in my setting, so there are 8 machines in total)
>
> I did see huge logs in /var/log/cassandra after such huge amount of
> insertions.
> Right now I  can't distinguish whether single insertion also cause huge
> logs.
>
> nodetool flush hanged (maybe because of 200G+ commitlog)
>
> Because these machines are not in production (guaranteed no more
> insertion/deletion)
> I ended up restarting cassandra one node each time, the commitlog
> shrinked back to
> 4G. I am doing repair on each node now.
>
> I'll try to re-import and keep logs when the commitlog increases insanely
> again.
>
> Sincerely,
> Hsiao
>
>
> On Mon, Nov 19, 2012 at 3:19 AM, aaron morton <aaron@thelastpickle.com>
> wrote:
> > I am wondering whether the huge commitlog size is the expected behavior
> or
> > not?
> >
> > Nope.
> >
> > Did you notice the large log size during or after the inserts ?
> > If after did the size settle ?
> > Are you using commit log archiving ? (in commitlog_archiving.properties)
> >
> > and around 700 mini column family (around 10M in data_file_directories)
> >
> > Can you describe how you created the 700 CF's ?
> >
> > and how can we reduce the size of commitlog?
> >
> > As a work around nodetool flush should checkpoint the log.
> >
> > Cheers
> >
> > -----------------
> > Aaron Morton
> > Freelance Cassandra Developer
> > New Zealand
> >
> > @aaronmorton
> > http://www.thelastpickle.com
> >
> > On 17/11/2012, at 2:30 PM, Chuan-Heng Hsiao <hsiao.chuanheng@gmail.com>
> > wrote:
> >
> > hi Cassandra Developers,
> >
> > I am experiencing huge commitlog size (200+G) after inserting huge
> > amount of data.
> > It is a 4-node cluster with RF= 3, and currently each has 200+G commit
> > log (so there are around 1T commit log in total)
> >
> > The setting of commitlog_total_space_in_mb is default.
> >
> > I am using 1.1.6.
> >
> > I did not do nodetool cleanup and nodetool flush yet, but
> > I did nodetool repair -pr for each column family.
> >
> > There is 1 huge column family (around 68G in data_file_directories),
> > and 18 mid-huge column family (around 1G in data_file_directories)
> > and around 700 mini column family (around 10M in data_file_directories)
> >
> > I am wondering whether the huge commitlog size is the expected behavior
> or
> > not?
> > and how can we reduce the size of commitlog?
> >
> > Sincerely,
> > Hsiao
> >
> >
>

Mime
View raw message