From user-return-17367-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Sun Jun 5 23:38:29 2011 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DFA78698E for ; Sun, 5 Jun 2011 23:38:29 +0000 (UTC) Received: (qmail 84187 invoked by uid 500); 5 Jun 2011 23:38:27 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 84161 invoked by uid 500); 5 Jun 2011 23:38:27 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 84152 invoked by uid 99); 5 Jun 2011 23:38:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Jun 2011 23:38:27 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,TRACKER_ID X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a52.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 05 Jun 2011 23:38:21 +0000 Received: from homiemail-a52.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a52.g.dreamhost.com (Postfix) with ESMTP id C0BD96B81EA for ; Sun, 5 Jun 2011 16:37:55 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=e5pFIVgIhN V7xVnxYpHuAx93zM6krU1wpgfjANkZADH33e5Uwlbr/X6UHldiVEKni9U3FZqBVz 81MKoUAD55CLqA743D072UbXWlGbQWkd0FjLAqwxZLCVm+MEm9CNWkCU+0CcCuYj 2pldyPy7Ox4oJ76mosDUJPvrbn1b1dW4E= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=+pB8U+ZfMy8Xpbma NSdfRfCiXAw=; b=ZwKrRrK1K7vlrPqVSL4yDOJEyNqsOvwwEN5R5U5kQS5E/4G1 QYF7Jeso3iuBql0qZMDZxT2CbrgAmPdTh1sXeZrLoQ4s0Dwpd8RT32k1czvRO6Fu bPwxF8rE50OtIWCugKEJuzNkt5agBxEeEPEUrHPkLMzDjnAgjSdLNBH49xk= Received: from [172.20.10.3] (164.253.227.49.dyn.cust.vf.net.nz [49.227.253.164]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a52.g.dreamhost.com (Postfix) with ESMTPSA id 4FC3D6B8058 for ; Sun, 5 Jun 2011 16:37:51 -0700 (PDT) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/alternative; boundary=Apple-Mail-5-444867359 Subject: Re: problems with many columns on a row Date: Mon, 6 Jun 2011 11:37:45 +1200 In-Reply-To: To: user@cassandra.apache.org References: Message-Id: X-Mailer: Apple Mail (2.1084) --Apple-Mail-5-444867359 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Ops, I misread "150 GB" in one of your earlier emails as "150 MB" so = forget what I said before. You have loads of free space :) How many files do you have in your data directory ? If it's 1 then that = log message was a small bug, that has been fixed. Cheers =20 ----------------- Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 5 Jun 2011, at 23:51, Mario Micklisch wrote: > I found a patch for the php extension here: >=20 > https://issues.apache.org/jira/browse/THRIFT-1067 >=20 > =85 this seemed to fix the issue. Thank you Jonathan and Aaron for = taking time to provide me with some help! >=20 > Regarding the compaction I would still love to hear your feedback on = how to configure Cassandra accordingly. >=20 > Here are my facts again: >=20 > Data size on disk: ~1GB > Data Volume: ~150GB free storage > OS Volume: ~3GB free storage >=20 > Tried: > nodetool --host localhost compact >=20 > The logfiles said: > ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,346 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting > ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,348 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting > ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,349 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting > INFO [CompactionExecutor:2] 2011-06-05 11:44:39,366 = CompactionManager.java (line 539) Compacting Major: = [SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-81-Data.d= b'), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-80-Data.db= '), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-79-Data.db= ')] > INFO [CompactionExecutor:2] 2011-06-05 11:44:39,502 = CompactionIterator.java (line 186) Major@23231510(system, LocationInfo, = 386/689) now compacting at 16777 bytes/ms. > INFO [CompactionExecutor:2] 2011-06-05 11:44:39,713 = CompactionManager.java (line 603) Compacted to = /mnt/cassandra/data/system/LocationInfo-tmp-g-82-Data.db. 689 to 446 = (~64% of original) bytes for 3 keys. Time: 346ms. > ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,726 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting > ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,726 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting >=20 > # du -hs /mnt/cassandra/ > 886M /mnt/cassandra/ >=20 > # df -H > Filesystem Size Used Avail Use% Mounted on > /dev/sda1 8.5G 4.9G 3.6G 49% / > /dev/sda2 158G 1.2G 149G 1% /mnt >=20 >=20 > # nodetool --host localhost info > 130915841875673808436922706546793999597 > Gossip active : true > Load : 1.48 MB > Generation No : 1307273982 > Uptime (seconds) : 383 > Heap Memory (MB) : 89.59 / 822.00 >=20 >=20 > Thank you, > Mario >=20 >=20 >=20 > 2011/6/5 Mario Micklisch > I tracked down the timestamp submission and everything was fine within = the PHP Libraries. >=20 > The thrift php extension however seems to have an overflow, because it = was now setting now timestamps with also negative values ( -1242277493 = ). I disabled the php extension and as a result I now got correct = microsecond timestamps: 1307270937122897 >=20 > I was using the latest version from = http://www.apache.org/dist/thrift/0.6.1/ to build the extension without = using any special parameters (just ./configure --enable-gen-php=3Dyes = and make install). >=20 > Downloaded and re-compiled it, without any change. I can also see no = compile parameters to set which might help. >=20 >=20 >=20 > Any advise where to go from here? >=20 >=20 >=20 > Thanks so far! >=20 > Mario >=20 >=20 --Apple-Mail-5-444867359 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Ops, = I misread "150 GB" in one of your earlier emails as "150 MB" so forget = what I said before. You have loads of free space = :)

How many files do you have in your data directory = ? If it's 1 then that log message was a small bug, that has been = fixed.

Cheers
 
http://www.thelastpickle.com

On 5 Jun 2011, at 23:51, Mario Micklisch wrote:

I found a patch for the php extension = here:

https://issues.= apache.org/jira/browse/THRIFT-1067

=85 this seemed to fix the = issue. Thank you Jonathan and Aaron for taking time to provide me with = some help!

Regarding the compaction I would still love to hear your feedback = on how to configure Cassandra accordingly.

Here are = my facts again:

Data size on disk: = ~1GB
Data Volume: ~150GB free storage
OS Volume: ~3GB free = storage

Tried:
nodetool --host = localhost compact

The logfiles = said:
ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,346 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting
ERROR = [CompactionExecutor:2] 2011-06-05 11:44:39,348 CompactionManager.java = (line 510) insufficient space to compact even the two smallest files, = aborting
ERROR = [CompactionExecutor:2] 2011-06-05 11:44:39,349 CompactionManager.java = (line 510) insufficient space to compact even the two smallest files, = aborting
 INFO [CompactionExecutor:2] 2011-06-05 11:44:39,366 = CompactionManager.java (line 539) Compacting Major: = [SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-81-Data.d= b'), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-80-Data.db= '), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-79-Data.db= ')]
 INFO [CompactionExecutor:2] 2011-06-05 11:44:39,502 = CompactionIterator.java (line 186) Major@23231510(system, LocationInfo, = 386/689) now compacting at 16777 bytes/ms.
 INFO [CompactionExecutor:2] 2011-06-05 11:44:39,713 = CompactionManager.java (line 603) Compacted to = /mnt/cassandra/data/system/LocationInfo-tmp-g-82-Data.db.  689 to = 446 (~64% of original) bytes for 3 keys.  Time: = 346ms.
ERROR = [CompactionExecutor:2] 2011-06-05 11:44:39,726 CompactionManager.java = (line 510) insufficient space to compact even the two smallest files, = aborting
ERROR = [CompactionExecutor:2] 2011-06-05 11:44:39,726 CompactionManager.java = (line 510) insufficient space to compact even the two smallest files, = aborting

# du -hs = /mnt/cassandra/
886M = /mnt/cassandra/

# df = -H
Filesystem             Size =   Used  Avail Use% Mounted on
/dev/sda1             =  8.5G   4.9G   3.6G  49% /
/dev/sda2     =          158G   1.2G   149G   1% = /mnt


# = nodetool --host localhost info
130915841875673808436922706546793999597
Gossip = active    : true
Load =             : 1.48 = MB
Generation No    : 1307273982
Uptime = (seconds) : 383
Heap Memory (MB) : 89.59 / = 822.00

Thank you,
 Mario



2011/6/5 Mario Micklisch <mario.micklisch@hpm-k= ommunikation.de>

I tracked down the timestamp submission and = everything was fine within the PHP Libraries.

The thrift php = extension however seems to have an overflow, because it was now setting = now timestamps with also negative values ( -1242277493 ). I = disabled the php extension and as a result I now got correct microsecond = timestamps: 1307270937122897

I was using the latest version = from http://www.apache.org/dist/thrift/0.6.1/ to build = the extension without using any special parameters = (just ./configure = --enable-gen-php=3Dyes and make install).

Downloaded and = re-compiled it, without any change. I can also  see no compile = parameters to set which might help.


Any advise where to = go from here?


Thanks so = far!

 Mario



= --Apple-Mail-5-444867359--