Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7F5044AED for ; Mon, 6 Jun 2011 19:42:39 +0000 (UTC) Received: (qmail 5190 invoked by uid 500); 6 Jun 2011 19:42:36 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 5137 invoked by uid 500); 6 Jun 2011 19:42:36 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 5129 invoked by uid 99); 6 Jun 2011 19:42:36 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Jun 2011 19:42:36 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,TRACKER_ID X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a53.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Jun 2011 19:42:29 +0000 Received: from homiemail-a53.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a53.g.dreamhost.com (Postfix) with ESMTP id 8812913806D for ; Mon, 6 Jun 2011 12:42:06 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; q=dns; s=thelastpickle.com; b=cJoFbUMsKc VhvQlRrcSGVx9f/ru1qChaKXmvdySoibAQtpMcpB8E4W9efLXpzRJ38380oPLRY9 6mrKLbu1NVm6PP2k6UqdDBiuU1m2JkKmwzSeXY197F/BFBghv/A3YTLASsaAH5Ud H1YAreDtXCmCJ1QASfbr/9BehTt0MP6NM= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :mime-version:content-type:subject:date:in-reply-to:to :references:message-id; s=thelastpickle.com; bh=zpLRPF7V27CI8sjk /1GUJeZWahU=; b=f1lnjwZJyYZrRo3KhfcfUVpkBpuPyEOZU9WPwIlEGCYf0u29 VZj4pbjEltnGGZe8M+SB47BwIHPY8yqpjDb5Qp7tpyjGid5p4+zeeicNkVUoA7gu TWoxJfHswKmIyT2IUcTZ5q6VRc7LFbTvJJrKJMenDLvQ9nQOUWphxq1EDeA= Received: from [10.0.1.151] (121-73-157-230.cable.telstraclear.net [121.73.157.230]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a53.g.dreamhost.com (Postfix) with ESMTPSA id 95AF7138069 for ; Mon, 6 Jun 2011 12:42:05 -0700 (PDT) From: aaron morton Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/alternative; boundary=Apple-Mail-15-517126177 Subject: Re: problems with many columns on a row Date: Tue, 7 Jun 2011 07:42:04 +1200 In-Reply-To: To: user@cassandra.apache.org References: Message-Id: <91642C4C-C13D-43A0-81C9-E1EA366CA16F@thelastpickle.com> X-Mailer: Apple Mail (2.1084) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail-15-517126177 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Can you upgrade to the official 0.8 release and try again with logging = set to DEBUG ?=20 Cheers ----------------- Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 6 Jun 2011, at 23:41, Mario Micklisch wrote: > :-) >=20 > There are several data Files: >=20 > # ls -al *-Data.db >=20 >=20 > -rw-r--r-- 1 cassandra cassandra 53785327 2011-06-05 14:44 = CFTest-g-21-Data.db > -rw-r--r-- 1 cassandra cassandra 56474656 2011-06-05 18:04 = CFTest-g-38-Data.db > -rw-r--r-- 1 cassandra cassandra 21705904 2011-06-05 20:02 = CFTest-g-45-Data.db > -rw------- 1 cassandra cassandra 3720201 2011-06-05 20:13 = CFTest-g-46-Data.db > -rw-r--r-- 1 cassandra cassandra 135684874 2011-06-05 20:14 = CFTest-g-47-Data.db > -rw-r--r-- 1 cassandra cassandra 4347351 2011-06-05 20:35 = CFTest-g-48-Data.db > -rw-r--r-- 1 cassandra cassandra 4341698 2011-06-05 20:56 = CFTest-g-49-Data.db > -rw-r--r-- 1 cassandra cassandra 4350949 2011-06-05 21:17 = CFTest-g-50-Data.db > -rw-r--r-- 1 cassandra cassandra 4346066 2011-06-05 21:38 = CFTest-g-51-Data.db > -rw-r--r-- 1 cassandra cassandra 17384858 2011-06-05 21:39 = CFTest-g-52-Data.db > -rw-r--r-- 1 cassandra cassandra 4342983 2011-06-05 21:59 = CFTest-g-53-Data.db > -rw-r--r-- 1 cassandra cassandra 4339771 2011-06-05 22:21 = CFTest-g-54-Data.db > -rw-r--r-- 1 cassandra cassandra 4349165 2011-06-05 22:42 = CFTest-g-55-Data.db > -rw-r--r-- 1 cassandra cassandra 30415571 2011-06-05 22:42 = CFTest-g-56-Data.db >=20 >=20 > Cheers, > Mario >=20 >=20 >=20 >=20 > 2011/6/6 aaron morton > Ops, I misread "150 GB" in one of your earlier emails as "150 MB" so = forget what I said before. You have loads of free space :) >=20 > How many files do you have in your data directory ? If it's 1 then = that log message was a small bug, that has been fixed. >=20 > Cheers > =20 > ----------------- > Aaron Morton > Freelance Cassandra Developer > @aaronmorton > http://www.thelastpickle.com >=20 > On 5 Jun 2011, at 23:51, Mario Micklisch wrote: >=20 >> I found a patch for the php extension here: >>=20 >> https://issues.apache.org/jira/browse/THRIFT-1067 >>=20 >> =85 this seemed to fix the issue. Thank you Jonathan and Aaron for = taking time to provide me with some help! >>=20 >> Regarding the compaction I would still love to hear your feedback on = how to configure Cassandra accordingly. >>=20 >> Here are my facts again: >>=20 >> Data size on disk: ~1GB >> Data Volume: ~150GB free storage >> OS Volume: ~3GB free storage >>=20 >> Tried: >> nodetool --host localhost compact >>=20 >> The logfiles said: >> ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,346 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting >> ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,348 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting >> ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,349 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting >> INFO [CompactionExecutor:2] 2011-06-05 11:44:39,366 = CompactionManager.java (line 539) Compacting Major: = [SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-81-Data.d= b'), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-80-Data.db= '), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-79-Data.db= ')] >> INFO [CompactionExecutor:2] 2011-06-05 11:44:39,502 = CompactionIterator.java (line 186) Major@23231510(system, LocationInfo, = 386/689) now compacting at 16777 bytes/ms. >> INFO [CompactionExecutor:2] 2011-06-05 11:44:39,713 = CompactionManager.java (line 603) Compacted to = /mnt/cassandra/data/system/LocationInfo-tmp-g-82-Data.db. 689 to 446 = (~64% of original) bytes for 3 keys. Time: 346ms. >> ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,726 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting >> ERROR [CompactionExecutor:2] 2011-06-05 11:44:39,726 = CompactionManager.java (line 510) insufficient space to compact even the = two smallest files, aborting >>=20 >> # du -hs /mnt/cassandra/ >> 886M /mnt/cassandra/ >>=20 >> # df -H >> Filesystem Size Used Avail Use% Mounted on >> /dev/sda1 8.5G 4.9G 3.6G 49% / >> /dev/sda2 158G 1.2G 149G 1% /mnt >>=20 >>=20 >> # nodetool --host localhost info >> 130915841875673808436922706546793999597 >> Gossip active : true >> Load : 1.48 MB >> Generation No : 1307273982 >> Uptime (seconds) : 383 >> Heap Memory (MB) : 89.59 / 822.00 >>=20 >>=20 >> Thank you, >> Mario >>=20 >>=20 >>=20 >> 2011/6/5 Mario Micklisch >> I tracked down the timestamp submission and everything was fine = within the PHP Libraries. >>=20 >> The thrift php extension however seems to have an overflow, because = it was now setting now timestamps with also negative values ( = -1242277493 ). I disabled the php extension and as a result I now got = correct microsecond timestamps: 1307270937122897 >>=20 >> I was using the latest version from = http://www.apache.org/dist/thrift/0.6.1/ to build the extension without = using any special parameters (just ./configure --enable-gen-php=3Dyes = and make install). >>=20 >> Downloaded and re-compiled it, without any change. I can also see no = compile parameters to set which might help. >>=20 >>=20 >>=20 >> Any advise where to go from here? >>=20 >>=20 >>=20 >> Thanks so far! >>=20 >> Mario >>=20 >>=20 >=20 >=20 --Apple-Mail-15-517126177 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252 Can = you upgrade to the official 0.8 release and try again with logging set = to DEBUG ? 

Cheers

http://www.thelastpickle.com

On 6 Jun 2011, at 23:41, Mario Micklisch wrote:

:-)

There are several data Files:

# = ls -al *-Data.db


-rw-r--r-- 1 cassandra = cassandra  53785327 2011-06-05 14:44 = CFTest-g-21-Data.db
-rw-r--r-- 1 cassandra cassandra  56474656 = 2011-06-05 18:04 CFTest-g-38-Data.db
-rw-r--r-- 1 cassandra cassandra  21705904 2011-06-05 20:02 = CFTest-g-45-Data.db
-rw------- 1 cassandra cassandra   3720201 = 2011-06-05 20:13 CFTest-g-46-Data.db
-rw-r--r-- 1 cassandra cassandra = 135684874 2011-06-05 20:14 CFTest-g-47-Data.db
-rw-r--r-- 1 cassandra cassandra   4347351 2011-06-05 20:35 = CFTest-g-48-Data.db
-rw-r--r-- 1 cassandra cassandra   4341698 = 2011-06-05 20:56 CFTest-g-49-Data.db
-rw-r--r-- 1 cassandra cassandra =   4350949 2011-06-05 21:17 CFTest-g-50-Data.db
-rw-r--r-- 1 cassandra cassandra   4346066 2011-06-05 21:38 = CFTest-g-51-Data.db
-rw-r--r-- 1 cassandra cassandra  17384858 = 2011-06-05 21:39 CFTest-g-52-Data.db
-rw-r--r-- 1 cassandra cassandra =   4342983 2011-06-05 21:59 CFTest-g-53-Data.db
-rw-r--r-- 1 cassandra cassandra   4339771 2011-06-05 22:21 = CFTest-g-54-Data.db
-rw-r--r-- 1 cassandra cassandra   4349165 = 2011-06-05 22:42 CFTest-g-55-Data.db
-rw-r--r-- 1 cassandra cassandra =  30415571 2011-06-05 22:42 CFTest-g-56-Data.db


Cheers,
 Mario




2011/6/6 aaron morton <aaron@thelastpickle.com>
Ops, I misread "150 GB" in one of = your earlier emails as "150 MB" so forget what I said before. You have = loads of free space :)

How many files do you have in = your data directory ? If it's 1 then that log message was a small bug, = that has been fixed.

Cheers
 
-----------------
Aaron Morton
Freelance = Cassandra Developer
@aaronmorton

On 5 Jun 2011, at 23:51, Mario = Micklisch wrote:

I found a patch for the php extension = here:

https://issues.apache.org/jira/browse/THRIFT-1067

=85 this seemed to fix the issue. Thank you Jonathan and Aaron for = taking time to provide me with some help!

Regarding the compaction I would still love to hear your feedback = on how to configure Cassandra accordingly.

Here are = my facts again:

Data size on disk: = ~1GB
Data Volume: ~150GB free storage
OS Volume: ~3GB free = storage

Tried:
nodetool --host = localhost compact

The logfiles = said:
ERROR [CompactionExecutor:2] = 2011-06-05 11:44:39,346 CompactionManager.java (line 510) insufficient = space to compact even the two smallest files, = aborting
ERROR [CompactionExecutor:2] = 2011-06-05 11:44:39,348 CompactionManager.java (line 510) insufficient = space to compact even the two smallest files, = aborting
ERROR [CompactionExecutor:2] = 2011-06-05 11:44:39,349 CompactionManager.java (line 510) insufficient = space to compact even the two smallest files, = aborting
 INFO [CompactionExecutor:2] = 2011-06-05 11:44:39,366 CompactionManager.java (line 539) Compacting = Major: = [SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-81-Data.d= b'), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-80-Data.db= '), = SSTableReader(path=3D'/mnt/cassandra/data/system/LocationInfo-g-79-Data.db= ')]
 INFO [CompactionExecutor:2] = 2011-06-05 11:44:39,502 CompactionIterator.java (line 186) = Major@23231510(system, LocationInfo, 386/689) now compacting at 16777 = bytes/ms.
 INFO [CompactionExecutor:2] = 2011-06-05 11:44:39,713 CompactionManager.java (line 603) Compacted to = /mnt/cassandra/data/system/LocationInfo-tmp-g-82-Data.db.  689 to = 446 (~64% of original) bytes for 3 keys.  Time: = 346ms.
ERROR [CompactionExecutor:2] = 2011-06-05 11:44:39,726 CompactionManager.java (line 510) insufficient = space to compact even the two smallest files, = aborting
ERROR [CompactionExecutor:2] = 2011-06-05 11:44:39,726 CompactionManager.java (line 510) insufficient = space to compact even the two smallest files, = aborting

# du = -hs /mnt/cassandra/
886M = /mnt/cassandra/

# df = -H
Filesystem       =       Size   Used  Avail Use% Mounted = on
/dev/sda1         =      8.5G   4.9G   3.6G  49% = /
/dev/sda2             =  158G   1.2G   149G   1% /mnt


# nodetool --host = localhost info
130915841875673808436922706546793999597=
Gossip active    : = true
Load         =     : 1.48 MB
Generation No =    : 1307273982
Uptime (seconds) : = 383
Heap Memory (MB) : 89.59 / = 822.00


Thank you,
 Mario



2011/6/5 Mario Micklisch <mario.micklisch@hpm-kommunikation.de>
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex">

I tracked down the timestamp submission and = everything was fine within the PHP Libraries.

The thrift php = extension however seems to have an overflow, because it was now setting = now timestamps with also negative values ( -1242277493 ). I = disabled the php extension and as a result I now got correct microsecond = timestamps: 1307270937122897

I was using the latest version = from http://www.apache.org/dist/thrift/0.6.1/ to build = the extension without using any special parameters = (just ./configure = --enable-gen-php=3Dyes and make install).

Downloaded and = re-compiled it, without any change. I can also  see no compile = parameters to set which might help.


Any advise where to = go from here?


Thanks so = far!

 Mario





= --Apple-Mail-15-517126177--