Return-Path: Delivered-To: apmail-incubator-cassandra-user-archive@minotaur.apache.org Received: (qmail 14838 invoked from network); 28 May 2009 20:51:36 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 28 May 2009 20:51:36 -0000 Received: (qmail 60844 invoked by uid 500); 28 May 2009 20:51:48 -0000 Delivered-To: apmail-incubator-cassandra-user-archive@incubator.apache.org Received: (qmail 60828 invoked by uid 500); 28 May 2009 20:51:48 -0000 Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cassandra-user@incubator.apache.org Delivered-To: mailing list cassandra-user@incubator.apache.org Received: (qmail 60809 invoked by uid 99); 28 May 2009 20:51:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 May 2009 20:51:46 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates 209.85.219.172 as permitted sender) Received: from [209.85.219.172] (HELO mail-ew0-f172.google.com) (209.85.219.172) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 May 2009 20:51:35 +0000 Received: by ewy20 with SMTP id 20so5879201ewy.12 for ; Thu, 28 May 2009 13:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=/oIIlHla/JqEle6zidL53X7+cqr+8uYiYG7hc6yi31A=; b=mi8EGxe3krJXZT+xqZLfFO4uxkAP6tpha/PYdnBttYcEELjdjTWIUa5sFX0OI6QYCw XglvLgzJS11ryausdofdLCKTlPZxKJvxfWZ+SyD1dZfcJFbi5RwZGYYnV99zQRr2oXtu imA/YglSfya9gV2mc56J0avBsd9NySyLfCrIM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=JKxjFcFl7bfHCe3+H9UFjDFXLNcnLG0676APYHzh+8IvrQniHRj16Yx9nxPlwIAsna bRqiQovcRIH0WlzOo4AMsmd7/BEtbiKrSbwVtuxtYoIVgUs/Ap42OF8HwAYk8YEGcS5F anTJ1YAkKEGqKLi8uffZ8cvgsHYM6zSJEaRNs= MIME-Version: 1.0 Received: by 10.216.19.210 with SMTP id n60mr674059wen.107.1243543874023; Thu, 28 May 2009 13:51:14 -0700 (PDT) In-Reply-To: <369521.28876.qm@web59605.mail.ac4.yahoo.com> References: <369521.28876.qm@web59605.mail.ac4.yahoo.com> Date: Thu, 28 May 2009 15:51:13 -0500 Message-ID: Subject: Re: Ingesting from Hadoop to Cassandra From: Jonathan Ellis To: cassandra-user@incubator.apache.org Content-Type: multipart/mixed; boundary=0016e6de04d5dfb22d046aff1e4e X-Virus-Checked: Checked by ClamAV on apache.org --0016e6de04d5dfb22d046aff1e4e Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit I can't reproduce with this, there is too much unspecified. (What is a Document? How do I get one?) Attached is a short program that successfully does 100k supercolumn inserts against a default configuration. Can you create a program like this for me to run? (Java is fine; Python is just more concise.) -Jonathan On Thu, May 28, 2009 at 11:03 AM, Alexandre Linares wrote: > Jonathan, sorry for the lengthy emails! Hope this one's more readable. > > So I'm fairly convinced it's not a Cassandra-side configuration problem; at > least not one that entails tweaking the object count threshold or the > memtable size. > > Given the client code at http://pastie.org/492753 : > --0016e6de04d5dfb22d046aff1e4e Content-Type: text/x-python; charset=US-ASCII; name="test.py" Content-Disposition: attachment; filename="test.py" Content-Transfer-Encoding: base64 X-Attachment-Id: f_fv9xwile1 ZnJvbSB0aHJpZnQudHJhbnNwb3J0IGltcG9ydCBUVHJhbnNwb3J0CmZyb20gdGhyaWZ0LnRyYW5z cG9ydCBpbXBvcnQgVFNvY2tldApmcm9tIHRocmlmdC50cmFuc3BvcnQgaW1wb3J0IFRIdHRwQ2xp ZW50CmZyb20gdGhyaWZ0LnByb3RvY29sIGltcG9ydCBUQmluYXJ5UHJvdG9jb2wKCmZyb20gY2Fz c2FuZHJhIGltcG9ydCBDYXNzYW5kcmEKZnJvbSBjYXNzYW5kcmEudHR5cGVzIGltcG9ydCBiYXRj aF9tdXRhdGlvbl90LCBiYXRjaF9tdXRhdGlvbl9zdXBlcl90LCBzdXBlckNvbHVtbl90LCBjb2x1 bW5fdCwgTm90Rm91bmRFeGNlcHRpb24sIEludmFsaWRSZXF1ZXN0RXhjZXB0aW9uCgoKc29ja2V0 ID0gVFNvY2tldC5UU29ja2V0KCdsb2NhbGhvc3QnLCA5MTYwKQp0cmFuc3BvcnQgPSBUVHJhbnNw b3J0LlRCdWZmZXJlZFRyYW5zcG9ydChzb2NrZXQpCnByb3RvY29sID0gVEJpbmFyeVByb3RvY29s LlRCaW5hcnlQcm90b2NvbCh0cmFuc3BvcnQpCmNsaWVudCA9IENhc3NhbmRyYS5DbGllbnQocHJv dG9jb2wpCnRyYW5zcG9ydC5vcGVuKCkKCgpmb3IgaSBpbiB4cmFuZ2UoMTAwMDAwKToKICAgIGRv Y19pZCA9IHN0cihpKQogICAgY29sdW1ucyA9IFtjb2x1bW5fdCgnaGVhZGVyJywgJ3gnKjEwMjQs IDApXQogICAgY2ZtYXAgPSB7J1N1cGVyMSc6IFtzdXBlckNvbHVtbl90KGRvY19pZCwgY29sdW1u cyldfQogICAgY2xpZW50LmJhdGNoX2luc2VydF9zdXBlckNvbHVtbihiYXRjaF9tdXRhdGlvbl90 KCdUYWJsZTEnLCBkb2NfaWQsIGNmbWFwKSwgVHJ1ZSkKICAgIHByaW50IGkK --0016e6de04d5dfb22d046aff1e4e--