From user-return-25288-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Tue Apr 3 11:08:51 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 34AF992E2 for ; Tue, 3 Apr 2012 11:08:51 +0000 (UTC) Received: (qmail 6377 invoked by uid 500); 3 Apr 2012 11:08:48 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 6351 invoked by uid 500); 3 Apr 2012 11:08:48 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 6339 invoked by uid 99); 3 Apr 2012 11:08:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2012 11:08:48 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.214.44] (HELO mail-bk0-f44.google.com) (209.85.214.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2012 11:08:39 +0000 Received: by bkuw5 with SMTP id w5so3526636bku.31 for ; Tue, 03 Apr 2012 04:08:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:content-type:subject:date:message-id:to:mime-version:x-mailer :x-gm-message-state; bh=6SUkbeNaHcbBhU7zDevFv8ntlHTPEf5LEM7oKAnheBc=; b=e5BVooNZZctXpn6b/AkcJ04Gfn2vPdMI9ixGwryHEDZQAR93p1Ovr6MF27pqsszrhK vKMOyWlUTlVWnyK7dGrpZZetRpTBLb0PaNCpGJCfrpnGeR5+NkPTpSMzmrkpHinick3l iujdmlwtpOnljsgGKgI+uKzXg9UvSNeD6kbsLveIlhcamm0m5IdVr4mHliF5TWi5j/WC S9neVHUk2YtGHKDABjSAF/JpI7p7lJMR7ve3sGYBk4WXMUk5DLBxcCgDhCeuykc/caHk VD5WBoQNNW/YgW4bK5PRzxS0jsvH9McOf/y8SAWkHNg5oP2M207xIRq24GSXMhvh3VVD 8oaw== Received: by 10.204.154.66 with SMTP id n2mr5440160bkw.77.1333451298347; Tue, 03 Apr 2012 04:08:18 -0700 (PDT) Received: from [10.10.1.63] (firewall.aspiro.com. [80.65.54.242]) by mx.google.com with ESMTPS id r14sm44979882bkv.11.2012.04.03.04.08.17 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 03 Apr 2012 04:08:17 -0700 (PDT) From: Jeff Williams Content-Type: multipart/mixed; boundary="Apple-Mail=_CBBD49BF-130F-4340-B9ED-D8733AE1BB82" Subject: Write performance compared to Postgresql Date: Tue, 3 Apr 2012 13:08:16 +0200 Message-Id: <7B1606F9-19D2-46EB-831D-DF373F3C8E8B@wherethebitsroam.com> To: user@cassandra.apache.org Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) X-Gm-Message-State: ALoCoQlr+O84HhLsAcu00uY4uczDm6pp+V880KNfrmhfVgjRT9tQcG5D7+pM7Kpm5WCpla30NW/m --Apple-Mail=_CBBD49BF-130F-4340-B9ED-D8733AE1BB82 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, I am looking at cassandra for a logging application. We currently log to = a Postgresql database. I set up 2 cassandra servers for testing. I did a benchmark where I had = 100 hashes representing logs entries, read from a json file. I then = looped over these to do 10,000 log inserts. I repeated the same writing = to a postgresql instance on one of the cassandra servers. The script is = attached. The cassandra writes appear to perform a lot worse. Is this = expected? jeff@transcoder01:~$ ruby cassandra-bm.rb=20 cassandra 3.170000 0.480000 3.650000 ( 12.032212) jeff@transcoder01:~$ ruby cassandra-bm.rb=20 postgres 2.140000 0.330000 2.470000 ( 7.002601) Regards, Jeff --Apple-Mail=_CBBD49BF-130F-4340-B9ED-D8733AE1BB82 Content-Disposition: attachment; filename=cassandra-bm.rb Content-Type: text/x-ruby-script; name="cassandra-bm.rb" Content-Transfer-Encoding: 7bit require 'rubygems' require 'cassandra-cql' require 'simple_uuid' require 'benchmark' require 'json' require 'active_record' type = 'postgres' #type = 'cassandra' puts type ActiveRecord::Base.establish_connection( #:adapter => "jdbcpostgresql", :adapter => "postgresql", :host => "meta01", :username => "postgres", :database => "test") db = nil if type == 'postgres' db = ActiveRecord::Base.connection else db = CassandraCQL::Database.new('meta01:9160', {:keyspace => 'PlayLog'}) end def cql_insert(table, key, key_value) cql = "INSERT INTO #{table} (KEY, " cql << key_value.keys.join(', ') cql << ") VALUES ('#{key}', " cql << (key_value.values.map {|x| "'#{x}'" }).join(', ') cql << ")" cql end def quote_value(x, type=nil) if x.nil? return 'NULL' else return "'#{x}'" end end def sql_insert(table, key_value) key_value.delete('time') cql = "INSERT INTO #{table} (" cql << key_value.keys.join(', ') cql << ") VALUES (" cql << (key_value.values.map {|x| quote_value(x) }).join(', ') cql << ")" cql end # load 100 hashes of log details rows = [] File.open('data.json') do |f| rows = JSON.load(f) end bm = Benchmark.measure do (1..10000).each do |i| row = rows[i%100] if type == 'postgres' fred = sql_insert('playlog', row) else fred = cql_insert('playlog', SimpleUUID::UUID.new.to_guid, row) end db.execute(fred) end end puts bm --Apple-Mail=_CBBD49BF-130F-4340-B9ED-D8733AE1BB82--