Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 66A4D7AD9 for ; Tue, 20 Sep 2011 21:26:48 +0000 (UTC) Received: (qmail 49738 invoked by uid 500); 20 Sep 2011 21:26:46 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 49707 invoked by uid 500); 20 Sep 2011 21:26:46 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 49696 invoked by uid 99); 20 Sep 2011 21:26:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Sep 2011 21:26:46 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of searchbrowser@gmail.com designates 209.85.161.44 as permitted sender) Received: from [209.85.161.44] (HELO mail-fx0-f44.google.com) (209.85.161.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Sep 2011 21:26:39 +0000 Received: by fxd18 with SMTP id 18so1034829fxd.31 for ; Tue, 20 Sep 2011 14:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=fZmGvHzg0CaJejcqohH7yrMYh+KLtlUHmhmB0rFu1KI=; b=FY2VOtTwiTGDD6Zcytn/HczpgyQj0JsU965w+cYTKNguwn+Wx4jbOYiz3uQjw2WKmV s1tZU34Sc6z4Wy8Wn3l6+Nu9sgHkPTgtF1p9HJc3tD2vbGlyI+bL1H4BmY+fxoi/9E5X QgEwan/xPgX5UU0il17MbQfV78Abyo+AHZH1I= MIME-Version: 1.0 Received: by 10.223.36.25 with SMTP id r25mr1986610fad.125.1316553978239; Tue, 20 Sep 2011 14:26:18 -0700 (PDT) Sender: searchbrowser@gmail.com Received: by 10.223.87.12 with HTTP; Tue, 20 Sep 2011 14:26:18 -0700 (PDT) Date: Tue, 20 Sep 2011 14:26:18 -0700 X-Google-Sender-Auth: AzG3w9qj5ktcRjVNmDmH14LHjF0 Message-ID: Subject: TransportException when storing large values From: Pete Warden To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001517476a20335a6404ad661c95 --001517476a20335a6404ad661c95 Content-Type: text/plain; charset=ISO-8859-1 I'm running into consistent problems when storing values larger than 15MB into Cassandra, and I was hoping for some advice on tracking down what's going wrong. From the FAQ it seems like what I'm trying to do is possible, so I assume I'm messing something up with my configuration. I have a minimal set of code to reproduce the issue below, which I've run on the DataStax 0.8.1 AMI I'm using in production (ami-9996c4dc) # To set up the test data structure on Cassandra: cassandra-cli connect localhost/9160; create keyspace TestKeyspace with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = [{replication_factor:3}]; use TestKeyspace; create column family TestFamily with comparator = UTF8Type and column_metadata = [ {column_name: test_column, validation_class: UTF8Type}, ]; # From bash on the same machine, with Ruby and the Cassandra gem installed: irb require 'rubygems' require 'cassandra/0.8' client = Cassandra.new('TestKeyspace', 'localhost:9160', :retries => 5, :connect_timeout => 5, :timeout => 10) # With data this size, the call works column_value = 'a' * (14*1024*1024) row_value = { 'column_name' => column_value } client.insert(:TestFamily, 'SomeKey', row_value) # With data this size, the call fails with the exception below column_value = 'a' * (15*1024*1024) row_value = { 'column_name' => column_value } client.insert(:TestFamily, 'SomeKey', row_value) # Results: This first call with a 14MB chunk of data succeeds, but the second one fails with this exception: CassandraThrift::Cassandra::Client::TransportException: CassandraThrift::Cassandra::Client::TransportException from /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/socket.rb:53:in `open' from /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/framed_transport.rb:37:in `open' from /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/connection/socket.rb:11:in `connect!' from /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:105:in `connect!' from /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:144:in `handled_proxy' from /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:60:in `batch_mutate' from /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/protocol.rb:7:in `_mutate' from /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/cassandra.rb:459:in `insert' from (irb):6 from :0 Any suggestions on how to dig deeper? I'll be reaching out to the Cassandra gem folks, etc too of course. cheers, Pete --001517476a20335a6404ad661c95 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I'm running into consistent problems when storing values larger than 15= MB into Cassandra, and I was hoping for some advice on tracking down what&#= 39;s going wrong. From the FAQ it seems like what I'm trying to do is p= ossible, so I assume I'm messing something up with my configuration. I = have a minimal set of code to reproduce the issue below, which I've run= on the DataStax 0.8.1 AMI I'm using in production (ami-9996c4dc)

# To set up the test data structure on Cassandra:
=
cassandra-cli
connect localhost/9160;
create keyspace TestKeyspace with=A0
=A0 placement_strategy = =3D 'org.apache.cassandra.locator.SimpleStrategy' and=A0
=A0 strategy_options =3D [{replication_factor:3}];
use TestK= eyspace;
create column family TestFamily with
=A0 compa= rator =3D UTF8Type and
=A0 column_metadata =3D
=A0 [
=A0 =A0 {column_name: test_column, validation_class: UTF8Type},
=A0 ];

# From bash on the same machine, with = Ruby and the Cassandra gem installed:

irb
require 'rubygems'
require 'cassandra/0.8'

client =3D Cassandra.new('TestKeyspace', 'l= ocalhost:9160', :retries =3D> 5, :connect_timeout =3D> 5, :timeou= t =3D> 10)

# With data this size, the call work= s
column_value =3D 'a' * (14*1024*1024)
row_value =3D = { 'column_name' =3D> column_value }
client.insert(:Tes= tFamily, 'SomeKey', row_value)


<= div> # With data this size, the call fails with the exception below
co= lumn_value =3D 'a' * (15*1024*1024)
row_value =3D { '= column_name' =3D> column_value }
client.insert(:TestFamily= , 'SomeKey', row_value)

# Results:

This first call wit= h a 14MB chunk of data succeeds, but the second one fails with this excepti= on:

CassandraThrift::Cassandra::Client::TransportE= xception: CassandraThrift::Cassandra::Client::TransportException
from = /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/socket.rb:53:= in `open'
from /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/tran= sport/framed_transport.rb:37:in `open'
from = /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/connectio= n/socket.rb:11:in `connect!'
from /usr/lib/ruby/gems/1.8/gems/thrift_cl= ient-0.7.1/lib/thrift_client/abstract_thrift_client.rb:105:in `connect!'= ;
from = /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_= thrift_client.rb:144:in `handled_proxy'
from /usr/lib/ruby/gems/1.8/gem= s/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:60:in `ba= tch_mutate'
from = /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/protocol.rb:7:in= `_mutate'
from /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassan= dra/cassandra.rb:459:in `insert'
from = (irb):6
= from :0

Any suggestions on how to di= g deeper? I'll be reaching out to the Cassandra gem folks, etc too of c= ourse.

cheers,
=A0 =A0 =A0 =A0 =A0 =A0Pete
--001517476a20335a6404ad661c95--