incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tyler Hobbs <ty...@datastax.com>
Subject Re: TransportException when storing large values
Date Wed, 21 Sep 2011 01:02:23 GMT
>From cassandra.yaml:

# Frame size for thrift (maximum field length).
# 0 disables TFramedTransport in favor of TSocket. This option
# is deprecated; we strongly recommend using Framed mode.
thrift_framed_transport_size_in_mb: 15

So you can either increase that limit, or split your write into multiple
operations.

On Tue, Sep 20, 2011 at 4:26 PM, Pete Warden <pete@petewarden.com> wrote:

> I'm running into consistent problems when storing values larger than 15MB
> into Cassandra, and I was hoping for some advice on tracking down what's
> going wrong. From the FAQ it seems like what I'm trying to do is possible,
> so I assume I'm messing something up with my configuration. I have a minimal
> set of code to reproduce the issue below, which I've run on the DataStax
> 0.8.1 AMI I'm using in production (ami-9996c4dc)
>
> # To set up the test data structure on Cassandra:
>
> cassandra-cli
> connect localhost/9160;
> create keyspace TestKeyspace with
>   placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and
>   strategy_options = [{replication_factor:3}];
> use TestKeyspace;
> create column family TestFamily with
>   comparator = UTF8Type and
>   column_metadata =
>   [
>     {column_name: test_column, validation_class: UTF8Type},
>   ];
>
> # From bash on the same machine, with Ruby and the Cassandra gem installed:
>
> irb
> require 'rubygems'
> require 'cassandra/0.8'
>
> client = Cassandra.new('TestKeyspace', 'localhost:9160', :retries => 5,
> :connect_timeout => 5, :timeout => 10)
>
> # With data this size, the call works
> column_value = 'a' * (14*1024*1024)
> row_value = { 'column_name' => column_value }
> client.insert(:TestFamily, 'SomeKey', row_value)
>
>
> # With data this size, the call fails with the exception below
> column_value = 'a' * (15*1024*1024)
> row_value = { 'column_name' => column_value }
> client.insert(:TestFamily, 'SomeKey', row_value)
>
> # Results:
>
> This first call with a 14MB chunk of data succeeds, but the second one
> fails with this exception:
>
> CassandraThrift::Cassandra::Client::TransportException:
> CassandraThrift::Cassandra::Client::TransportException
>  from
> /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/socket.rb:53:in
> `open'
> from
> /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/framed_transport.rb:37:in
> `open'
>  from
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/connection/socket.rb:11:in
> `connect!'
> from
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:105:in
> `connect!'
>  from
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:144:in
> `handled_proxy'
> from
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:60:in
> `batch_mutate'
>  from
> /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/protocol.rb:7:in
> `_mutate'
> from
> /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/cassandra.rb:459:in
> `insert'
>  from (irb):6
> from :0
>
> Any suggestions on how to dig deeper? I'll be reaching out to the Cassandra
> gem folks, etc too of course.
>
> cheers,
>            Pete
>



-- 
Tyler Hobbs
Software Engineer, DataStax <http://datastax.com/>
Maintainer of the pycassa <http://github.com/pycassa/pycassa> Cassandra
Python client library

Mime
View raw message