incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pete Warden <p...@petewarden.com>
Subject Re: TransportException when storing large values
Date Wed, 21 Sep 2011 22:06:00 GMT
Thanks guys, that was precisely the problem. I also upped
the thrift_max_message_length_in_mb setting, since I guessed the default of
16MB would also block larger messages.

On Tue, Sep 20, 2011 at 6:47 PM, Jim Ancona <jim@anconafamily.com> wrote:

> Pete,
>
> See this thread
>
> http://groups.google.com/group/hector-users/browse_thread/thread/cb3e72c85dbdd398/82b18ffca0e3940a?#82b18ffca0e3940a
> for a bit more info.
>
> Jim
>
> On Tue, Sep 20, 2011 at 9:02 PM, Tyler Hobbs <tyler@datastax.com> wrote:
> > From cassandra.yaml:
> >
> > # Frame size for thrift (maximum field length).
> > # 0 disables TFramedTransport in favor of TSocket. This option
> > # is deprecated; we strongly recommend using Framed mode.
> > thrift_framed_transport_size_in_mb: 15
> >
> > So you can either increase that limit, or split your write into multiple
> > operations.
> >
> > On Tue, Sep 20, 2011 at 4:26 PM, Pete Warden <pete@petewarden.com>
> wrote:
> >>
> >> I'm running into consistent problems when storing values larger than
> 15MB
> >> into Cassandra, and I was hoping for some advice on tracking down what's
> >> going wrong. From the FAQ it seems like what I'm trying to do is
> possible,
> >> so I assume I'm messing something up with my configuration. I have a
> minimal
> >> set of code to reproduce the issue below, which I've run on the DataStax
> >> 0.8.1 AMI I'm using in production (ami-9996c4dc)
> >> # To set up the test data structure on Cassandra:
> >> cassandra-cli
> >> connect localhost/9160;
> >> create keyspace TestKeyspace with
> >>   placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and
> >>   strategy_options = [{replication_factor:3}];
> >> use TestKeyspace;
> >> create column family TestFamily with
> >>   comparator = UTF8Type and
> >>   column_metadata =
> >>   [
> >>     {column_name: test_column, validation_class: UTF8Type},
> >>   ];
> >> # From bash on the same machine, with Ruby and the Cassandra gem
> >> installed:
> >> irb
> >> require 'rubygems'
> >> require 'cassandra/0.8'
> >> client = Cassandra.new('TestKeyspace', 'localhost:9160', :retries => 5,
> >> :connect_timeout => 5, :timeout => 10)
> >> # With data this size, the call works
> >> column_value = 'a' * (14*1024*1024)
> >> row_value = { 'column_name' => column_value }
> >> client.insert(:TestFamily, 'SomeKey', row_value)
> >>
> >> # With data this size, the call fails with the exception below
> >> column_value = 'a' * (15*1024*1024)
> >> row_value = { 'column_name' => column_value }
> >> client.insert(:TestFamily, 'SomeKey', row_value)
> >> # Results:
> >> This first call with a 14MB chunk of data succeeds, but the second one
> >> fails with this exception:
> >> CassandraThrift::Cassandra::Client::TransportException:
> >> CassandraThrift::Cassandra::Client::TransportException
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/socket.rb:53:in
> >> `open'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/thrift-0.7.0/lib/thrift/transport/framed_transport.rb:37:in
> >> `open'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/connection/socket.rb:11:in
> >> `connect!'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:105:in
> >> `connect!'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:144:in
> >> `handled_proxy'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/thrift_client-0.7.1/lib/thrift_client/abstract_thrift_client.rb:60:in
> >> `batch_mutate'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/protocol.rb:7:in
> >> `_mutate'
> >> from
> >>
> /usr/lib/ruby/gems/1.8/gems/cassandra-0.12.1/lib/cassandra/cassandra.rb:459:in
> >> `insert'
> >> from (irb):6
> >> from :0
> >> Any suggestions on how to dig deeper? I'll be reaching out to the
> >> Cassandra gem folks, etc too of course.
> >> cheers,
> >>            Pete
> >
> >
> > --
> > Tyler Hobbs
> > Software Engineer, DataStax
> > Maintainer of the pycassa Cassandra Python client library
> >
> >
>

Mime
View raw message