Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of okrammarko@gmail.com designates
 209.85.160.44 as permitted sender)
From: Marko Rodriguez <okrammarko@gmail.com>
Mime-Version: 1.0 (Apple Message framework v1085)
Content-Type: multipart/alternative; boundary=Apple-Mail-5--47784290
Subject: Re: Frame size exceptions occurring with ColumnFamilyInputFormat for
 very large rows
Date: Tue, 27 Nov 2012 19:17:33 -0700
In-Reply-To: <48334066-2C22-411D-B2F5-DCE1EBD71375@gmail.com>
To: user@cassandra.apache.org
References: <48334066-2C22-411D-B2F5-DCE1EBD71375@gmail.com>
Message-Id: <0D139925-F4F6-4FD4-8224-C7E3894F541C@gmail.com>


--Apple-Mail-5--47784290
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hello,

I was wondering if anyone had an answer to my previous message below.=20

Seems another is having the same problem, but unfortunately with no =
response as well.
	=
http://mail-archives.apache.org/mod_mbox/cassandra-user/201211.mbox/%3C509=
A4A1F.8070506@semantico.com%3E
=09
Any help would be much appreciated.

Thank you,
Marko.

http://markorodriguez.com

On Nov 9, 2012, at 3:02 PM, Marko Rodriguez wrote:

> Hello,
>=20
> I am trying to run a Hadoop job that pulls data out of Cassandra via =
ColumnFamilyInputFormat. I am getting a "frame size" exception. To =
remedy that, I have set both the thrift_framed_transport_size_in_mb and =
thrift_max_message_length_in_mb to an "infinite" amount at 100000mb on =
all nodes. Moreover, I have restarted the cluster and the cassandra.yaml =
files have been reloaded.
>=20
> However, I am still getting:
>=20
> 12/11/09 21:39:52 INFO mapred.JobClient:  map 62% reduce 0%
> 12/11/09 21:40:09 INFO mapred.JobClient: Task Id : =
attempt_201211082011_0015_m_000479_2, Status : FAILED
> java.lang.RuntimeException: =
org.apache.thrift.transport.TTransportException: Frame size (30046945) =
larger than max length (16384000)!
> 	at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.may=
beInit(ColumnFamilyRecordReader.java:400)
> 	at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.com=
puteNext(ColumnFamilyRecordReader.java:406)
> 	at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.com=
puteNext(ColumnFamilyRecordReader.java:324)
> 	at =
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterat=
or.java:143)
> 	at =
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:1=
38)
> 	at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFa=
milyRecordReader.java:189)
>=20
> Question: Why is 16384000 bytes (I assume) !=3D  100000mb?
>=20
> Next, I made this parameter true as a last hail mary attempt:
> 	cassandra.input.widerows=3Dtrue
> ...still with no luck.
>=20
> Does someone know what I might be missing?
>=20
> Thank you very much for your time,
> Marko.
>=20
> http://markorodriguez.com


--Apple-Mail-5--47784290
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
">Hello,<div><br></div><div>I was wondering if anyone had an answer to =
my previous message below.&nbsp;</div><div><br></div><div>Seems another =
is having the same problem, but unfortunately with no response as =
well.</div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	=
</span><a =
href=3D"http://mail-archives.apache.org/mod_mbox/cassandra-user/201211.mbo=
x/&lt;509A4A1F.8070506@semantico.com&gt;">http://mail-archives.apache.org/=
mod_mbox/cassandra-user/201211.mbox/%3C509A4A1F.8070506@semantico.com%3E</=
a></div><div><span class=3D"Apple-tab-span" style=3D"white-space:pre">	=
</span></div><div>Any help would be much =
appreciated.</div><div><br></div><div>Thank =
you,</div><div>Marko.</div><div><br><div>
<a href=3D"http://markorodriguez.com">http://markorodriguez.com</a>
</div>
<br><div><div>On Nov 9, 2012, at 3:02 PM, Marko Rodriguez =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><div style=3D"word-wrap: break-word; -webkit-nbsp-mode: =
space; -webkit-line-break: after-white-space; ">Hello,<br><br>I am =
trying to run a Hadoop job that pulls data out of Cassandra via =
ColumnFamilyInputFormat. I am getting a "frame size" exception. To =
remedy that, I have set both the thrift_framed_transport_size_in_mb and =
thrift_max_message_length_in_mb to an "infinite" amount at 100000mb on =
all nodes. Moreover, I have restarted the cluster and the cassandra.yaml =
files have been reloaded.<br><br>However, I am still =
getting:<br><br>12/11/09 21:39:52 INFO mapred.JobClient: &nbsp;map 62% =
reduce 0%<br>12/11/09 21:40:09 INFO mapred.JobClient: Task Id : =
attempt_201211082011_0015_m_000479_2, Status : =
FAILED<br>java.lang.RuntimeException: =
org.apache.thrift.transport.TTransportException: Frame size (30046945) =
larger than max length (16384000)!<br><span class=3D"Apple-tab-span" =
style=3D"white-space: pre; ">	</span>at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.may=
beInit(ColumnFamilyRecordReader.java:400)<br><span =
class=3D"Apple-tab-span" style=3D"white-space: pre; ">	</span>at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.com=
puteNext(ColumnFamilyRecordReader.java:406)<br><span =
class=3D"Apple-tab-span" style=3D"white-space: pre; ">	</span>at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.com=
puteNext(ColumnFamilyRecordReader.java:324)<br><span =
class=3D"Apple-tab-span" style=3D"white-space: pre; ">	</span>at =
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterat=
or.java:143)<br><span class=3D"Apple-tab-span" style=3D"white-space: =
pre; ">	</span>at =
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:1=
38)<br><span class=3D"Apple-tab-span" style=3D"white-space: pre; ">	=
</span>at =
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFa=
milyRecordReader.java:189)<br><br>Question: Why is 16384000 bytes (I =
assume) !=3D &nbsp;100000mb?<br><br>Next, I made this parameter true as =
a last hail mary attempt:<br><span class=3D"Apple-tab-span" =
style=3D"white-space: pre; ">	=
</span>cassandra.input.widerows=3Dtrue<br>...still with no =
luck.<br><br>Does someone know what I might be missing?<br><br>Thank you =
very much for your time,<br>Marko.<br><br><a =
href=3D"http://markorodriguez.com/">http://markorodriguez.com</a><br></div=
></blockquote></div><br></div></body></html>=

--Apple-Mail-5--47784290--