Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of dean.hiller@gmail.com
 designates 209.85.213.172 as permitted sender)
MIME-Version: 1.0
Sender: dean.hiller@gmail.com
From: Dean Hiller <dean@alvazan.com>
Date: Mon, 14 Nov 2011 19:28:53 -0700
Message-ID: 
 <CAAtG5j5qQQ-j-mtG-g2bgaF+sJ+PVRLV2z56pBe1oPpvL6gMxA@mail.gmail.com>
Subject: This seems to fix it.....wordcount example exception that I think
 should not occur
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=bcaec5215f77d6035f04b1bcc097

--bcaec5215f77d6035f04b1bcc097
Content-Type: text/plain; charset=ISO-8859-1

Well, by edting
src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordReader.java

in version 1.0.2 cassandra src just before the

            totalRead++;
            KeySlice ks = rows.get(i++);
            SortedMap<ByteBuffer, IColumn> map = new TreeMap<ByteBuffer,
IColumn>(comparator);

I added the code

            if(i >= rows.size())
                return endOfData();

That seems to fix it AND this is only needed by the jobtracker so you don't
need to redeploy cassandra for the fix which is nice....it seems to work
now with rows that don't contain that column.  If anyone thinks this will
have bad side affects, please let me know.

later,
Dean  http://blog.alvazan.com


On Mon, Nov 14, 2011 at 7:01 PM, Dean Hiller <dean@alvazan.com> wrote:

> There are 4 jobs submitted by the wordcount cassandra example and the
> first one fails and the other 3 all pass and work with results.
>
> The first job I noticed is looking for column name text0 due to i being 0
> in the loop.  The exception is not going through the wordcount code at all
> though, but this text0 column does not exist in the cassandra database(is
> this an InputFormat bug in the cassandra code to wire in the map/reduce
> stuff or is there a way to fix this???)
>
> The exception...
>
> 2011-11-14 18:16:26,540 INFO org.apache.hadoop.mapred.TaskInProgress:
> Error from attempt_201111141812_0002_m_000000_1:
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>     at java.util.ArrayList.RangeCheck(ArrayList.java:547)
>     at java.util.ArrayList.get(ArrayList.java:322)
>     at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:300)
>     at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:183)
>     at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
>     at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
>     at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:139)
>     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:423)
>     at
> org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>     at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
> thanks,
> Dean
>

--bcaec5215f77d6035f04b1bcc097
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Well, by edting src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordRead=
er.java<br><br>in version 1.0.2 cassandra src just before the <br><br>=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 totalRead++;<br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0 KeySlice ks =3D rows.get(i++);<br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =
SortedMap&lt;ByteBuffer, IColumn&gt; map =3D new TreeMap&lt;ByteBuffer, ICo=
lumn&gt;(comparator);<br>

<br>I added the code<br><br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if(i &gt;=3D =
rows.size())<br>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 return endOfD=
ata();<br><br>That seems to fix it AND this is only needed by the jobtracke=
r so you don&#39;t need to redeploy cassandra for the fix which is nice....=
it seems to work now with rows that don&#39;t contain that column.=A0 If an=
yone thinks this will have bad side affects, please let me know.<br>

<br>later,<br>
Dean=A0 <a href=3D"http://blog.alvazan.com">http://blog.alvazan.com</a><br>=
<br><br><br><div class=3D"gmail_quote">On Mon, Nov 14, 2011 at 7:01 PM, Dea=
n Hiller <span dir=3D"ltr">&lt;<a href=3D"mailto:dean@alvazan.com">dean@alv=
azan.com</a>&gt;</span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">There are 4 jobs submitted by the wordcount=
 cassandra example and the first one fails and the other 3 all pass and wor=
k with results.<br>

<br>The first job I noticed is looking for column name text0 due to i being=
 0 in the loop.=A0 The exception is not going through the wordcount code at=
 all though, but this text0 column does not exist in the cassandra database=
(is this an InputFormat bug in the cassandra code to wire in the map/reduce=
 stuff or is there a way to fix this???)<br>


<br>The exception...<br><br>2011-11-14 18:16:26,540 INFO org.apache.hadoop.=
mapred.TaskInProgress: Error from attempt_201111141812_0002_m_000000_1: jav=
a.lang.IndexOutOfBoundsException: Index: 0, Size: 0<br>=A0=A0=A0 at java.ut=
il.ArrayList.RangeCheck(ArrayList.java:547)<br>


=A0=A0=A0 at java.util.ArrayList.get(ArrayList.java:322)<br>=A0=A0=A0 at or=
g.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(=
ColumnFamilyRecordReader.java:300)<br>=A0=A0=A0 at org.apache.cassandra.had=
oop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordRead=
er.java:183)<br>


=A0=A0=A0 at com.google.common.collect.AbstractIterator.tryToComputeNext(Ab=
stractIterator.java:140)<br>=A0=A0=A0 at com.google.common.collect.Abstract=
Iterator.hasNext(AbstractIterator.java:135)<br>=A0=A0=A0 at org.apache.cass=
andra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader=
.java:139)<br>


=A0=A0=A0 at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextK=
eyValue(MapTask.java:423)<br>=A0=A0=A0 at org.apache.hadoop.mapreduce.MapCo=
ntext.nextKeyValue(MapContext.java:67)<br>=A0=A0=A0 at org.apache.hadoop.ma=
preduce.Mapper.run(Mapper.java:143)<br>


=A0=A0=A0 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621=
)<br>=A0=A0=A0 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)<br=
>=A0=A0=A0 at org.apache.hadoop.mapred.Child.main(Child.java:170)<br><br>th=
anks,<br>Dean<br>


</blockquote></div><br>

--bcaec5215f77d6035f04b1bcc097--