Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CDB7B9E58 for ; Fri, 24 Feb 2012 13:21:35 +0000 (UTC) Received: (qmail 30815 invoked by uid 500); 24 Feb 2012 13:21:33 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 30781 invoked by uid 500); 24 Feb 2012 13:21:33 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 30773 invoked by uid 99); 24 Feb 2012 13:21:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Feb 2012 13:21:33 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of patrik.modesto@gmail.com designates 209.85.215.44 as permitted sender) Received: from [209.85.215.44] (HELO mail-lpp01m010-f44.google.com) (209.85.215.44) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Feb 2012 13:21:25 +0000 Received: by lahl5 with SMTP id l5so3308185lah.31 for ; Fri, 24 Feb 2012 05:21:05 -0800 (PST) Received-SPF: pass (google.com: domain of patrik.modesto@gmail.com designates 10.152.102.145 as permitted sender) client-ip=10.152.102.145; Authentication-Results: mr.google.com; spf=pass (google.com: domain of patrik.modesto@gmail.com designates 10.152.102.145 as permitted sender) smtp.mail=patrik.modesto@gmail.com; dkim=pass header.i=patrik.modesto@gmail.com Received: from mr.google.com ([10.152.102.145]) by 10.152.102.145 with SMTP id fo17mr1863338lab.2.1330089665293 (num_hops = 1); Fri, 24 Feb 2012 05:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; bh=GwPtjJooWa4MSjGOtAi92aywVU9qPIlJNPwgsvz14tw=; b=h807/eAsgU+VyNZqeo/CQY2L+VlfPwKscbpcrCIMa01MqiUg77GZDO7OZsoiCr3MSL VhN2xtRuDsKayYIZXo67Tabvdh08128Oqt1L8vxFj4NDZq7J38aWbYPxpw22TIcrtDof ekJhmEnbqUee3XZIMqSl7ZxTE5FtnaEdZVzSs= Received: by 10.152.102.145 with SMTP id fo17mr1544188lab.2.1330089665237; Fri, 24 Feb 2012 05:21:05 -0800 (PST) MIME-Version: 1.0 Received: by 10.152.129.131 with HTTP; Fri, 24 Feb 2012 05:20:35 -0800 (PST) From: Patrik Modesto Date: Fri, 24 Feb 2012 14:20:35 +0100 Message-ID: Subject: newer Cassandra + Hadoop = TimedOutException() To: user@cassandra.apache.org Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org Hi, I can see some strange behaviour on my test cluster and in production. Both running cassandra 0.8.10. Strange is that when I compile my mapreduce job against cassandra-all 0.8.7 everything is ok, but if I use higher version I get quite a lots of TimedOutException. java.lang.RuntimeException: TimedOutException() at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.maybeInit(ColumnFamilyRecordReader.java:319) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:333) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.computeNext(ColumnFamilyRecordReader.java:207) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130) at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:163) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456) at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:32 There is nothing in the cassandra log, the cluster is idle, no one else is accessing the cluster. There are just few rows, nothing big: INFO mapred.JobClient: Reduce input groups=14 INFO mapred.JobClient: Combine output records=0 INFO mapred.JobClient: Map input records=544009 INFO mapred.JobClient: Reduce shuffle bytes=33876 INFO mapred.JobClient: Reduce output records=0 INFO mapred.JobClient: Spilled Records=38 INFO mapred.JobClient: Map output bytes=33656 INFO mapred.JobClient: Combine input records=0 INFO mapred.JobClient: Map output records=19 INFO mapred.JobClient: SPLIT_RAW_BYTES=3937 INFO mapred.JobClient: Reduce input records=19 What could be the problem? Regards, P.