Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A05B7108D1 for ; Sat, 17 Jan 2015 01:12:20 +0000 (UTC) Received: (qmail 29508 invoked by uid 500); 17 Jan 2015 01:12:19 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 29464 invoked by uid 500); 17 Jan 2015 01:12:19 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 29454 invoked by uid 99); 17 Jan 2015 01:12:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 Jan 2015 01:12:19 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ragrawal@clearpoolgroup.com designates 72.5.42.24 as permitted sender) Received: from [72.5.42.24] (HELO webmail.fusionts.com) (72.5.42.24) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 Jan 2015 01:12:14 +0000 Received: from exchange2010.dimcap.corp ([::1]) by exchange2010.dimcap.corp ([::1]) with mapi id 14.01.0218.012; Fri, 16 Jan 2015 20:11:09 -0500 From: Ravi Agrawal To: "user@cassandra.apache.org" Subject: RE: Retrieving all row keys of a CF Thread-Topic: Retrieving all row keys of a CF Thread-Index: AQHQMe0UR699dOF4JkqKE8muGHRjJpzDgOMg Date: Sat, 17 Jan 2015 01:11:09 +0000 Message-ID: <1E807EB9EA831F4184920D0403D4CD6D014EC204@exchange2010.dimcap.corp> References: <1E807EB9EA831F4184920D0403D4CD6D014EC193@exchange2010.dimcap.corp> <045D8FD556C73347A47F956EE65F82201857C0B4@S11MAILD013N2.sh11.lan> In-Reply-To: <045D8FD556C73347A47F956EE65F82201857C0B4@S11MAILD013N2.sh11.lan> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.17.12.191] Content-Type: multipart/alternative; boundary="_000_1E807EB9EA831F4184920D0403D4CD6D014EC204exchange2010dim_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_1E807EB9EA831F4184920D0403D4CD6D014EC204exchange2010dim_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable 1) What is the heap size and total memory on each node? 8GB, 8GB 2) How big is the cluster? 4 3) What are the read and range timeouts (in cassandra.yaml) on t= he C* nodes? 10 secs, 10 secs 4) What are the timeouts for the Astyanax client? 2 secs 5) Do you see GC pressure on the C* nodes? How long does GC for = new gen and old gen take? occurs every 5 secs dont see huge gc pressure, <5= 0ms 6) Does any node crash with OOM error when you try AllRowsReader= ? No From: Mohammed Guller [mailto:mohammed@glassbeam.com] Sent: Friday, January 16, 2015 7:30 PM To: user@cassandra.apache.org Subject: RE: Retrieving all row keys of a CF A few questions: 1) What is the heap size and total memory on each node? 2) How big is the cluster? 3) What are the read and range timeouts (in cassandra.yaml) on the C* = nodes? 4) What are the timeouts for the Astyanax client? 5) Do you see GC pressure on the C* nodes? How long does GC for new ge= n and old gen take? 6) Does any node crash with OOM error when you try AllRowsReader? Mohammed From: Ravi Agrawal [mailto:ragrawal@clearpoolgroup.com] Sent: Friday, January 16, 2015 4:14 PM To: user@cassandra.apache.org Subject: Re: Retrieving all row keys of a CF Hi, I and Ruchir tried query using AllRowsReader recipe but had no luck. We are= seeing PoolTimeoutException. SEVERE: [Thread_1] Error reading RowKeys com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException: PoolTi= meoutException: [host=3Dservername, latency=3D2003(2003), attempts=3D4]Time= d out waiting for connection at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool= .waitForConnection(SimpleHostConnectionPool.java:231) at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool= .borrowConnection(SimpleHostConnectionPool.java:198) at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFai= lover.borrowConnection(RoundRobinExecuteWithFailover.java:84) at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailo= verImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:117) at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionCon= nectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:33= 8) at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$2.execute= (ThriftColumnFamilyQueryImpl.java:397) at com.netflix.astyanax.recipes.reader.AllRowsReader$1.call(AllRowsR= eader.java:447) at com.netflix.astyanax.recipes.reader.AllRowsReader$1.call(AllRowsR= eader.java:419) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecu= tor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExec= utor.java:615) at java.lang.Thread.run(Thread.java:745) We did receive a portion of data which changes on every try. We used follow= ing method. boolean result =3D new AllRowsReader.Builder(keyspace, CF_S= TANDARD1) .withColumnRange(null, null, false, 0) .withPartitioner(null) // this will use keyspace's partitioner .forEachRow(new Function, Boolean>() { @Override public Boolean apply(@Nullable Row row) { // Process the row here ... return true; } }) .build() .call(); Tried setting concurrency level as mentioned in this post (https://github.c= om/Netflix/astyanax/issues/411) as well on both astyanax 1.56.49 and 2.0.0.= Still nothing. --_000_1E807EB9EA831F4184920D0403D4CD6D014EC204exchange2010dim_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

 

1)        =     What is the heap size and total memory on each node? 8GB, 8GB

2)        =     How big is the cluster? 4

3)        =     What are the read and range timeouts (in cassandra.yaml)= on the C* nodes? 10 secs, 10 secs

4)        =     What are the timeouts for the Astyanax client? 2 secs

5)        =     Do you see GC pressure on the C* nodes? How long does GC= for new gen and old gen take? occurs every 5 secs dont see huge gc press= ure, <50ms

6)        =     Does any node crash with OOM error when you try AllRowsR= eader? No

 

From: Mohammed Guller [mailto:mohammed@glassb= eam.com]
Sent: Friday, January 16, 2015 7:30 PM
To: user@cassandra.apache.org
Subject: RE: Retrieving all row keys of a CF

 

A few questions:<= /o:p>

 

1)&n= bsp;     What is the he= ap size and total memory on each node?

2)&n= bsp;     How big is the= cluster?

3)&n= bsp;     What are the r= ead and range timeouts (in cassandra.yaml) on the C* nodes?

4)&n= bsp;     What are the t= imeouts for the Astyanax client?

5)&n= bsp;     Do you see GC = pressure on the C* nodes? How long does GC for new gen and old gen take?

6)&n= bsp;     Does any node = crash with OOM error when you try AllRowsReader?

 

Mohammed

 

From: Ravi Agrawal [mailto:ragrawal@clearpoolgroup.com]
Sent: Friday, January 16, 2015 4:14 PM
To: user@cassandra.apac= he.org
Subject: Re: Retrieving all row keys of a CF

 

Hi,

I and= Ruchir tried query using AllRowsReader recipe but had no luck. We are seei= ng PoolTimeoutException.

SEVERE: [Thread_1] Error reading = RowKeys

com.netflix.astyanax.conne= ctionpool.exceptions.PoolTimeoutException: PoolTimeoutException: [host=3Dservername, latency=3D2003(2003), attempts= =3D4]Timed out waiting for connection

     &nb= sp; at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.wa= itForConnection(SimpleHostConnectionPool.java:231)

     &nb= sp; at com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.bo= rrowConnection(SimpleHostConnectionPool.java:198)

     &nb= sp; at com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailov= er.borrowConnection(RoundRobinExecuteWithFailover.java:84)

     &nb= sp; at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailover= Impl.tryOperation(AbstractExecuteWithFailoverImpl.java:117)

     &nb= sp; at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnec= tionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:33= 8)<= /span>

     &nb= sp; at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$2.execute(ThriftColumnFamilyQueryImpl.java:397)

     &nb= sp; at com.netflix.astyanax.recipes.reader.AllRowsReader$1.call(<= span style=3D"font-size:10.0pt;font-family:Consolas;color:#0066CC">AllRowsR= eader.java:447)

     &nb= sp; at com.netflix.astyanax.recipes.reader.AllRowsReader$1.call(<= span style=3D"font-size:10.0pt;font-family:Consolas;color:#0066CC">AllRowsR= eader.java:419)

     &nb= sp; at java.util.concurrent.FutureTask.run(FutureTask.java:262)

     &nb= sp; at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecu= tor.java:1145)=

     &nb= sp; at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExec= utor.java:615)=

       at java.lang.Thread.run(Thread.java:745)

 

We di= d receive a portion of data which changes on every try. We used following m= ethod.

boolean result = =3D = new AllRowsReader.Builder<String, = String>(keyspace, = CF_STANDARD1)

     =    .withColumnRange(null, = null, = false, = 0)

     =    .withPartitioner(null) = // this will use keyspace's partitioner

     =    .forEachRow(new = Function<Row<String, = String>, = Boolean>() {

     =        = @Override

     =        = public Boolean = apply(@Nullable = Row<String, = String> = row) {

     =            = // Process the row here ...

     =            = return = true;

     =        }

     =    })

     =    .build()

     =    .call();

=  

Tried= setting concurrency level as mentioned in this post (https://github.com/Netflix/astyanax/i= ssues/411) as well on both astyanax 1.56.49 and 2.0.0. Still nothing.

--_000_1E807EB9EA831F4184920D0403D4CD6D014EC204exchange2010dim_--