Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8819D200B3C for ; Wed, 13 Jul 2016 22:20:21 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 868A7160A6A; Wed, 13 Jul 2016 20:20:21 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 7EA81160A62 for ; Wed, 13 Jul 2016 22:20:20 +0200 (CEST) Received: (qmail 96859 invoked by uid 500); 13 Jul 2016 20:20:19 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 96849 invoked by uid 99); 13 Jul 2016 20:20:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Jul 2016 20:20:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id B161F18028E for ; Wed, 13 Jul 2016 20:20:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.198 X-Spam-Level: * X-Spam-Status: No, score=1.198 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Jz4ZYsVxeVyn for ; Wed, 13 Jul 2016 20:20:16 +0000 (UTC) Received: from mail-oi0-f44.google.com (mail-oi0-f44.google.com [209.85.218.44]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id A9DA160CCF for ; Wed, 13 Jul 2016 20:20:15 +0000 (UTC) Received: by mail-oi0-f44.google.com with SMTP id j185so81090110oih.0 for ; Wed, 13 Jul 2016 13:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ovI052zBwbx7XPkCi6GzPeMI9Ct8sLtSlposzsriIVc=; b=c7kRddMor6r7VJgF/Im02/6uQxGXFHZQsXaV4Lzs5xM/Cshbddz0YTnu2uOh1Camna A+7jk7CDV6jTDWqytrV5+9xfrMFswIVGS5BoItftJZVnmNUIQijhTeT8F3hnF6dgqQ/v 7CGqgDlWKyxaaHWY2z/ESys5X+jRKsWMfzcrr6dFSS9Jhfpo2VDi0M5NMgPWrjrD61WK o5IKIH6+rC/GlSi3x33QST7G0TxXq+CUFsOr2GQ57xqwkgFXydQtEThxTnPnPSWaRk9g 9K67etjfz0YtZbClaiVsdtXv/VvisBJGQdxRA0g/4bTE7VVbvCI4BcR4zKCI8/aaQ890 910w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ovI052zBwbx7XPkCi6GzPeMI9Ct8sLtSlposzsriIVc=; b=fTgVoe2PTrImEb0V6B4qxSJO1+hLrnVYpVeAM8gAk0Z3RV3SRMhTOKppScoI3DpJ0b p8l37uZyAeTaE6TCTmw1aULBNyyWXpCcamOAJEtBjt9hecac8DHrcCTTm5NB759CMhP8 VN2TUL8/NvIdRoyhlIPK1shSr5AKLZndq3mJnlkj2Fw+0pxogFGX7zQhaIjcSLvMxRjJ MQLkEqOSKjGzANCbLcpDtzE9lJ95JUDEe9Ql+8dsIJ4WHU6PTctDcUO22ENoK7hIRd0n 1k9B6Kw6tuo0Nr0nDWh+9Z312yZVwf5rG8spYJTDJDy9dRU79r2zcL5w8eN5MmQeyYkK xqzw== X-Gm-Message-State: ALyK8tKote3CFoK2CamriIDpdeNu26pekMn3i6yspLk10IOyVqu2m8NMgkJ1Nk97m/VVsijrvT7luNr2L6A9zA== X-Received: by 10.202.80.78 with SMTP id e75mr2384071oib.91.1468441214444; Wed, 13 Jul 2016 13:20:14 -0700 (PDT) MIME-Version: 1.0 References: <1058113985.4230290.1468440156697.JavaMail.yahoo@mail.yahoo.com> In-Reply-To: <1058113985.4230290.1468440156697.JavaMail.yahoo@mail.yahoo.com> From: Abhinav Solan Date: Wed, 13 Jul 2016 20:20:04 +0000 Message-ID: Subject: Re: NoHostAvailableException coming up on our server To: Romain Hardouin , "user@cassandra.apache.org" Content-Type: multipart/alternative; boundary=001a113d64b2f5ea6605378a1c4b archived-at: Wed, 13 Jul 2016 20:20:21 -0000 --001a113d64b2f5ea6605378a1c4b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks a lot for suggestion Romain, I have done the setup to see the driver logs, but haven't seen that error again. Also thanks for the MaxRequestPerConnection tip, I will change it to 32K. Regards, Abhinav On Wed, Jul 13, 2016 at 1:02 PM Romain Hardouin wrote= : > Put the driver logs in debug mode to see what's happen. > Btw I am surprised by the few requests by connections in your setup: > > .setConnectionsPerHost(HostDistance.LOCAL, 20, 20) > .setMaxRequestsPerConnection(HostDistance.LOCAL, 128) > > It looks like a protocol v2 settings (Cassandra 2.0) because it was > limited to 128 requests per connection. You're using C* 3.3 so the protoc= ol > v4. > You can go up to 32K since protocol v3. As a first step I would try to > open only 2 connections with 16K in MaxRequestsPerConnection. Then try to > fine tune. > > Best, > > Romain > > > Le Mardi 12 juillet 2016 23h57, Abhinav Solan a > =C3=A9crit : > > > I am using 3.0.0 version over apache-cassandra-3.3 > > On Tue, Jul 12, 2016 at 2:37 PM Riccardo Ferrari > wrote: > > What driver version are you using? > > You can look at the LoggingRetryPolicy to have more meaningful messages i= n > your logs. > > best, > > On Tue, Jul 12, 2016 at 9:02 PM, Abhinav Solan > wrote: > > Thanks, Johnny > Actually, they were running .. it went through a series of read and write= s > .. and recovered after the error. > Is there any settings I can specify in preparing the Session at java > client driver level, here are my current settings - > > PoolingOptions poolingOptions =3D new PoolingOptions() > .setConnectionsPerHost(HostDistance.LOCAL, 20, 20) > .setMaxRequestsPerConnection(HostDistance.LOCAL, 128) > .setNewConnectionThreshold(HostDistance.LOCAL, 100); > > Cluster.Builder builder =3D Cluster.builder() > .addContactPoints(cp) > .withPoolingOptions(poolingOptions) > .withProtocolVersion(ProtocolVersion.NEWEST_SUPPORTED) > .withPort(port); > > > > On Tue, Jul 12, 2016 at 11:47 AM Johnny Miller > wrote: > > Abhinav - your getting that as the driver isn=E2=80=99t finding any hosts= up for > your query. You probably need to check if all the nodes in your cluster a= re > running. > > See: > http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/exc= eptions/NoHostAvailableException.html > > > Johnny > > On 12 Jul 2016, at 18:46, Abhinav Solan wrote: > > Hi Everyone, > > I am getting this error on our server, it comes and goes seems the > connection drops a comes back after a while - > > Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: = All host(s) tried for query failed (tried: ****************:9042 (com.datas= tax.driver.core.exceptions.ConnectionException: [****************] Pool is = CLOSING)) > at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHand= ler.java:218) > at com.datastax.driver.core.RequestHandler.access$1000(RequestHandler.ja= va:43) > at com.datastax.driver.core.RequestHandler$SpeculativeExecution.sendRequ= est(RequestHandler.java:284) > at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHand= ler.java:115) > at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.ja= va:91) > at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.j= ava:129) > > Can anyone suggest me what can be done to handle this error ? > > > Thanks, > > Abhinav > > > > > > --001a113d64b2f5ea6605378a1c4b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks a lot for suggestion Romain, I have done the setup = to see the driver logs, but haven't seen that error again.
Also tha= nks for the MaxRequestPerConnection tip, I will change it to 32K.

Regards,
Abhinav

On Wed, Jul 13, 2016 at 1:02 PM Romain Hardouin = <romainh_ml@yahoo.fr> wrot= e:
Put the dr= iver logs in debug mode to see what's happen.
Btw I am surprised by the few requests by connections in your setu= p:
        .setConnection=
sPerHost(HostDistance.LOCAL, 20=
, 20)
         .setMaxRequestsPerConnection(HostDistance.LOCAL, 128)
=
It looks like a protocol v2 settings (Ca= ssandra 2.0) because it was limited to 128 requests per connection. You'= ;re using C* 3.3 so the protocol v4.
You can go up to 32K since protocol v3. As a first step I would try= to open only 2 connections with 16K in=C2=A0MaxRequestsPerConnection. Then try to fine tune.

Best,

Romain

=
= Le Mardi 12 juillet 2016 23h57, Abhinav So= lan <abhina= v.solan@gmail.com> a =C3=A9crit :


I am usin= g 3.0.0 version over apache-cassandra-3.3=C2=A0

On = Tue, Jul 12, 2016 at 2:37 PM Riccardo Ferrari <ferrarir@gma= il.com> wrote:
What driver version are you using?

You can look at th= e LoggingRetryPolicy to have more meaningful messages in your logs.<= /div>

best,

On Tue, Jul 12, 2016 at 9:02 PM, Abhinav Sol= an <abhinav.solan@gmail.com><= /span> wrote:
Thanks, Johnny
Actually, they were= running .. it went through a series of read and writes .. and recovered af= ter the error.
Is there any settings I can spec= ify in preparing the Session at java client driver level, here are my curre= nt settings -=C2=A0
PoolingOp=
tions poolingOptions =3D new P=
oolingOptions()
         .setConnectionsPerHost(HostDista=
nce.LOCAL, 20, 20)
         .setMaxR=
equestsPerConnection(HostDistance.LOCAL, =
128)
      =
   .setNewConnectionThreshold(HostDistance.LOCAL=
, 100);

 Cluster.Builder builder =3D Cluster.builder()
         .addContactPoints(cp)
         .withPoolingOptions(poolingOptions)
         .withProtocolVersion(ProtocolVersion.NEWEST_SUPPORTED)
 =
        .withPort(port);

On Tue, Jul 1= 2, 2016 at 11:47 AM Johnny Miller <johnny.p.miller@g= mail.com> wrote:
Abhinav - your getting that as t= he driver isn=E2=80=99t finding any hosts up for your query. You probably n= eed to check if all the nodes in your cluster are running.



Johnny

On 12 Jul 2016, at 18:46, Abhinav= Solan <abhinav.solan@gmail.com> wrote:<= /div>
Hi Everyone,

I am getting this error on our server, it c= omes and goes seems the connection drops a comes back after a while -
Caused by: com.datastax.driver.core.exception=
s.NoHostAvailableException: All host(s) tried for query failed (tried: ****=
************:9042 (com.datastax.driver.core.exceptions.ConnectionException:=
 [****************] Pool is CLOSING))
	at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandle=
r.java:218)
	at com.datastax.driver.core.RequestHandler.access$1000(RequestHandler.java=
:43)
	at com.datastax.driver.core.RequestHandler$SpeculativeExecution.sendReques=
t(RequestHandler.java:284)
	at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandle=
r.java:115)
	at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java=
:91)
	at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.jav=
a:129)
Can anyone suggest me what can be done=
 to handle this error ? 

<=
/font>
Thanks,
Abhinav=

<= /div>

<= /div>


= --001a113d64b2f5ea6605378a1c4b--