Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 04EE0200D3B for ; Fri, 10 Nov 2017 21:21:36 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 03516160BEE; Fri, 10 Nov 2017 20:21:36 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 23955160BE0 for ; Fri, 10 Nov 2017 21:21:34 +0100 (CET) Received: (qmail 89388 invoked by uid 500); 10 Nov 2017 20:21:33 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 89377 invoked by uid 99); 10 Nov 2017 20:21:33 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Nov 2017 20:21:33 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B3EA11A02B2 for ; Fri, 10 Nov 2017 20:21:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.979 X-Spam-Level: * X-Spam-Status: No, score=1.979 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=instaclustr-com.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id S6WwpF9Fw6wX for ; Fri, 10 Nov 2017 20:21:30 +0000 (UTC) Received: from mail-yw0-f179.google.com (mail-yw0-f179.google.com [209.85.161.179]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 0D3665FD7E for ; Fri, 10 Nov 2017 20:21:29 +0000 (UTC) Received: by mail-yw0-f179.google.com with SMTP id d2so4567355ywb.11 for ; Fri, 10 Nov 2017 12:21:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=instaclustr-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=t6RtB/UBWO8YP82x7VQAUs5ErL4LfQmMZTQ81Dsl68s=; b=XVk4mfc6vYcN+/Yn7AhnH5w7Y6xDpCc2nl/sOFY2GGXwhUJLBTgwuw7TLn512wvQ4k ZvZzMBKeB/7iAKh+IT4eW+moMMocAXSicTltQpNhhPvpt4I3AOniSY2wwWs+aUVyj3N8 tmo5NP/OoQQi7K84hGjRPOlgJb01anIflngwqXZSLeuOGeu0vh4EIwGZE5qe/AeII7r8 2xuQKbQ565cMYw8j8wCUHeJOsi16eR1VlStOSzMi5XOp9LtamLUerSdML4pgzlHP/USL ak2dosBf+CDkQ3y0VFcJAWulQX18DQHFaHVIg1ckVPmdvFmTPuDD0yQcIaDHDFmaiVIQ fiug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=t6RtB/UBWO8YP82x7VQAUs5ErL4LfQmMZTQ81Dsl68s=; b=QL9WUHdFAoPGyP4CxGdYKVsjSJYA6ECkFlYchciXQTW5W76sqV88JBevRAXZ0LeCYi vuJcVr8bCwuIAbkfW51fgcsm+CIujDg0kHI06C7qbi9XgxSnMVYuHXHEhkyExJuxxAV3 qNtVi2azhMthgdzEkHQ2/nZQGlRxV+8dhzA16qgqAZft6kbeVZZvUZ+Sh1QnsfLGQGap 2J0rqK8ybxwGRfouN/POcDz/3LW5BAtYQbtQy+hadyTbu21UHcZBxMfEnTshT1E3rd0i tsVOdi85Wh6ch1BJeiZ4E2MpvtzgXMK9CcQN7zj8P+seZbb2WVmd03fzzdPygKcu6gyI 9o6Q== X-Gm-Message-State: AJaThX72MLyqT380yiUY+TU06xbxbxGG9ED6VKdbcZXBou46tqmudOdb P6kCv+ryAVorJZ014dbshXr7YhrdOXvmyD6fSA2NaWIQ X-Google-Smtp-Source: AGs4zMY5idNBfxqQXjkPho8CnKUAMghY0UpDyndW+ksVxJ+iU2x3oPmIVKNmEvYt+H7KyEM+e0dN7yxNg9qFEQamJUE= X-Received: by 10.37.75.68 with SMTP id y65mr1165459yba.211.1510345282835; Fri, 10 Nov 2017 12:21:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.13.242.132 with HTTP; Fri, 10 Nov 2017 12:21:22 -0800 (PST) Received: by 10.13.242.132 with HTTP; Fri, 10 Nov 2017 12:21:22 -0800 (PST) In-Reply-To: References: From: kurt greaves Date: Fri, 10 Nov 2017 20:21:22 +0000 Message-ID: Subject: Re: Cassandra open port native_transport_port after a random time To: User Content-Type: multipart/alternative; boundary="94eb2c0942b81270bf055da6aab8" archived-at: Fri, 10 Nov 2017 20:21:36 -0000 --94eb2c0942b81270bf055da6aab8 Content-Type: text/plain; charset="UTF-8" Can you upload some logs from startup? Obviously getting stuck starting but it'd be nice to know where it got up to. On 11 Nov. 2017 04:49, "Octavian Rinciog" wrote: > Hello! > > I'm running a cluster of 10 Cassandra 3.10 and I saw a very strange > behavior: after restart, a node won't open immediately > native_transport_port (9042). > > After one node restart, the flow is : > > - node finishes to read all commitlog, > - update all its data, > - it's visible for other nodes in the cluster, > - wait for random time (from 1 minute to hours) to open 9042 port > > My logs are in DEBUG mode, and nothing is written about opening this > port. What is happening and how can I debug this problem? > > Output for several nodetool commands are: > > - nodetool enablebinary: does not return at all (it just hangs) > - nodetool compactionstats: 0 pending tasks > - nodetool netstats: Mode: STARTING. Not sending any streams. > - nodetool info: Gossip active : true Thrift active : false Native > Transport active: false > > Thank you. > > > > -- > Octavian Rinciog > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org > For additional commands, e-mail: user-help@cassandra.apache.org > > --94eb2c0942b81270bf055da6aab8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Can you upload some logs from startup? Obviously getting = stuck starting but it'd be nice to know where it got up to.

On 11 Nov. 2017 04:49, = "Octavian Rinciog" <octavian.rinciog@gmail.com> wrote:
Hello!

I'm running a cluster of 10 Cassandra 3.10 and I saw a very strange
behavior: after restart, a node won't open immediately
native_transport_port (9042).

After one node restart, the flow is :

- node finishes to read all commitlog,
- update all its data,
- it's visible for other nodes in the cluster,
- wait for random time (from 1 minute to hours) to open 9042 port

My logs are in DEBUG mode, and nothing is written about opening this
port. What is happening and how can I debug this problem?

Output for several nodetool commands are:

- nodetool enablebinary: does not return at all (it just hangs)
- nodetool compactionstats: 0 pending tasks
- nodetool netstats: Mode: STARTING. Not sending any streams.
- nodetool info: Gossip active : true Thrift active : false Native
Transport active: false

Thank you.



--
Octavian Rinciog

-----------------------------------------------------------------= ----
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org

--94eb2c0942b81270bf055da6aab8--