Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 06165200C23 for ; Wed, 8 Feb 2017 05:22:50 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 04A72160B68; Wed, 8 Feb 2017 04:22:50 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 50C20160B3E for ; Wed, 8 Feb 2017 05:22:49 +0100 (CET) Received: (qmail 22787 invoked by uid 500); 8 Feb 2017 04:22:48 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Received: (qmail 22776 invoked by uid 99); 8 Feb 2017 04:22:48 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Feb 2017 04:22:48 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 7F8FBC0951 for ; Wed, 8 Feb 2017 04:22:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id ORNu7gCf5u4R for ; Wed, 8 Feb 2017 04:22:46 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id EF3C75F24B for ; Wed, 8 Feb 2017 04:22:45 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 110E9E026E for ; Wed, 8 Feb 2017 04:22:42 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id BDE982528C for ; Wed, 8 Feb 2017 04:22:41 +0000 (UTC) Date: Wed, 8 Feb 2017 04:22:41 +0000 (UTC) From: "Kfir Lev-Ari (JIRA)" To: dev@zookeeper.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (ZOOKEEPER-2684) Fix a crashing bug in the mixed workloads commit processor MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 08 Feb 2017 04:22:50 -0000 [ https://issues.apache.org/jira/browse/ZOOKEEPER-2684?page=3Dcom.atlas= sian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D= 15857357#comment-15857357 ]=20 Kfir Lev-Ari edited comment on ZOOKEEPER-2684 at 2/8/17 4:21 AM: ----------------------------------------------------------------- + to what Alex wrote - handle the case of < as a remote session commit.=20 I.e., something like these changes: [here|https://github.com/apache/zookeeper/blob/master/src/java/main/org/apa= che/zookeeper/server/quorum/CommitProcessor.java#L256] change the "!=3D" to= ">". and [here|https://github.com/apache/zookeeper/blob/master/src/java/main/org= /apache/zookeeper/server/quorum/CommitProcessor.java#L255] need to "peek" i= nstead of "poll". and [here|https://github.com/apache/zookeeper/blob/master/src/java/main/org= /apache/zookeeper/server/quorum/CommitProcessor.java#L271] need to add=20 "if (request.cxid =3D=3D topPending.cxid) { sessionQueue.poll();" and the rest of the local request processing up until [this|https://github.= com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/server/= quorum/CommitProcessor.java#L274] line. was (Author: kfirlevari): + to what Alex wrote - handle the case of < as a remote session commit, I.e= ., the "else" case. > Fix a crashing bug in the mixed workloads commit processor > ---------------------------------------------------------- > > Key: ZOOKEEPER-2684 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2684 > Project: ZooKeeper > Issue Type: Bug > Components: server > Affects Versions: 3.6.0 > Environment: with pretty heavy load on a real cluster > Reporter: Ryan Zhang > Assignee: Ryan Zhang > Priority: Blocker > > We deployed our build with ZOOKEEPER-2024 and it quickly started to crash= with the following error > atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:24:42,305 - ERROR [CommitPr= ocessor:2] -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitPr= ocessor.java:268) =E2=80=93 Got cxid 0x119fa expected 0x11fc5 for client se= ssion id 1009079ba470055 > atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:32:04,746 - ERROR [CommitPr= ocessor:2] -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitPr= ocessor.java:268) =E2=80=93 Got cxid 0x698 expected 0x928 for client sessio= n id 4002eeb3fd0009d > atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:34:46,648 - ERROR [CommitPr= ocessor:2] -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitPr= ocessor.java:268) =E2=80=93 Got cxid 0x8904 expected 0x8f34 for client sess= ion id 51b8905c90251 > atla-buh-05-sr1.prod.twttr.net: 2017-01-18 22:43:46,834 - ERROR [CommitPr= ocessor:2] -org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitPr= ocessor.java:268) =E2=80=93 Got cxid 0x3a8d expected 0x3ebc for client sess= ion id 2051af11af900cc > clearly something is not right in the new commit processor per session qu= eue implementation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)