From dev-return-39500-archive-asf-public=cust-asf.ponee.io@subversion.apache.org Mon Oct 7 20:56:10 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1E5B51804BB for ; Mon, 7 Oct 2019 22:56:10 +0200 (CEST) Received: (qmail 67051 invoked by uid 500); 7 Oct 2019 20:56:09 -0000 Mailing-List: contact dev-help@subversion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list dev@subversion.apache.org Received: (qmail 66966 invoked by uid 99); 7 Oct 2019 20:56:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Oct 2019 20:56:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 36E0EC25F3 for ; Mon, 7 Oct 2019 20:56:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.809 X-Spam-Level: * X-Spam-Status: No, score=1.809 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=wandisco.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id SoVkwPzRm8cP for ; Mon, 7 Oct 2019 20:56:05 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.222.48; helo=mail-ua1-f48.google.com; envelope-from=doug.robinson@wandisco.com; receiver= Received: from mail-ua1-f48.google.com (mail-ua1-f48.google.com [209.85.222.48]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id CDC62BC9D7 for ; Mon, 7 Oct 2019 20:56:04 +0000 (UTC) Received: by mail-ua1-f48.google.com with SMTP id j5so4512110uak.12 for ; Mon, 07 Oct 2019 13:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wandisco.com; s=gapps; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8c0QeZx65dmr/a4gfQISkiuuHr+RcCeZ/R+wkqtVRII=; b=gSPY8UDTtyowpkzaqaC9l2WDgLg4X4upiTk2mPHLXJKCWHe412oDpyuM7LzYY2LxLF UcfpWKvj1delhjKywE4fj7GX6auHWQRtDDeZpdncaJI+X+piTIz1uo9mKUT650IWEL1u eJGXAxNZsKjUIAGlWBncSyXeJsJ+sI3sdtk4k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8c0QeZx65dmr/a4gfQISkiuuHr+RcCeZ/R+wkqtVRII=; b=LUR947OEoiCmOJDxT3mcMMzQkInvicoDuPq14Lw8DPnTRFtGuOCVax2rYTPTW3Dg5W 7u40jV/MTBYybJsYo5J5tJhVRs7sJHrUG+IprIjCaOQzKyTI2yU5GsYmxaXOyOxSbXFt Q1JV3m5P4F2YX7t9rAvJJ8tHSYRVxXJBc4TuNt56p14E8Yo1KZ4pB17KprUFDTuidaHA giWX0cpZAwUZ94ItboEwz00Fmxz3C17OjGZK3ewZYEw30wN0hXWbrQQVCYg8vQl2yttP bPWCRR+CtBckbAiM0SG+SkQ+tb/6Fo93w4poYIeXkPcOaDE9lacxwsUfVq//Bma/eb5i /l7Q== X-Gm-Message-State: APjAAAXwbTh19KKfP30r0P6YXInPe+5iT94A+YxgWFwt3Hg4VsAS5v44 /kmUJjaEqkZP93WaB7cspgLxydYDuWDBM9HZgWBD1FYeQOc7Jcma8+MlUGdNe7puZZsfypHQKNF Mdb1rg82rBFZMhH3u7UXgX1Tjwv3h X-Google-Smtp-Source: APXvYqzlqcYMr9o1m4gRxjwX6L1OTCoIyBl7OlZTbAFQOYq9laNQ6qfNIGZcyhO5pz3j7YmUe6HL0jt/WJtqAXBHGVY= X-Received: by 2002:a9f:2404:: with SMTP id 4mr3025765uaq.60.1570481763996; Mon, 07 Oct 2019 13:56:03 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Doug Robinson Date: Mon, 7 Oct 2019 16:55:52 -0400 Message-ID: Subject: Re: Better choice for Linux semaphore than spinlock? To: Ruediger Pluem Cc: =?UTF-8?Q?Branko_=C4=8Cibej?= , dev@subversion.apache.org, dev@apr.apache.org Content-Type: multipart/alternative; boundary="000000000000ab6c1d059458470e" --000000000000ab6c1d059458470e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable R=C3=BCdiger: On Mon, Oct 7, 2019 at 3:51 PM Ruediger Pluem wrote: > On 10/07/2019 08:40 PM, Branko =C4=8Cibej wrote: > > On Mon, 7 Oct 2019, 19:47 Doug Robinson, > wrote: > > > > Folks: > > > > I spoke with this user late last week. They stated that they can > only get approximately 400 parallel SVN operations > > before the "system time" consumes all available CPU for an 8-core > machine. Adding more cores won't help because of > > the nature of spin locks (it makes things worse). Turns out that > even with ~100 parallel SVN operations the "system > > time" starts becoming significant/measurable (~10%). Both HTTP > (mod_dav_svn) and "svnserve" protocols participate > > in the lock contention. > > > > Your help would be greatly appreciated. > > > > Whew. So. Reducing this issue to "use a more efficient lock" is not > going to work, and you provided far too little > > information to even attempt a diagnosis. For starters, I recommend > gathering as much info as possible (anonymised of > > course) about the server configuration, everything from httpd an > svnserve to the repository config and underlying > > filesystem, if possible. Getting stack traces of the "stuck" threads > would be necessary, too. Without knowing exactly > > what is happening, these kinds of problems are extremely hard to > understand, let alone fix. > > Plus depending on which part of the code requires this lock a different > locking mechanism that might suit better for > this use case can possibly be chosen via configuration changes (e.g. http= d > allows this for most of its locking). > That would be awesome! I'll definitely try to get those stack tracebacks. Cheers. Doug --=20 *DOUGLAS B ROBINSON* SENIOR PRODUCT MANAGER T +1 925 396 1125 *E* doug.robinson@wandisco.com --=20 * * **The=C2=A0*LiveData*=C2=A0Company *Find out more=C2=A0 *wandisco.com * =20 THIS MESSAGE AND ANY ATTACHMENTS=20 ARE CONFIDENTIAL, PROPRIETARY AND MAY BE PRIVILEGED * If this message was=20 misdirected, WANdisco, Inc. and its subsidiaries, ("WANdisco") does not=20 waive any confidentiality or privilege. If you are not the intended=20 recipient, please notify us immediately and destroy the message without=20 disclosing its contents to anyone. Any distribution, use or copying of this= =20 email or the information it contains by other than an intended recipient is= =20 unauthorized. The views and opinions expressed in this email message are=20 the author's own and may not reflect the views and opinions of WANdisco,=20 unless the author is authorized by WANdisco to express such views or=20 opinions on its behalf. All email sent to or from this address is subject= =20 to electronic storage and review by WANdisco. Although WANdisco operates=20 anti-virus programs, it does not accept responsibility for any damage=20 whatsoever caused by viruses being passed. --000000000000ab6c1d059458470e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
R=C3=BCdiger:

On Mon, Oct 7, 2019 at 3= :51 PM Ruediger Pluem <rpluem@apach= e.org> wrote:
On 10/07/2019 08:40 PM, Branko =C4=8Cibej wrote:
> On Mon, 7 Oct 2019, 19:47 Doug Robinson, <doug.robinson@wandisco.com <m= ailto:doug.= robinson@wandisco.com>> wrote:
>
>=C2=A0 =C2=A0 =C2=A0Folks:
>
>=C2=A0 =C2=A0 =C2=A0I spoke with this user late last week.=C2=A0 They s= tated that they can only get approximately 400 parallel SVN operations
>=C2=A0 =C2=A0 =C2=A0before the "system time" consumes all ava= ilable CPU for an 8-core machine.=C2=A0 Adding more cores won't help be= cause of
>=C2=A0 =C2=A0 =C2=A0the nature of spin locks (it makes things worse).= =C2=A0 Turns out that even with ~100 parallel SVN operations the "syst= em
>=C2=A0 =C2=A0 =C2=A0time" starts becoming significant/measurable (= ~10%).=C2=A0 Both HTTP (mod_dav_svn) and "svnserve" protocols par= ticipate
>=C2=A0 =C2=A0 =C2=A0in the lock contention.
>
>=C2=A0 =C2=A0 =C2=A0Your help would be greatly appreciated.
>
> Whew. So. Reducing this issue to "use a more efficient l= ock" is not going to work, and you provided far too little
> information to even attempt a diagnosis. For starters, I recommend gat= hering as much info as possible (anonymised of
> course) about the server configuration, everything from httpd an svnse= rve to the repository config and underlying
> filesystem, if possible. Getting stack traces of the "stuck"= threads would be necessary, too. Without knowing exactly
> what is happening, these kinds of problems are extremely hard to under= stand, let alone fix.

Plus depending on which part of the code requires this lock a different loc= king mechanism that might suit better for
this use case can possibly be chosen via configuration changes (e.g. httpd = allows this for most of its locking).

T= hat would be awesome!=C2=A0 I'll definitely try to get those stack trac= ebacks.

Cheers.

Doug
--
<= div>
DOUGLAS B ROBINS= ON=C2=A0SENI= OR PRODUCT MANAGER

The=C2=A0LiveData=C2=A0Company
Find out more= =C2=A0wandisco.com


THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PRO= PRIETARY AND MAY BE PRIVILEGED

If this message was misdirected, WANdisco, In= c. and its subsidiaries, ("WANdisco") does not waive any confiden= tiality or privilege. If you are not the intended recipient, please notify = us immediately and destroy the message without disclosing its contents to a= nyone. Any distribution, use or copying of this email or the information it= contains by other than an intended recipient is unauthorized. The views an= d opinions expressed in this email message are the author's own and may= not reflect the views and opinions of WANdisco, unless the author is autho= rized by WANdisco to express such views or opinions on its behalf. All emai= l sent to or from this address is subject to electronic storage and review = by WANdisco. Although WANdisco operates anti-virus programs, it does not ac= cept responsibility for any damage whatsoever caused by viruses being passe= d.

--000000000000ab6c1d059458470e--