Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7890A200BF6 for ; Tue, 10 Jan 2017 16:06:03 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 771FA160B29; Tue, 10 Jan 2017 15:06:03 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 75EE8160B3D for ; Tue, 10 Jan 2017 16:06:02 +0100 (CET) Received: (qmail 73619 invoked by uid 500); 10 Jan 2017 15:05:56 -0000 Mailing-List: contact user-help@mesos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@mesos.apache.org Delivered-To: mailing list user@mesos.apache.org Received: (qmail 73600 invoked by uid 99); 10 Jan 2017 15:05:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jan 2017 15:05:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 1AC61C11DC; Tue, 10 Jan 2017 15:05:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.399 X-Spam-Level: ** X-Spam-Status: No, score=2.399 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id nyjGi3sR9wTa; Tue, 10 Jan 2017 15:05:53 +0000 (UTC) Received: from mail-ua0-f172.google.com (mail-ua0-f172.google.com [209.85.217.172]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id B14525F30F; Tue, 10 Jan 2017 15:05:52 +0000 (UTC) Received: by mail-ua0-f172.google.com with SMTP id 96so64329685uaq.3; Tue, 10 Jan 2017 07:05:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xFGvBvUtg984fNSX7hq9qim/mArpLvptNYAJkjyi/E0=; b=c8uqVQZEKb8DnWrs0Yd0r2cWVeKk3TVK2cAYwtChJRkYm3ykdCIlRmsI458Ymmm88S i+hrFAeGUomstWv4RYpaeQ/fGqRUJpmu2t1dnMOJQEyo11kdUrUw5cSnDZGWni1FJwhy yeiPdQ6y6vK44Bqzv+t6b4GE9nDMLyP+Hk57ifME3oQZJYVpokkXwXOgCeXASOhvYuqL 8XcJjRCMAZaRpwmNGWmoT5H8zTlow+Lnq6KaRQqJ0ElRrNixfuphbWMtFTN4seHoSP1r xjf/sUJZZKlOpTf/A5XwgfwAyYItbVHImWAhjN8Z37Q2dLqgpODZ2QYJh6hqA65qQ+3U UnBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xFGvBvUtg984fNSX7hq9qim/mArpLvptNYAJkjyi/E0=; b=eZMom4eeMjg50ynsSbm8jrwEcQq4YBKP83/CRI0XHbeXhssTSmWG/luVw7C2T/XWcQ JlRLdK/BbK/A2ENGqFhz55imxPwr5qhNHBhtrIt2gSDq5uQt9r7ZnPGwTwDy2ydG9KnZ /Y3GPEfecp9cyLW8lh2nYtdEvlYKJHZuJzjjvO63/Xhs8M+pbOPaZDfJAfN1mrsUoaaJ VV5fpeBHNe80WYROFPqQmMDb8bYQLLW7T2AOkkl7AAmTXV0Evf0zPuZjCvq6RHbNbfrF rLfKNm4kdBrUDtrZ6yFxwlYbo5XV3m71jpN7paAxPkq2LDVqWd6/kYpknuFrsZSu4iNw TArw== X-Gm-Message-State: AIkVDXKjFAe6OsUWW+oCjaxk2x8cb1eaFPBZKjF0xAYhTlOFZF0bdwmDxR+HGVV3qngzy0Sl8b5uQzdM/6Fz8w== X-Received: by 10.159.38.73 with SMTP id 67mr1797782uag.155.1484060746031; Tue, 10 Jan 2017 07:05:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.89.196 with HTTP; Tue, 10 Jan 2017 07:05:45 -0800 (PST) In-Reply-To: References: <7A6B773BF2F721479D668347BF48E8A7575714C9@SZXEMI501-MBS.china.huawei.com> <7A6B773BF2F721479D668347BF48E8A757571608@SZXEMI501-MBS.china.huawei.com> From: tommy xiao Date: Tue, 10 Jan 2017 23:05:45 +0800 Message-ID: Subject: =?UTF-8?Q?Re=3A_=E7=AD=94=E5=A4=8D=3A_Optimize_libprocess_performance?= To: dev Cc: "user@mesos.apache.org" Content-Type: multipart/alternative; boundary=001a114950dc97a7050545bed185 archived-at: Tue, 10 Jan 2017 15:06:03 -0000 --001a114950dc97a7050545bed185 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Looking forward to seeing the benchmarks. 2017-01-06 5:52 GMT+08:00 Benjamin Mahler : > Ok great, thanks for sharing. Looking forward to seeing the benchmarks. > > On Wed, Jan 4, 2017 at 10:31 PM, pangbingqiang > wrote: > > > We write a light k-v database ,use for metadata store and nameservice > like > > etcd, but we test its TPS just 1200+(one client), the network is not th= e > > bottleneck, so the RPC layer is too heavy. > > > > -----=E9=82=AE=E4=BB=B6=E5=8E=9F=E4=BB=B6----- > > =E5=8F=91=E4=BB=B6=E4=BA=BA: Benjamin Mahler [mailto:bmahler@apache.org= ] > > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2017=E5=B9=B41=E6=9C=885=E6=97=A5= 9:26 > > =E6=94=B6=E4=BB=B6=E4=BA=BA: dev > > =E6=8A=84=E9=80=81: user@mesos.apache.org > > =E4=B8=BB=E9=A2=98: Re: Optimize libprocess performance > > > > Which areas does the performance not meet your needs? There are a lot o= f > > aspects to libprocess that can be optimized, so it would be good to foc= us > > on each of your particular use cases via benchmarks, this allows us to > have > > a shared way to profile and measure improvements. > > > > Copy elimination is one area where a lot of improvement can be made > across > > libprocess, note that libprocess was implemented before we had C++11 mo= ve > > support available. We've recently made some improvements to update the > HTTP > > serving path towards zero-copies but it's not completely done. Can you > > submit patches for the ProcessBase::send() path copy elimination? We ca= n > > have a move overload for ProcessBase::send and have > ProtobufProcess::send() > > and encode() perform moves instead of a copy. > > > > With respect to the MessageEncoder, since it's less trivial, you can > > submit a benchmark that captures the use case you care about and we can > > drive improvements using it. I have some suggestions here as well but w= e > > can discuss once we have the benchmarks committed. > > > > How does that sound to start? > > > > On Tue, Jan 3, 2017 at 7:31 PM, pangbingqiang > > wrote: > > > > > Hi All: > > > > > > We use libprocess as our underlying communication library, but we > > > find it=E2=80=99s performance don=E2=80=99t meet, we want to optimize= it, for example: > > > > > > * =E2=80=98send=E2=80=99 function *implementation one metadata has f= our times memory > > > copy, > > > > > > *1. ProtobufMessage SerializeToString then processbase =E2=80=98encod= e=E2=80=99 > > > construct string once;* > > > > > > *2. In =E2=80=98encode=E2=80=99 function Message body copy again;* > > > > > > *3. In MessageEncoder in order to construct HTTP Request, copy again;= * > > > > > > *4. **MessageEncoder return copy again;* > > > > > > How to optimize this scenario may be useful. > > > > > > Also , in libprocess it has so many lock: > > > > > > *1. **SocketManager: std::recursive_mutex mutex;* > > > > > > *2. **ProcessManager: std::recursive_mutex processes_mutex;* > > *std::recursive_mutex > > > runq_mutex; std::recursive_mutex firewall_mutex;* > > > > > > In particular, everytime event enqueue/dequeue both need to get lock, > > > maybe use lookfree struct is better. > > > > > > > > > > > > If have any optimize suggestion or discussion, please let me know, > > thanks. > > > > > > > > > > > > [image: cid:image001.png@01D0E8C5.8D08F440] > > > > > > > > > > > > Bingqiang Pang(=E5=BA=9E=E5=85=B5=E5=BC=BA) > > > > > > > > > > > > Distributed and Parallel Software Lab > > > > > > Huawei Technologies Co., Ltd. > > > > > > Email:pangbingqiang@huawei.com > > > > > > > > > > > > > > > > > > --=20 Deshi Xiao Twitter: xds2000 E-mail: xiaods(AT)gmail.com --001a114950dc97a7050545bed185 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Looking forward to seeing t= he benchmarks.

2017-01-06 5:52 GMT+08:00 Benjamin Mahler &= lt;bmahler@apache.o= rg>:
Ok great, thanks for s= haring. Looking forward to seeing the benchmarks.

On Wed, Jan 4, 2017 at 10:31 PM, pangbingqiang <pangbingqiang@huawei.com>
wrote:

> We write a light k-v database ,use for metadata store and nameservice = like
> etcd, but we test its TPS just 1200+(one client), the network is not t= he
> bottleneck, so the RPC layer is too heavy.
>
> -----=E9=82=AE=E4=BB=B6=E5=8E=9F=E4=BB=B6-----
> =E5=8F=91=E4=BB=B6=E4=BA=BA: Benjamin Mahler [mailto:bmahler@apache.org]
> =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2017=E5=B9=B41=E6=9C=885=E6=97= =A5 9:26
> =E6=94=B6=E4=BB=B6=E4=BA=BA: dev
> =E6=8A=84=E9=80=81: user@meso= s.apache.org
> =E4=B8=BB=E9=A2=98: Re: Optimize libprocess performance
>
> Which areas does the performance not meet your needs? There are a lot = of
> aspects to libprocess that can be optimized, so it would be good to fo= cus
> on each of your particular use cases via benchmarks, this allows us to= have
> a shared way to profile and measure improvements.
>
> Copy elimination is one area where a lot of improvement can be made ac= ross
> libprocess, note that libprocess was implemented before we had C++11 m= ove
> support available. We've recently made some improvements to update= the HTTP
> serving path towards zero-copies but it's not completely done. Can= you
> submit patches for the ProcessBase::send() path copy elimination? We c= an
> have a move overload for ProcessBase::send and have ProtobufProcess::s= end()
> and encode() perform moves instead of a copy.
>
> With respect to the MessageEncoder, since it's less trivial, you c= an
> submit a benchmark that captures the use case you care about and we ca= n
> drive improvements using it. I have some suggestions here as well but = we
> can discuss once we have the benchmarks committed.
>
> How does that sound to start?
>
> On Tue, Jan 3, 2017 at 7:31 PM, pangbingqiang <pangbingqiang@huawei.com>
> wrote:
>
> > Hi All:
> >
> >=C2=A0 =C2=A0We use libprocess as our underlying communication lib= rary, but we
> > find it=E2=80=99s performance don=E2=80=99t meet, we want to opti= mize it, for example:
> >
> > *=C2=A0 =E2=80=98send=E2=80=99 function *implementation one metad= ata has four times memory
> > copy,
> >
> > *1. ProtobufMessage SerializeToString then processbase =E2=80=98e= ncode=E2=80=99
> > construct string once;*
> >
> > *2. In =E2=80=98encode=E2=80=99 function Message body copy again;= *
> >
> > *3. In MessageEncoder in order to construct HTTP Request, copy ag= ain;*
> >
> > *4.=C2=A0 =C2=A0 =C2=A0 =C2=A0**MessageEncoder return copy again;= *
> >
> >=C2=A0 =C2=A0How to optimize this scenario may be useful.
> >
> >=C2=A0 =C2=A0Also , in libprocess it has so many lock:
> >
> > *1.=C2=A0 =C2=A0 =C2=A0 =C2=A0**SocketManager:=C2=A0 =C2=A0std::r= ecursive_mutex mutex;*
> >
> > *2.=C2=A0 =C2=A0 =C2=A0 =C2=A0**ProcessManager:=C2=A0 std::recurs= ive_mutex processes_mutex;*
> *std::recursive_mutex
> > runq_mutex; std::recursive_mutex firewall_mutex;*
> >
> > In particular, everytime event enqueue/dequeue both need to get l= ock,
> > maybe use lookfree struct is better.
> >
> >
> >
> > If have any optimize suggestion or discussion, please let me know= ,
> thanks.
> >
> >
> >
> > [image: cid:image001.png@01D0E8C5.8D08F440]
> >
> >
> >
> > Bingqiang Pang(=E5=BA=9E=E5=85=B5=E5=BC=BA)
> >
> >
> >
> > Distributed and Parallel Software Lab
> >
> > Huawei Technologies Co., Ltd.
> >
> > Email:pangbin= gqiang@huawei.com <suteng@huawe= i.com>
> >
> >
> >
> >
> >
>



--
=
Deshi Xia= o
Twitter: xds2000
E-mail: xiaods(AT)gmail.com
--001a114950dc97a7050545bed185--