Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 1FABA200C00 for ; Wed, 18 Jan 2017 11:43:17 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 1E192160B44; Wed, 18 Jan 2017 10:43:17 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1D170160B3A for ; Wed, 18 Jan 2017 11:43:15 +0100 (CET) Received: (qmail 4970 invoked by uid 500); 18 Jan 2017 10:43:15 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 4960 invoked by uid 99); 18 Jan 2017 10:43:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Jan 2017 10:43:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id BE95A18066C for ; Wed, 18 Jan 2017 10:43:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.002 X-Spam-Level: ** X-Spam-Status: No, score=2.002 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id vqQ5c2gy7jGL for ; Wed, 18 Jan 2017 10:43:09 +0000 (UTC) Received: from smtp2205-175.mail.aliyun.com (smtp2205-175.mail.aliyun.com [121.197.205.175]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 6AB4A5F2EF for ; Wed, 18 Jan 2017 10:43:03 +0000 (UTC) X-Alimail-AntiSpam: AC=CONTINUE;BC=0.07264779|-1;FP=0|0|0|0|0|-1|-1|-1;HT=e02c03310;MF=shawn.du@neulion.com.cn;NM=1;PH=DS;RN=1;RT=1;SR=0;TI=SMTPD_---.7Tbt3sQ_1484736171; Received: from dellPC(mailfrom:shawn.du@neulion.com.cn ip:112.65.125.42) by smtp.aliyun-inc.com(10.147.42.135); Wed, 18 Jan 2017 18:42:51 +0800 From: "Shawn Du" To: References: <022b01d27172$07ecb710$17c62530$@neulion.com.cn> <584ADA3C-BDC3-4E71-8F84-2DF17E8A3D64@gmail.com> In-Reply-To: <584ADA3C-BDC3-4E71-8F84-2DF17E8A3D64@gmail.com> Subject: =?UTF-8?Q?=E7=AD=94=E5=A4=8D:_how_to_increase_CPU_utilizat?= =?UTF-8?Q?ion_to_increase_compute_performa?= =?UTF-8?Q?nce?= Date: Wed, 18 Jan 2017 18:42:52 +0800 Message-ID: <023901d27177$9eaf2110$dc0d6330$@neulion.com.cn> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_023A_01D271BA.ACD59560" X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQKq2B914ecgRTmu6xpEpDQsPiGisQGryp1Hn3/farA= Content-Language: zh-cn archived-at: Wed, 18 Jan 2017 10:43:17 -0000 This is a multipart message in MIME format. ------=_NextPart_000_023A_01D271BA.ACD59560 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable This is the configuration. All just default. =20 localhost:47500..47509 =20 When running the job, both IO and CPU load are very low.=20 Now I am testing split less jobs. Also tune the publicThreadPoolSize. = It seems that splitting less jobs has positive effect. Tune the publicThreadPoolSize high has negative effect. Please confirm = and give suggestions. =20 Thanks Shawn =20 =E5=8F=91=E4=BB=B6=E4=BA=BA: Artem Schitow = [mailto:artem.schitow@gmail.com]=20 =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2017=E5=B9=B41=E6=9C=8818=E6=97=A5 = 18:08 =E6=94=B6=E4=BB=B6=E4=BA=BA: user@ignite.apache.org =E4=B8=BB=E9=A2=98: Re: how to increase CPU utilization to increase = compute performance =20 Hi, Shawn! =20 Can you please attach you Ignite configuration? What are your disk I/O = load and CPU load when you=E2=80=99re running the job? =E2=80=94 Artem Schitow artem.schitow@gmail.com =20 =20 =20 =20 On 18 Jan 2017, at 13:02, Shawn Du > wrote: =20 Hi,=20 =20 I have a task to compute on ignite. My Service has 8 cores. I split the = task into more than 1K jobs and merge the result. From client see, the task run more than 3 seconds, and sometimes more = than 10 seconds. The ignite server load is very slow. I wonder to know how to increase the CPU utilization to increase the = performance? =20 Thanks Shawn =20 ------=_NextPart_000_023A_01D271BA.ACD59560 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

This is the configuration. All just default.

 

<property = name=3D"discoverySpi">

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = <bean = class=3D"org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"&g= t;

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 <property = name=3D"ipFinder">

=C2=A0=C2=A0=C2=A0 = =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0<bean = class=3D"org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscover= yVmIpFinder">

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = <property name=3D"addresses">

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 <list>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = <value>localhost:47500..47509</value>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 </list>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = </property>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = </bean>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 </property>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = </bean>

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = </property>

 

When running the job, both IO and CPU load are very low. =

Now I am testing split less jobs. Also tune the = publicThreadPoolSize.=C2=A0 It seems that splitting less jobs has = positive effect.

Tune the publicThreadPoolSize high has negative effect. =C2=A0Please = confirm and give suggestions.

 

Thanks

Shawn

 

=E5=8F=91=E4=BB=B6=E4=BA=BA: Artem Schitow [mailto:artem.schitow@gmail.com] =
=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2017=E5=B9=B41=E6=9C=8818=E6=97=A5 = 18:08
=E6=94=B6=E4=BB=B6=E4=BA=BA: = user@ignite.apache.org
=E4=B8=BB=E9=A2=98: Re: how to increase CPU = utilization to increase compute = performance

 

Hi, = Shawn!

 

Can you please attach you Ignite = configuration? What are your disk I/O load and CPU load when = you=E2=80=99re running the job?


=E2=80=94

Artem Schitow

 

 

 

On 18 Jan 2017, at 13:02, Shawn Du = <shawn.du@neulion.com.cn> = wrote:

 

Hi, 

 =

I have a = task to compute on ignite. My Service has 8 cores. I split the task into = more than 1K jobs and merge the result.

From = client see, the task run more than 3 seconds, and sometimes more than 10 = seconds. The ignite server load is very slow.

I wonder = to know how to increase the CPU utilization to increase the = performance?

 =

Thanks=

Shawn<= /o:p>

 

------=_NextPart_000_023A_01D271BA.ACD59560--