Mailing-List: contact dev-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@aurora.apache.org
From: Shyam Patel <sham.patel04@gmail.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_D2CC70A0-3544-496C-B75F-21E3739C27E5"
Subject: Aurora performance impact with hourly query runs
Message-Id: <97CCCE93-4A9D-4C26-840B-7207D9EA1718@gmail.com>
Date: Wed, 8 Jun 2016 16:34:17 -0700
To: dev@aurora.apache.org
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
archived-at: Wed, 08 Jun 2016 23:34:06 -0000

--Apple-Mail=_D2CC70A0-3544-496C-B75F-21E3739C27E5
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8

Hi,

While running LnP testing, I=E2=80=99m spinning of 8K docker jobs. =
During the run, I ran into issue where TaskStatUpdate and TaskReconciler =
queries taking real long times. During the time, Aurora is pretty much =
freezing and at a point dying.  Also, tried the same run w/o the docker =
jobs and faced the same issue.=20


Is there a way to keep the Aurora performance intact during the query =
runs ?


Here is snipped from log :


I0602 00:53:37.527 [TaskStatUpdaterService RUNNING, DbTaskStore:104] =
Query took 1243517 ms: TaskQuery(owner:null, role:null, =
environment:null, jobName:null, taskIds:null, statuses:[STARTING, =
THROTTLED, RUNNING, DRAINING, ASSIGNED, KILLING, RESTARTING, PENDING, =
PREEMPTING], instanceIds:null, slaveHosts:null, jobKeys:null, offset:0, =
limit:0)=20


I0602 00:56:54.180 [TaskReconciler-0, DbTaskStore:104] Query took =
1380169 ms: TaskQuery(owner:null, role:null, environment:null, =
jobName:null, taskIds:null, statuses:[STARTING, RUNNING, DRAINING, =
ASSIGNED, KILLING, RESTARTING, PREEMPTING], instanceIds:null, =
slaveHosts:null, jobKeys:null, offset:0, limit:0)


Appreciate any insights..


Thanks,
Sham


--Apple-Mail=_D2CC70A0-3544-496C-B75F-21E3739C27E5--