Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
MIME-Version: 1.0
In-Reply-To: <5E89C64C-E616-4061-9C67-290424944028@canoga.com>
References: <97EB699F861AD841B5908C7CA9C9565604A103BDD587@VSERVER1.canoga.com>
 <CANi-yg-5Kf=aUn6HgvC80CVSCLEZEuhTKW4mhz9dabyM-AxjYQ@mail.gmail.com> <5E89C64C-E616-4061-9C67-290424944028@canoga.com>
From: Bryan Pendleton <bpendleton.derby@gmail.com>
Date: Wed, 7 Jun 2017 06:46:40 -0700
Message-ID: <CANi-yg9Wj2SX348KR5Bm3hvyWiR2m99huMQbOZp+O-mz6u1tDQ@mail.gmail.com>
Subject: Re: Could use some thoughts on a problem with a query in production
To: derby-dev@db.apache.org
Content-Type: multipart/alternative; boundary="001a114a9b084a138105515ef70f"
archived-at: Wed, 07 Jun 2017 13:46:48 -0000

--001a114a9b084a138105515ef70f
Content-Type: text/plain; charset="UTF-8"

Hi Brett,

Cache cleaning in general is a very I/O intensive activity, so I agree that
it is odd that your system appeared to be CPU busy during that time.

It's interesting that whatever it was, persisted for such a long time.

In the past, I have developed small operator scripts which collect system
information, which can be run by a simple monitoring tool and can keep a
history about the observations they make.

If you were to build and install such a monitoring system, you might be
able to develop more clues about the "larger picture" of activity on your
machine during that time period.

Sorry to be so vague and non-specific, but I find that retrospective
analysis of problems like this is VERY challenging.

Often, the best you can do is:
1) Try to figure out some way to make the problem happen on demand, so you
can cause it and observe it.
2) Instrument everything, and ensure you are preserving a history of your
instrumentation recordings, so that you can have a mountain of detail when
those rare events occur.

It strikes me as being like high-energy physics, where your experiment
generates volumes of data, and it takes weeks or months of analysis
afterward to figure out what actually occurred.

Not that I'm much good at high-energy physics, either, I'm afraid. :)

bryan

--001a114a9b084a138105515ef70f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra">Hi Brett,</div><div class=3D"gm=
ail_extra"><br></div><div class=3D"gmail_extra">Cache cleaning in general i=
s a very I/O intensive activity, so I agree that it is odd that your system=
 appeared to be CPU busy during that time.</div><div class=3D"gmail_extra">=
<br></div><div class=3D"gmail_extra">It&#39;s interesting that whatever it =
was, persisted for such a long time.</div><div class=3D"gmail_extra"><br></=
div><div class=3D"gmail_extra">In the past, I have developed small operator=
 scripts which collect system information, which can be run by a simple mon=
itoring tool and can keep a history about the observations they make.</div>=
<div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">If you were=
 to build and install such a monitoring system, you might be able to develo=
p more clues about the &quot;larger picture&quot; of activity on your machi=
ne during that time period.</div><div class=3D"gmail_extra"><br></div><div =
class=3D"gmail_extra">Sorry to be so vague and non-specific, but I find tha=
t retrospective analysis of problems like this is VERY challenging.</div><d=
iv class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Often, the be=
st you can do is:</div><div class=3D"gmail_extra">1) Try to figure out some=
 way to make the problem happen on demand, so you can cause it and observe =
it.</div><div class=3D"gmail_extra">2) Instrument everything, and ensure yo=
u are preserving a history of your instrumentation recordings, so that you =
can have a mountain of detail when those rare events occur.</div><div class=
=3D"gmail_extra"><br></div><div class=3D"gmail_extra">It strikes me as bein=
g like high-energy physics, where your experiment generates volumes of data=
, and it takes weeks or months of analysis afterward to figure out what act=
ually occurred.</div><div class=3D"gmail_extra"><br></div><div class=3D"gma=
il_extra">Not that I&#39;m much good at high-energy physics, either, I&#39;=
m afraid. :)</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_=
extra">bryan</div><div class=3D"gmail_extra"><br></div></div>

--001a114a9b084a138105515ef70f--