db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <kristian.waa...@oracle.com>
Subject Re: CPU Utilization
Date Wed, 16 Jan 2013 07:58:56 GMT
On 09.01.2013 20:45, Pavel Bortnovskiy wrote:
>
> Thank you, Kristian, for your quick response.
>
> How can I diagnose this and determine what Derby is doing during the 
> periods of high CPU utilization?
>

I'm not quite sure how to do this easily. Some suggestions:
  o Use jmap to dump the heap histogram.
    While easy to do, getting useful information from the histogram 
generally requires knowledge about Derby's internals. This may or may 
not result in clues about what's going on.
  o Use your favorite Java profiler/sampler to see where the processing 
time is being spent.
    If you don't have a favorite tool for this, you might get away with 
using VisualVM, NetBeans, or Eclipse. In some cases it can be useful to 
run HPROF directly.

> As far as I can determine from watching the log and the "top" 
> simultaneously, it seems that utilization peeks when data are accessed.
>
> In other words, the query itself seems to run very fast, but accessing 
> the data (with the JDBC's ResultSet) seems to peek the CPU usage.
>

What you describe above is typical, since the actual processing happens 
when you start accessing the data (i.e. typically through ResultSet.next()).
Depending on the query, it can for instance start reading and sorting 
data when you invoke next the first time. You might want to check the 
query plans of your queries, or at least inspect them visually, to 
understand if they would read all the data of a table.

As usual there are more questions than answers here:
  o how big is your dataset?
  o how big is your page cache?
  o do you have enough space on the heap? (excessive GC can cause high 
CPU utilization)
  o what's the degree of concurrency? (single vs multiple connections 
executing queries, I assume concurrent access but in theory it's 
possible to see high CPU usage otherwise too)
  o what do your queries look like?


-- 
Kristian

> P.
>
> *From:*Kristian Waagan [mailto:kristian.waagan@oracle.com]
> *Sent:* Wednesday, January 09, 2013 2:42 PM
> *To:* derby-user@db.apache.org
> *Subject:* Re: CPU Utilization
>
> On 07.01.2013 20:51, Pavel Bortnovskiy wrote:
>
> Hello:
>
> This is more of a general question. Our application uses Derby in the 
> in-memory mode only.
>
> When the application is configured to use complex queries, such 
> configuration causes CPU utilization (on the Linux server) to go as 
> high as 300% or even higher. Using simpler queries don't seem to lead 
> to such high utilization. Is there any way to control or lower CPU 
> utilization by Derby?
>
>
> Hi Pavel,
>
> I don't believe there is such a mechanism in place, at least not 
> specifically for the in-memory back end.
>
> Since this is in-memory, the latency associated with page operations 
> is very low. Do you know if Derby is using the CPU for something 
> useful (i.e. processing queries effectively), or is the CPU spent on 
> wasteful activities?
> One potential wasteful activity is moving data back and forth from the 
> page cache (page cache too small?).
> Is this a highly concurrent scenario?
>
>
> Regards,
> -- 
> Kristian
>
>
> Thank you,
> Pavel.
>
>
>
>             Jefferies archives and monitors outgoing and incoming
>             e-mail. The contents of this email, including any
>             attachments, are confidential to the ordinary user of the
>             email address to which it was addressed. If you are not
>             the addressee of this email you may not copy, forward,
>             disclose or otherwise use it or any part of it in any form
>             whatsoever. This email may be produced at the request of
>             regulators or in connection with civil litigation.
>             Jefferies accepts no liability for any errors or omissions
>             arising as a result of transmission. Use by other than
>             intended recipients is prohibited. In the United Kingdom,
>             Jefferies operates as Jefferies International Limited;
>             registered in England: no. 1978621; registered office:
>             Vintners Place, 68 Upper Thames Street, London EC4V 3BJ.
>             Jefferies International Limited is authorised and
>             regulated by the Financial Services Authority.
>
>
>
>             Jefferies archives and monitors outgoing and incoming
>             e-mail. The contents of this email, including any
>             attachments, are confidential to the ordinary user of the
>             email address to which it was addressed. If you are not
>             the addressee of this email you may not copy, forward,
>             disclose or otherwise use it or any part of it in any form
>             whatsoever. This email may be produced at the request of
>             regulators or in connection with civil litigation.
>             Jefferies accepts no liability for any errors or omissions
>             arising as a result of transmission. Use by other than
>             intended recipients is prohibited. In the United Kingdom,
>             Jefferies operates as Jefferies International Limited;
>             registered in England: no. 1978621; registered office:
>             Vintners Place, 68 Upper Thames Street, London EC4V 3BJ.
>             Jefferies International Limited is authorised and
>             regulated by the Financial Services Authority.
>

Mime
View raw message