Return-Path: Delivered-To: apmail-db-derby-user-archive@www.apache.org Received: (qmail 20985 invoked from network); 18 Feb 2010 17:56:48 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 18 Feb 2010 17:56:48 -0000 Received: (qmail 40251 invoked by uid 500); 18 Feb 2010 17:56:48 -0000 Delivered-To: apmail-db-derby-user-archive@db.apache.org Received: (qmail 40226 invoked by uid 500); 18 Feb 2010 17:56:48 -0000 Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Reply-To: "Derby Discussion" Delivered-To: mailing list derby-user@db.apache.org Received: (qmail 40218 invoked by uid 99); 18 Feb 2010 17:56:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Feb 2010 17:56:48 +0000 X-ASF-Spam-Status: No, hits=-1.8 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [216.82.254.195] (HELO mail200.messagelabs.com) (216.82.254.195) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Feb 2010 17:56:39 +0000 X-VirusChecked: Checked X-Env-Sender: PBortnovskiy@Jefferies.com X-Msg-Ref: server-9.tower-200.messagelabs.com!1266515777!46749893!1 X-StarScan-Version: 6.2.4; banners=-,-,- X-Originating-IP: [169.196.176.23] Received: (qmail 21742 invoked from network); 18 Feb 2010 17:56:18 -0000 Received: from jefsmtp11.jefferies.com (HELO JEFSMTP11.jefferies.com) (169.196.176.23) by server-9.tower-200.messagelabs.com with RC4-SHA encrypted SMTP; 18 Feb 2010 17:56:18 -0000 In-Reply-To: To: "Derby Discussion" Cc: Kristian.Waagan@Sun.COM Subject: Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby MIME-Version: 1.0 X-Mailer: Lotus Notes Release 7.0.3 September 26, 2007 From: Pavel Bortnovskiy Message-ID: Date: Thu, 18 Feb 2010 12:56:13 -0500 X-MIMETrack: Serialize by Router on JefSMTP11/JEFCO(Release 7.0.3|September 26, 2007) at 02/18/2010 12:56:18 PM, Serialize complete at 02/18/2010 12:56:18 PM Content-Type: multipart/alternative; boundary="=_alternative 0062881D852576CE_=" This is a multipart message in MIME format. --=_alternative 0062881D852576CE_= Content-Type: text/plain; charset="US-ASCII" Thank you, Kristian and Bryan for all your help. I did locate the setting which allows me to suspend on the thread which hit the breakpoint while allowing others to execute. But it was quite an interesting exercise, nevertheless. Pavel Bortnovskiy/JEFCO 02/18/2010 12:47 PM To "Derby Discussion" cc Kristian.Waagan@Sun.COM Subject Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby Hello, Kristian: Thanks for your response. I think I found what the problem is... When my IDE (IntelliJ IDEA) hits a breakpoint, it suspends all threads. I find it funny that in all these years of programming and using IDEA, I've never realized it. (possibly because I was too occupied debugging the code at the breakpoint and not thinking about other threads at that moment). I've searched through the options but couldn't find any setting which will prevent other threads from being suspended. Will continue looking. I am now curious, what IDE do you use that it worked for you? And did you need to enable any special settings to allow all other threads to continue execution? Regards, Pavel. Kristian Waagan Sent by: Kristian.Waagan@Sun.COM 02/16/2010 03:32 AM Please respond to "Derby Discussion" To Derby Discussion cc Subject Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby On 16.02.10 03:02, Pavel Bortnovskiy wrote: > Bryan: thank you for your response. > > I do see the thread you mention: > > Thread Group [derby.daemons]{10}(Daemon) > Thread [derby.antiGC]{1}(Daemon)(WAITING) > Thread [derby.rawStoreDaemon]{5}(Daemon)(TIMED_WAITING) > Thread [derby.NetworkServerStarter]{5}(Daemon)(WAITING) > Thread [NetworkServerThread_2]{5}(Daemon)(RUNNABLE) > Thread [DRDAConnThread_3]{5}(Daemon)(WAITING) > > However the behavior is still the same - if any other thread stops (for > instance in debugger), NetworkServerControl becomes unresponsive. > For instance, if I connect to it from the "outside" with another db app > (Aqua Data Studio, for instance), then connection to it breaks or the db > app can't access it. > But as soon as I let the thread run in that debugger, the connection > becomes alive and everything seems to work. > > Has anyone else seen this kind of behavior? > Or perhaps, someone can share a piece of code (best practice) of how to > instantiate and run the NetworkServerControl, so that it's not > experiencing > such hang-ups. > Hi Pavel, The code I'm about to post is by no means to be considered best practice, but I think it does what you want it to do. However, the code is basically yours :) (I copied it from your mail) When I hit the breakpoint (set on the System.out.println in the static main method), the ping thread continued to run, and I could also access and do things with the database through ij. Are you sure you're only stopping the current thread in your debugger? When the system appears to hang, can you obtain a stack trace (using jstack or similar) and check what the Derby threads are doing? Are you sure your stopped thread isn't holding on to some monitors / locks required by the other threads? (sounds unlikely, as the Derby threads should operate on their own, but I'm asking anyway...) See the code I tested with below (I'm using localhost as host, so you cannot connect to Derby remotely). Regards, -- Kristian import java.io.PrintWriter; import java.net.InetAddress; import java.util.concurrent.atomic.AtomicBoolean; import org.apache.derby.drda.NetworkServerControl; public class ServerTestDebug implements Runnable { private final InetAddress m_host; private final int m_port = 1527; private final AtomicBoolean m_done = new AtomicBoolean(false); private final int m_sleep = 5000; private NetworkServerControl m_server; public ServerTestDebug() throws Exception { m_host = InetAddress.getByName("localhost"); } final public void start() throws Exception { new Thread(this, "Derby Server").start(); } public void stop() { System.out.println("Telling server to stop..."); m_done.set(true); } public void run() { try { m_server = new NetworkServerControl(m_host, m_port); m_server.start(new PrintWriter(System.out)); } catch (Exception e) { e.printStackTrace(System.out); return; } try { while (!m_done.get()) { System.out.println("pinging..."); m_server.ping(); try { Thread.sleep(m_sleep); } catch (InterruptedException e) { /* log */ } } } catch (Exception e) { /* log - errror pinging server */ } try { System.out.println("Shutting down..."); m_server.shutdown(); } catch (Exception e) { e.printStackTrace(); } } public static void main(String[] args) throws Exception { ServerTestDebug server = new ServerTestDebug(); server.start(); System.out.println("Starting main loop..."); int iterations = 30; while (iterations > 0) { if (iterations == 20) { System.out.println("breakpoint here!"); } try { Thread.sleep(2000); } catch (InterruptedException ie) { } iterations--; } server.stop(); } } > Much appreciated, > > Pavel. > > > > > > > > > > > > Bryan Pendleton > 02/05/2010 12:07 PM > Please respond to > "Derby Discussion" > > > To > Derby Discussion > cc > > Subject > Re: Using NetworkServerControl to access in-memory (only) tables of > Embedded Derby > > > > > > > >> DerbyServer thread seems to still be running, yet the server is >> unresponsive. >> > In addition to your own DerbyServer thread, which I don't think > actually *needs* to still be running, there should be a separate > thread which is started by the Derby network server code itself, > which has the job of accepting connections and delivering them > to other threads to be processed. > > In my running network server, for example, when I look at the > threads that are active, I see: > > "NetworkServerThread_2" prio=6 tid=0x03044c80 nid=0x27c runnable > [0x033cf000..0x033cfae8] > at java.net.PlainSocketImpl.socketAccept(Native Method) > at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) > - locked<0x22a857d8> (a java.net.SocksSocketImpl) > at java.net.ServerSocket.implAccept(ServerSocket.java:450) > at java.net.ServerSocket.accept(ServerSocket.java:421) > at org.apache.derby.impl.drda.ClientThread$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at org.apache.derby.impl.drda.ClientThread.run(Unknown Source) > > Do you see a thread like that? > > When your server becomes stuck/wedged/unresponsive, why don't you try > this: > - collect a thread dump of the entire JVM > - look through the threads for any which mention "org.apache.derby" in > their stacks > - edit out all the other threads from your thread dump > > Then post a message with a cut-and-paste of just the derby-related threads > in your wedged server, and maybe it will be more clear to somebody else > what the problem is. > > thanks, > > bryan > > > > > > > > > Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority. > > Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority. --=_alternative 0062881D852576CE_= Content-Type: text/html; charset="US-ASCII"
Thank you, Kristian and Bryan for all your help. I did locate the setting which allows me to suspend on the thread which hit the breakpoint while allowing others to execute.
But it was quite an interesting exercise, nevertheless.






Pavel Bortnovskiy/JEFCO

02/18/2010 12:47 PM

To
"Derby Discussion" <derby-user@db.apache.org>
cc
Kristian.Waagan@Sun.COM
Subject
Re: Using NetworkServerControl to access in-memory (only) tables of Embedded DerbyLink




Hello, Kristian:

Thanks for your response.

I think I found what the problem is... When my IDE (IntelliJ IDEA) hits a breakpoint, it suspends all threads. I find it funny that in all these years of programming and using IDEA, I've never realized it.
(possibly because I was too occupied debugging the code at the breakpoint and not thinking about other threads at that moment). I've searched through the options but couldn't find any setting
which will prevent other threads from being suspended. Will continue looking.

I am now curious, what IDE do you use that it worked for you? And did you need to enable any special settings to allow all other threads to continue execution?

Regards,

Pavel.






Kristian Waagan <Kristian.Waagan@Sun.COM>
Sent by: Kristian.Waagan@Sun.COM

02/16/2010 03:32 AM
Please respond to
"Derby Discussion" <derby-user@db.apache.org>
To
Derby Discussion <derby-user@db.apache.org>
cc
Subject
Re: Using NetworkServerControl to access in-memory (only) tables of Embedded Derby





On 16.02.10 03:02, Pavel Bortnovskiy wrote:
> Bryan: thank you for your response.
>
> I do see the thread you mention:
>
>          Thread Group [derby.daemons]{10}(Daemon)
>              Thread [derby.antiGC]{1}(Daemon)(WAITING)
>              Thread [derby.rawStoreDaemon]{5}(Daemon)(TIMED_WAITING)
>              Thread [derby.NetworkServerStarter]{5}(Daemon)(WAITING)
>              Thread [NetworkServerThread_2]{5}(Daemon)(RUNNABLE)
>              Thread [DRDAConnThread_3]{5}(Daemon)(WAITING)
>
> However the behavior is still the same - if any other thread stops (for
> instance in debugger), NetworkServerControl becomes unresponsive.
> For instance, if I connect to it from the "outside" with another db app
> (Aqua Data Studio, for instance), then connection to it breaks or the db
> app can't access it.
> But as soon as I let the thread run in that debugger, the connection
> becomes alive and everything seems to work.
>
> Has anyone else seen this kind of behavior?
> Or perhaps, someone can share a piece of code (best practice) of how to
> instantiate and run the NetworkServerControl, so that it's not
> experiencing
> such hang-ups.
>    

Hi Pavel,

The code I'm about to post is by no means to be considered best
practice, but I think it does what you want it to do. However, the code
is basically yours :) (I copied it from your mail)
When I hit the breakpoint (set on the System.out.println in the static
main method), the ping thread continued to run, and I could also access
and do things with the database through ij.

Are you sure you're only stopping the current thread in your debugger?
When the system appears to hang, can you obtain a stack trace (using
jstack or similar) and check what the Derby threads are doing?
Are you sure your stopped thread isn't holding on to some monitors /
locks required by the other threads? (sounds unlikely, as the Derby
threads should operate on their own, but I'm asking anyway...)

See the code I tested with below (I'm using localhost as host, so you
cannot connect to Derby remotely).


Regards,
--
Kristian


import java.io.PrintWriter;
import java.net.InetAddress;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.derby.drda.NetworkServerControl;

public class ServerTestDebug implements Runnable {

    private final InetAddress m_host;
    private final int m_port = 1527;
    private final AtomicBoolean m_done = new AtomicBoolean(false);
    private final int m_sleep = 5000;

    private NetworkServerControl m_server;

    public ServerTestDebug()
            throws Exception {
        m_host = InetAddress.getByName("localhost");
    }

  final public void start() throws Exception {
    new Thread(this, "Derby Server").start();
  }

  public void stop() {
      System.out.println("Telling server to stop...");
      m_done.set(true);
  }

  public void run() {
      try {
        m_server = new NetworkServerControl(m_host, m_port);
        m_server.start(new PrintWriter(System.out));
      } catch (Exception e) {
            e.printStackTrace(System.out);
            return;
      }

    try {
      while (!m_done.get()) {
            System.out.println("pinging...");
            m_server.ping();

        try {
          Thread.sleep(m_sleep);
        } catch (InterruptedException e) {
          /* log */
        }
      }
    } catch (Exception e) {
      /* log - errror pinging server */
    }
      try {
          System.out.println("Shutting down...");
        m_server.shutdown();
      } catch (Exception e) {
          e.printStackTrace();
      }
  }

    public static void main(String[] args)
            throws Exception {
        ServerTestDebug server = new ServerTestDebug();
        server.start();
        System.out.println("Starting main loop...");
        int iterations = 30;
        while (iterations > 0) {
            if (iterations == 20) {
               System.out.println("breakpoint here!");
            }
            try {
                Thread.sleep(2000);
            } catch (InterruptedException ie) { }
            iterations--;
        }
        server.stop();
    }
}





> Much appreciated,
>
> Pavel.
>
>
>
>
>
>
>
>
>
>
>
> Bryan Pendleton<bpendleton@amberpoint.com>
> 02/05/2010 12:07 PM
> Please respond to
> "Derby Discussion"<derby-user@db.apache.org>
>
>
> To
> Derby Discussion<derby-user@db.apache.org>
> cc
>
> Subject
> Re: Using NetworkServerControl to access in-memory (only) tables of
> Embedded Derby
>
>
>
>
>
>
>    
>> DerbyServer thread seems to still be running, yet the server is
>> unresponsive.
>>      
> In addition to your own DerbyServer thread, which I don't think
> actually *needs* to still be running, there should be a separate
> thread which is started by the Derby network server code itself,
> which has the job of accepting connections and delivering them
> to other threads to be processed.
>
> In my running network server, for example, when I look at the
> threads that are active, I see:
>
> "NetworkServerThread_2" prio=6 tid=0x03044c80 nid=0x27c runnable
> [0x033cf000..0x033cfae8]
>           at java.net.PlainSocketImpl.socketAccept(Native Method)
>           at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
>           - locked<0x22a857d8>  (a java.net.SocksSocketImpl)
>           at java.net.ServerSocket.implAccept(ServerSocket.java:450)
>           at java.net.ServerSocket.accept(ServerSocket.java:421)
>           at org.apache.derby.impl.drda.ClientThread$1.run(Unknown Source)
>           at java.security.AccessController.doPrivileged(Native Method)
>           at org.apache.derby.impl.drda.ClientThread.run(Unknown Source)
>
> Do you see a thread like that?
>
> When your server becomes stuck/wedged/unresponsive, why don't you try
> this:
>    - collect a thread dump of the entire JVM
>    - look through the threads for any which mention "org.apache.derby" in
> their stacks
>    - edit out all the other threads from your thread dump
>
> Then post a message with a cut-and-paste of just the derby-related threads
> in your wedged server, and maybe it will be more clear to somebody else
> what the problem is.
>
> thanks,
>
> bryan
>
>
>
>
>
>
>
>
> Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited.  In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ.  Jefferies International Limited is authorised and regulated by the Financial Services Authority.
>
>    





Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited.  In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ.  Jefferies International Limited is authorised and regulated by the Financial Services Authority. --=_alternative 0062881D852576CE_=--