From derby-user-return-14638-apmail-db-derby-user-archive=db.apache.org@db.apache.org Wed Sep 26 17:03:43 2012 Return-Path: X-Original-To: apmail-db-derby-user-archive@www.apache.org Delivered-To: apmail-db-derby-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C6B3ADCA8 for ; Wed, 26 Sep 2012 17:03:43 +0000 (UTC) Received: (qmail 64126 invoked by uid 500); 26 Sep 2012 17:03:43 -0000 Delivered-To: apmail-db-derby-user-archive@db.apache.org Received: (qmail 64090 invoked by uid 500); 26 Sep 2012 17:03:43 -0000 Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Reply-To: "Derby Discussion" Delivered-To: mailing list derby-user@db.apache.org Received: (qmail 64080 invoked by uid 99); 26 Sep 2012 17:03:43 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Sep 2012 17:03:43 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [93.17.128.118] (HELO smtp25.services.sfr.fr) (93.17.128.118) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 26 Sep 2012 17:03:35 +0000 Received: from filter.sfr.fr (localhost [127.0.0.1]) by msfrf2519.sfr.fr (SMTP Server) with ESMTP id 0982C7000041 for ; Wed, 26 Sep 2012 19:03:15 +0200 (CEST) Received: from [192.168.1.31] (196.168.9.109.rev.sfr.net [109.9.168.196]) by msfrf2519.sfr.fr (SMTP Server) with ESMTP id AFD3770002AB for ; Wed, 26 Sep 2012 19:03:14 +0200 (CEST) X-SFR-UUID: 20120926170314720.AFD3770002AB@msfrf2519.sfr.fr Message-ID: <50633559.1030608@yahoo.fr> Date: Wed, 26 Sep 2012 19:03:21 +0200 From: Mo Maison User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.6esrpre) Gecko/20120817 Icedove/10.0.6 MIME-Version: 1.0 To: derby-user@db.apache.org Subject: Re: Stopping network server & cold backup References: <501844D4.6020708@yahoo.fr> <505C9BAF.4030208@yahoo.fr> <505CCCC1.3020204@oracle.com> In-Reply-To: <505CCCC1.3020204@oracle.com> Content-Type: multipart/alternative; boundary=------------030504090902060707060307 This is a multi-part message in MIME format. --------------030504090902060707060307 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit I have two more questions : 1 - in case server has been shutdown abruptly, db.lck and dbex.lck files are still present in . Later on, if derby server is only started but no client ever connects to this database, these files remain there, unchanged. Thus, even after derby server has been properly shutdown, db.lck files are to be expected : in this case, this does not indicate that is unsuitable for copy. 2 - OK, I'll try to connect to server port to see if it succeeds or not. However, is it correct to assume that "1527 port no more bound" implies "all are safe for backup" ? In case derby server first closes its server socket, _then_ closes properly all the opened databases, this assumption may be false. (I haven't looked into derby server code yet ; I have observed though that server shutdown may require several seconds, depending on file system type, machine load...) So in conclusion, it is still quite hard to figure out _when_ it is safe to backup a . Am I wrong ? Anyway I'll try to deal with all these informations. Thank you for your help. Regards, M. Maison Le 21/09/2012 22:23, Dag Wanvik a écrit : > > On 21.09.2012 18:54, Mo Maison wrote: >> >> Thank you for this tip. >> I do not feel confident with whole DB freeze, as I do not know DB size, >> thus I can't estimate the time required to copy files. Also I fear >> something >> goes wrong and UNFREEZE is never called. >> In case, is restarting the Derby server sufficient to "forget" this >> frozen state ? > > Yes. > >> Regarding my initial question : is there somewhere a file to watch >> that indicates that server is fully shutdown ? >> There is probably a kind of lock file that indicates that a DB is >> being used ? > > /db.lock is created when a database is booted by a VM, to avoid > several VMs trying to boot the same database. > The derby *server* (engine) can open several databases. By by shutting > down a database > the corresponding lock file is removed, but the server can still be > alive waiting for connections. Presently, I think the only way to portably > check if the server is still alive is to try to connect to its port. > > Thanks, > Dag > >> Watching server process in /proc would be ugly and not portable. > > > >> >> M. Maison >> >> >> Le 31/07/2012 22:59, José Ventura a écrit : >>> I'm using the tips in this page to achieve a similar goal: >>> >>> http://db.apache.org/derby/docs/10.6/adminguide/cadminhubbkup75469.html >>> >>> Basically, instead of shutting down the server, you use a command to >>> freeze the DB, then copy the files, then unfreeze the DB. The freeze >>> call will block until it is safe to copy the files. >>> >>> On Tue, Jul 31, 2012 at 5:49 PM, Mo Maison >> > wrote: >>> >>> Hello Derby users, >>> >>> I encounter sometimes a problem during cold backups : >>> Derby is launched as a server (listening localhost only ) ; >>> I stop the network server with standard command : >>> java -cp ... \ >>> org.apache.derby.drda.NetworkServerControl shutdown \ >>> -h localhost -p 1527 >>> >>> Right after command has returned, I perform a tar of the >>> database files. >>> However, I have observed that, on a slow filesystem, somes >>> files get deleted while tar is reading them. Which let me think >>> that server is not fully stopped when process returns >>> (and thus backup may be corrupted). >>> >>> Is that expected ? >>> How can I be sure that everything is quiescent before performing >>> the cold backup ? >>> I use Derby 10.6.2 on 32 bits Linux. >>> >>> Thank you for your advices, >>> >>> M. Maison >>> >>> >> > --------------030504090902060707060307 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
I have two more questions :

1 - in case server has been shutdown abruptly, db.lck
  and dbex.lck files are still present in <dbdir>.
  Later on, if derby server is only started but no client ever
  connects to this database, these files remain there, unchanged.
  Thus, even after derby server has been properly shutdown,
  db.lck files are to be expected : in this case, this does not
  indicate that <dbdir> is unsuitable for copy.

2 - OK, I'll try to connect to server port to see if it succeeds
  or not. However, is it correct to assume that
  "1527 port no more bound" implies "all <dbdir> are safe for backup" ?
  In case derby server first closes its server socket, _then_ closes
  properly all the opened databases, this assumption may be false.
  (I haven't looked into derby server code yet ; I have observed though
  that server shutdown may require several seconds, depending on
  file system type, machine load...)

So in conclusion, it is still quite hard to figure out _when_ it is safe
to backup a <dbdir>. Am I wrong ?
Anyway I'll try to deal with all these informations.
Thank you for your help.

Regards,
  M. Maison

Le 21/09/2012 22:23, Dag Wanvik a écrit :

On 21.09.2012 18:54, Mo Maison wrote:

Thank you for this tip.
I do not feel confident with whole DB freeze, as I do not know DB size,
thus I can't estimate the time required to copy files. Also I fear something
goes wrong and UNFREEZE is never called.
In case, is restarting the Derby server sufficient to "forget" this frozen state ?

Yes.

Regarding my initial question : is there somewhere a file to watch
that indicates that server is fully shutdown ?
There is probably a kind of lock file that indicates that a DB is being used ?

<dbdir>/db.lock is created when a database is booted by a VM, to avoid several VMs trying to boot the same database.
The derby *server* (engine) can open several databases. By by shutting down a database
the corresponding lock file is removed, but the server can still be alive waiting for connections. Presently, I think the only way to portably
check if the server is still alive is to try to connect to its port.

Thanks,
Dag

Watching server process in /proc would be ugly and not portable.




M. Maison


Le 31/07/2012 22:59, José Ventura a écrit :
I'm using the tips in this page to achieve a similar goal:


Basically, instead of shutting down the server, you use a command to freeze the DB, then copy the files, then unfreeze the DB. The freeze call will block until it is safe to copy the files.

On Tue, Jul 31, 2012 at 5:49 PM, Mo Maison <momaison@yahoo.fr> wrote:
Hello Derby users,

I encounter sometimes a problem during cold backups :
Derby is launched as a server (listening localhost only ) ;
I stop the network server with standard command :
  java -cp ... \
  org.apache.derby.drda.NetworkServerControl shutdown \
  -h localhost -p 1527

Right after command has returned, I perform a tar of the
database files.
However, I have observed that, on a slow filesystem, somes
files get deleted while tar is reading them. Which let me think
that server is not fully stopped when process returns
(and thus backup may be corrupted).

Is that expected ?
How can I be sure that everything is quiescent before performing
the cold backup ?
I use Derby 10.6.2 on 32 bits Linux.

Thank you for your advices,

 M. Maison




--------------030504090902060707060307--