Return-Path: X-Original-To: apmail-subversion-users-archive@minotaur.apache.org Delivered-To: apmail-subversion-users-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6355018133 for ; Wed, 25 Nov 2015 22:01:40 +0000 (UTC) Received: (qmail 47870 invoked by uid 500); 25 Nov 2015 22:01:39 -0000 Delivered-To: apmail-subversion-users-archive@subversion.apache.org Received: (qmail 47833 invoked by uid 500); 25 Nov 2015 22:01:39 -0000 Mailing-List: contact users-help@subversion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@subversion.apache.org Received: (qmail 47823 invoked by uid 99); 25 Nov 2015 22:01:39 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Nov 2015 22:01:39 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 3895E1A2229 for ; Wed, 25 Nov 2015 22:01:39 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=qqmail.nl Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id A8HJKtl3bO7v for ; Wed, 25 Nov 2015 22:01:37 +0000 (UTC) Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 8207620EA4 for ; Wed, 25 Nov 2015 22:01:36 +0000 (UTC) Received: by wmvv187 with SMTP id v187so5819820wmv.1 for ; Wed, 25 Nov 2015 14:01:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qqmail.nl; s=google; h=from:to:references:in-reply-to:subject:date:message-id:mime-version :content-type:thread-index:content-language; bh=PrYwa09yzFCK9d0WdrFrDmhlMcVXqdpNiiLJNt4Hghk=; b=fC6jMkbd7YEkjPvbXieU6ywUNz0h+kHObkiyZE4jKNLgcGDvYL/fA8Ct9TZstU8NBa fkHGSTyeWkeJU8IdpIzywpp/LKzhk4nlTqp96eeAXInwj6ghrJTuL+VbOHnqU55utHlA 5tPyfXDwATtaMXRu3c9CC6zSCZm87Acy4Bseg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:references:in-reply-to:subject:date :message-id:mime-version:content-type:thread-index:content-language; bh=PrYwa09yzFCK9d0WdrFrDmhlMcVXqdpNiiLJNt4Hghk=; b=ES7JZNv+X8Am32+V1V9MB+SUKP1LsGi5Pct0aU428xzBeU7h/tr+e3JuUFD1hxgURz S8MRrXOZt1nJw5ZM4xX/IVKPoBgYFMSypU305gtcbTwU7tzt+n5kMxcvqbHx+ygwpS05 Ik3txv2DngOezzTDxB3yyUS/sjT5o3uXUJZWPrcgW8I8dPantnSt0Df2bNJaRPDL2Jc9 b9mh7LQ3AZkd/kYeMw1STq6330oH368GSoeXRBh2uxKZY5+VdpY+zjOvsAQ2OidWYi6s Y1zRfP7GEynEbhP4OeWDCYDJJOA4kQ9myXoH1MylOSay7sPRyZJDbGfpwytXz+FE9oF4 obSw== X-Gm-Message-State: ALoCoQlyaXY7HVgr9NI5DghTCoAUBxG3suD6PHKgd5GgF69zyko7MdA7pa9HHh8U+QIvjqoh7wbE X-Received: by 10.28.12.3 with SMTP id 3mr7314535wmm.84.1448488895196; Wed, 25 Nov 2015 14:01:35 -0800 (PST) Received: from i72600 ([2001:610:66e:0:52e5:49ff:fee1:96b7]) by smtp.gmail.com with ESMTPSA id t194sm5354886wmt.11.2015.11.25.14.01.34 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 25 Nov 2015 14:01:34 -0800 (PST) From: "Bert Huijben" To: "'Barry Gershenfeld'" , "'Subversion'" References: In-Reply-To: Subject: RE: SVN 1.6.17 dump is growing larger than repository size (approx. more than 10 times) Date: Wed, 25 Nov 2015 23:01:27 +0100 Message-ID: <00c501d127cc$d564cde0$802e69a0$@qqmail.nl> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00C6_01D127D5.372A2040" X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQLR0QOiuQvDpCC/lAZ4RpC9x9guxwMzalkwAwbsb3aceiRpwA== Content-Language: nl This is a multipart message in MIME format. ------=_NextPart_000_00C6_01D127D5.372A2040 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable By default svnadmin dump doesn=E2=80=99t produce deltas=E2=80=A6 so a = small change on a file will have it include the entire file in the = dumpfile. =20 If you pass =E2=80=98--deltas' to svnadmin dump (see =E2=80=98svnadmin = help dump=E2=80=99) your file will be smaller, at the cost of some extra = processing during the dump creation. =20 Bert =20 From: Barry Gershenfeld [mailto:gbarry42@gmail.com]=20 Sent: woensdag 25 november 2015 21:57 To: Subversion Subject: Re: SVN 1.6.17 dump is growing larger than repository size = (approx. more than 10 times) =20 Unless svn's changed, you can look in your repositories directory on the = server, e.g., repos/myproj/db/revs/0/ and you can see each revision = stored there. Under the db directory is revs and also revprops, and = under each of those is one or more subdirectory (mine just had one = called "0"). If you see one that is outrageously large, then dump is = just trying to do its job. If they are all reasonable size, then maybe = there's a circular reference or some other bug. =20 Also, svnadmin dump just goes to standard-out, so you can send it to the = screen, or through a grep filter or some other ingenious scheme to try = to get a handle on what's happening. =20 Barry =20 =20 =20 On Wed, Nov 25, 2015 at 4:41 AM, Nico Kadel-Garcia > wrote: On Wed, Nov 25, 2015 at 4:25 AM, arun prasath > wrote: > Hello Team, > > I am creating Subversion 1.6.17 dump for a repository hosted in Linux > server. SVNSERVE is serving the repository. We are migrating to SVN = 1.9.2. > > While creating the dump for repository of size 1.8 GB (revisions = 3000+), the > dump command completes revision 119 and hangs and keep updating the = dumpfile > which grows to 7-8 GBs. dump command is not moving to next revision = but kept > updating the dump file. So, i just closed the putty to stop creating = the > dump. Which Linux? And are you using the verndor provided version, or a locally compiled one? Subversion 1.6 was last up to 1.6.23: is there any reason you can't upgrade that? Ideally with a hotcopy or "rsync" based copy of the repository to somehwere else, for testing the copy? > However, I ran the svnadmin verify which completes revision 119 and = take > some time to verify revision 120 and complete the verification = successfully > for all the repository revisions. Hmm. You might consider skipping the svnadmin dump command, and simply setting up an svnsync mirror on the Subversion 1.9 server. Even using "rsync" to bring the old repository over should let you run a dump and reload on the server with Subversion 1.9, as long as there's not some other weird corruption with revision 119 or 120. > Since, there is no error output I have no logs to attach. Please = suggest the > work around for this situation. How can create the dump and migrate to > target server. I am planning to use rsync from the server. I will post = how > it goes. > > I am expecting the workaround to this situation and let me know if = this is a > known issue in SVN 1.6.17. This is my active repository and created = the dump > without stopping the svnserve program in the production server. It's not one I've seen, but I don't let my repositories get that big. And I'd look at revisions 119 and 120 to see if there were erroneous commits of huge binary files, in which case I'd want to exclude them from the backup. =20 ------=_NextPart_000_00C6_01D127D5.372A2040 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable

By default svnadmin dump doesn=E2=80=99t = produce deltas=E2=80=A6 so a small change on=C2=A0 a file will have it = include the entire file in the dumpfile.

 

If you pass =E2=80=98--deltas' to svnadmin = dump (see =E2=80=98svnadmin help dump=E2=80=99) your file will be = smaller, at the cost of some extra processing during the dump = creation.

 

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Bert

 

From:<= /b> Barry = Gershenfeld [mailto:gbarry42@gmail.com]
Sent: woensdag 25 = november 2015 21:57
To: Subversion = <users@subversion.apache.org>
Subject: Re: SVN 1.6.17 = dump is growing larger than repository size (approx. more than 10 = times)

 

Unless = svn's changed, you can look in your repositories directory on the = server, e.g., repos/myproj/db/revs/0/ and you can see each revision = stored there.  Under the db directory is revs and also revprops, = and under each of those is one or more subdirectory (mine just had one = called "0").   If you see one that is outrageously large, = then dump is just trying to do its job.  If they are all reasonable = size, then maybe there's a circular reference or some other = bug.

 

Also, svnadmin dump just goes to standard-out, so you = can send it to the screen, or through a grep filter or some other = ingenious scheme to try to get a handle on what's = happening.

 

Barry

 

 

 

On Wed, = Nov 25, 2015 at 4:41 AM, Nico Kadel-Garcia <nkadel@gmail.com> = wrote:

On Wed, = Nov 25, 2015 at 4:25 AM, arun prasath <get2arun@gmail.com> = wrote:
> Hello Team,
>
> I am creating Subversion = 1.6.17 dump for a repository hosted in Linux
> server. SVNSERVE is = serving the repository. We are migrating to SVN 1.9.2.
>
> = While creating the dump for repository of size 1.8 GB (revisions 3000+), = the
> dump command completes revision 119 and hangs and keep = updating the dumpfile
> which grows to 7-8 GBs. dump command is = not moving to next revision but kept
> updating the dump file. So, = i just closed the putty to stop creating the
> dump.

Which = Linux? And are you using the verndor provided version, or a
locally = compiled one? Subversion 1.6 was last up to 1.6.23: is there
any = reason you can't upgrade that? Ideally with a hotcopy or = "rsync"
based copy of the repository to somehwere else, for = testing the copy?

> However, I ran the svnadmin verify which = completes revision 119 and take
> some time to verify revision 120 = and complete the verification successfully
> for all the = repository revisions.

Hmm. You might consider skipping the = svnadmin dump command, and simply
setting up an svnsync mirror on the = Subversion 1.9 server. Even using
"rsync" to bring the old = repository over should let you run a dump and
reload on the server = with Subversion 1.9, as long as there's not some
other weird = corruption with revision 119 or 120.

> Since, there is no = error output I have no logs to attach. Please suggest the
> work = around for this situation. How can create the dump and migrate = to
> target server. I am planning to use rsync from the server. I = will post how
> it goes.
>
> I am expecting the = workaround to this situation and let me know if this is a
> known = issue in SVN 1.6.17. This is my active repository and created the = dump
> without stopping the svnserve program in the production = server.

It's not one I've seen, but I don't let my repositories = get that big.
And I'd look at revisions 119 and 120 to see if there = were erroneous
commits of huge binary files, in which case I'd want = to exclude them
from the backup.

 

------=_NextPart_000_00C6_01D127D5.372A2040--