From users-return-11218-apmail-subversion-users-archive=subversion.apache.org@subversion.apache.org Mon Oct 3 08:11:15 2011 Return-Path: X-Original-To: apmail-subversion-users-archive@minotaur.apache.org Delivered-To: apmail-subversion-users-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2E4409BE7 for ; Mon, 3 Oct 2011 08:11:15 +0000 (UTC) Received: (qmail 78403 invoked by uid 500); 3 Oct 2011 08:11:14 -0000 Delivered-To: apmail-subversion-users-archive@subversion.apache.org Received: (qmail 78317 invoked by uid 500); 3 Oct 2011 08:11:14 -0000 Mailing-List: contact users-help@subversion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@subversion.apache.org Received: (qmail 78307 invoked by uid 99); 3 Oct 2011 08:11:14 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Oct 2011 08:11:14 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jcorvel@gmail.com designates 209.85.216.43 as permitted sender) Received: from [209.85.216.43] (HELO mail-qw0-f43.google.com) (209.85.216.43) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Oct 2011 08:11:07 +0000 Received: by qabg14 with SMTP id g14so1987166qab.16 for ; Mon, 03 Oct 2011 01:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=Nsd63SCJtTY6llOtCBk09XzmI8yeYx7+56Eix7H7ZCw=; b=GNc+3yTThWLr2icCGRgPXMKiST/EdPmS+hdbIfaekjn05UYPgWIpJ79K7ACik/p3SZ k7RPVxUimoVXakXagWUQqbytT9I2Ogv1EHa5yR/6CehYnGWlacMUn4dHkyFh+aICWkTw AeXMpXNlWsdM/w8vCgSzkW56Hk8jUQ0OEB98k= Received: by 10.229.243.161 with SMTP id lm33mr11298212qcb.4.1317629445123; Mon, 03 Oct 2011 01:10:45 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.246.12 with HTTP; Mon, 3 Oct 2011 01:10:25 -0700 (PDT) In-Reply-To: References: <20111001191045.GA25864@daniel3.local> <20111001235546.GA30144@daniel3.local> <20111003000955.GA24599@daniel3.local> From: Johan Corveleyn Date: Mon, 3 Oct 2011 10:10:25 +0200 Message-ID: Subject: Re: svn merge operation extremely slow To: Kyle Leber Cc: Daniel Shahaf , users@subversion.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org [ Again: please don't top-post on this list. I'm moving your reply to the bottom. More below ... ] On Mon, Oct 3, 2011 at 2:24 AM, Kyle Leber wrote: > On Sun, Oct 2, 2011 at 8:10 PM, Daniel Shahaf > wrote: >> >> Kyle Leber wrote on Sun, Oct 02, 2011 at 20:05:19 -0400: >> > Johan, >> > >> > I did a little more digging. =A0There were a few different places wher= e >> > svn >> > seems to get hung up so I ran the gprof report on just the first one >> > (the >> > merge takes hours otherwise). =A0In this particular case, svn prints o= ut >> > that >> > it is merging from a small text file while it is hanging for more than= a >> > minute @ 100% CPU. =A0When I examine "lsof", however, it see it actual= ly >> > has a >> > different file open. =A0This one is a large (15 MB) "binary" file. =A0= It >> > turns >> > out this binary file did not have a property in the trunk (which I thi= nk >> > means it's treated as text, right?). =A0But in the branch it was marke= d as >> > octet stream. =A0 So perhaps svn is doing a text-based diff on this bi= nary >> > file because it used to be incorrectly marked as text? >> > >> >> If either side is marked as binary then svn will defer to the "Use >> merge-right if merge-left =3D=3D base, else conflict" algorithm. >> >> Could you share the value of 'svn proplist --verbose' on both files? >> > Yup, trunk version has empty properties > branch version has: > > svn:mime-type > application/octet-stream > What is the merge target? Is it a trunk working copy (the one without mime-type), or a branch working copy (with svn:mime-type=3Dapplication/octet-stream)? I think it's the mime-type of the merge target that determines if merge will take the "binary" route, or the "text" route. See this snippet from libsvn_wc/merge.c [1] (in the function svn_wc__internal_merge): [[[ /* Decide if the merge target is a text or binary file. */ if ((mimeprop =3D get_prop(&mt, SVN_PROP_MIME_TYPE)) && mimeprop->value) is_binary =3D svn_mime_type_is_binary(mimeprop->value->data); else { const char *value =3D svn_prop_get_value(mt.actual_props, SVN_PROP_MIME_TYPE); is_binary =3D value && svn_mime_type_is_binary(value); } ]]] (mt is the merge target) I'm not terribly familiar with this part of the codebase. But on first sight, this seems to say: (1) Look at the mime-type of the "base version" of the merge target. If that's binary, then we'll go binary. (2) If the "base" of the merge target doesn't have a mime-type, look if it has one on the "actual" node (the uncommitted local modifications). If that's binary, then we'll go binary. (3) Else: text merge So I'm guessing that you're merging to trunk, the target without mime-type property, which makes svn take the "text" route for merging. Is that correct? If that's the case, maybe you can simply set the mime-type on that binary file in your merge target, as a local modification (I don't think you need to even commit it). Can you try that? --=20 Johan [1] http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_wc/m= erge.c