Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of markrmiller@gmail.com
 designates 209.85.221.188 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject:references
         :in-reply-to:x-enigmail-version:content-type
         :content-transfer-encoding;
        b=qCXM1rPJoaJ3OXhYS04eUQO61XtmNmChSVBVsx6FBKYRz13tuhO/jf2WGUuZud9/ni
         nkTZTEndE2E3qFRRywfl060Rwu+aEGGjKz2lvAB3RUkdbixvtl9/j//yB7O3keMTHXVt
         VS2xZai0KqKnri7W1nkvsLbrMyHhjd2/fr890=
Message-ID: <4B61A43A.5080501@gmail.com>
Date: Thu, 28 Jan 2010 09:50:34 -0500
From: Mark Miller <markrmiller@gmail.com>
User-Agent: Thunderbird 2.0.0.23 (X11/20090817)
MIME-Version: 1.0
To: solr-dev@lucene.apache.org
Subject: Re: svn commit: r899979 -
 /lucene/solr/trunk/example/solr/conf/solrconfig.xml
References: <20100116153355.084D1238890A@eris.apache.org>
  <Pine.LNX.4.64.1001161236290.13090@radix.cryptio.net>
  <c68e39171001161253t69f6fdcfl6d9e666cf0378c55@mail.gmail.com>
  <Pine.LNX.4.64.1001172210470.27805@radix.cryptio.net>
 <c68e39171001180634m37fc08b0k506ec6839d588b77@mail.gmail.com>
 <4B54931B.6090302@gmail.com>
 <Pine.LNX.4.64.1001191332270.10224@radix.cryptio.net>
 <4B5641B2.2020507@gmail.com>
 <Pine.LNX.4.64.1001191626550.10224@radix.cryptio.net>
In-Reply-To: <Pine.LNX.4.64.1001191626550.10224@radix.cryptio.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Chris Hostetter wrote:
> : > At a minimu, shouldn't NativeFSLock.obtain() be checking for 
> : > OverlappingFileLockException and treating that as a failure to acquire the 
> : > lock?
> 	...
> : Perhaps - that should make it work in more cases - but in my simple
> : testing its not 100% reliable.
> 	...
> : File locks are held on behalf of the entire Java virtual machine.
> :      * They are not suitable for controlling access to a file by multiple
> :      * threads within the same virtual machine.
>
> ...Grrr....  so where does that leave us?
>
> Yonik's added comment was that "native" isnt' recommended when running 
> multiple webapps in the same container.  in truth, "native" *can* 
> work when running multiple webapps in the same container, just as long as 
> those cotnainers don't refrence the same data dirs
>
> I'm worried that we should recommend people avoid native altogether 
> because even if you are only running one webapp, it seems like a "reload" 
> or that app could trigger some similar bad behavior.
>
> So what/how should we document all of this?
>
> -Hoss
>
>   
I've got more info on this.

Checking for OverlappingFileLockException *should* actually work when
using Java 1.6. Java 1.6 started using a *system wide* thread safe check
for this.

Previous to Java 1.6, checks for this *were* limited to an instance of
FileChannel - the FileChannel maintained its own personal lock list. So
you have to use
the same Channel to even have any hope of seeing an
OverlappingFileLockException. Even then though, its not properly thread
safe. They did not sync across
checking if the lock exists and acquiring the lock - they separately
sync each action - leaving room to acquire the lock twice from two
different threads like I was seeing.

Interestingly, Java 1.6 has a back compat mode you can turn on that
doesn't use the system wide lock list, and they have fixed this thread
safety issue in that impl - there is a sync across checking
and getting the lock so that it is properly thread safe - but not in
Java 1.4, 1.5.

Looking at GCC - uh ... I don't think you want to use GCC - they don't
appear to use a lock list and check for this at all :)

But the point is, this is fixable on Java 6 if we check for
OverlappingFileLockException - it *should* work across webapps, and it
is actually thread safe, unlike Java 1.4,1.5.

-- 
- Mark

http://www.lucidimagination.com