Mailing-List: contact dev-help@clerezza.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@clerezza.apache.org
Received-SPF: pass (nike.apache.org: domain of andy.seaborne.apache@gmail.com
 designates 209.85.212.178 as permitted sender)
Sender: Andy Seaborne <andy.seaborne.apache@gmail.com>
Message-ID: <5148B726.2050802@apache.org>
Date: Tue, 19 Mar 2013 19:06:14 +0000
From: Andy Seaborne <andy@apache.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:17.0) Gecko/20130308 Thunderbird/17.0.4
MIME-Version: 1.0
To: dev@clerezza.apache.org
Subject: Re: Test failures on fresh build
References: <5141A1FE.8060509@multimach.com>
 <CALvhUEXdWkBz_HFXfnzOr-VO-14u+5R6nhCgU3SeYQppqUEP0A@mail.gmail.com>
 <CALvhUEXikr8fAZ8D=Qvps5ov5G5k=n+8OKnwOT5RKW5wE4yZfg@mail.gmail.com>
 <51477F8E.6060905@xup.nl>
 <CAA7LAO2oi9+7AhPdLBm0zzUqzQQcR8b-O432JdQmPr5ZDAedxQ@mail.gmail.com>
 <CALvhUEV56ViTmLC+m=N0qjnRE9OxUSzXhuvRQFYqdEtAqBghww@mail.gmail.com>
 <CALvhUEW0375Ld5xSc1D5yzYKTL0M_B7ixRdfni71OXAQUumjuw@mail.gmail.com>
 <51483974.2020306@xup.nl>
 <CALvhUEVM+Cf3hZmpDOdQGnXPOb8+FKbhFEwU-dHvNhqfN97nfg@mail.gmail.com>
 <CALvhUEXqAGND6TXK7YBz4Xq3AAcaDiRHNNMjMKhQOxGC=ufOBQ@mail.gmail.com>
 <CALvhUEXFEUWXODK5eWH8zFLaHO7wn8rwVbe6O5+-u95XiMJb5A@mail.gmail.com>
 <51487D3F.4060105@apache.org>
 <CALvhUEUh-aZ9iTZ1HM085pH=gv+owUEXXYRGKaLaH2fKV8b3Mg@mail.gmail.com>
 <514888AA.6020006@apache.org>
 <CALvhUEWSnPGhbJSfOapenKE8zk-WZCQaxVGJ-H-h+tjVFJ5DKQ@mail.gmail.com>
In-Reply-To: 
 <CALvhUEWSnPGhbJSfOapenKE8zk-WZCQaxVGJ-H-h+tjVFJ5DKQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

On 19/03/13 16:55, Reto Bachmann-Gm�r wrote:
> On Tue, Mar 19, 2013 at 4:47 PM, Andy Seaborne <andy@apache.org> wrote:
>
>> On 19/03/13 15:35, Reto Bachmann-Gm�r wrote:
>>
>>> On Tue, Mar 19, 2013 at 3:59 PM, Andy Seaborne <andy@apache.org> wrote:
>>>
>>>   On 19/03/13 13:57, Reto Bachmann-Gm�r wrote:
>>>>
>>>>   So for the problem with the tdb tc provider I've made a first commit on
>>>>> issue CLEREZZA-749.
>>>>>
>>>>> It doesn't solve the problem only shows it more clearly.
>>>>>
>>>>> Apparently jena tdb keeps the files open even after Graph.close has been
>>>>> called making it impossible to delete them. I'm not sure how this could
>>>>> be
>>>>> solved or if we should raise an issue again Jena TDB.
>>>>>
>>>>>
>>>> (you are using an old version of TDB - even without transactions,
>>>> updating
>>>> will pick up various important bug fixes)
>>>>
>>>>
>>> Ok.
>>>
>>>
>>>> A graph is just a stateless view on the database so closing a graph is a
>>>> bit of a no-op (it syncs the files if not transactional).  Just
>>>> forgetting
>>>> about them is fine, no close required.  Or do a removeAll() to clear
>>>> them.
>>>>
>>>>
>>> The problem is that the underlying files are not closed and so they cannot
>>> be deleted under windows. removeAll seems not to be available in the
>>> version we are using.
>>>
>>
>> Graph.remove(null, null, null) ;
>> Model.removeAll() ;
>>
>>
>>   @Minto is the patch for CLEREZZA-726 ready to be applied or are you
>>> working
>>> on an update version?
>>>
>>>
>>>
>>>> TDB caches databases - it's expensive to keep opening and closing them
>>>> (no
>>>> disk caching benefits) and tracking whether its in use in Java isn't very
>>>> helpful.  You can flush the cache but that will not avoid the windows
>>>> issue.
>>>>
>>>>
>>> It could close the file when close on the graph is explicitely invoked?
>>>
>>
>> One graph != one file.  It's a groups of files for all graphs in the
>> dataset.  A named graph is a view of the quad table.
>>
>>
>>   Besides the window issue unix systems aren't happy either if they have to
>>> keep unsued filehandler open.
>>>
>>
>> If you flush the cache, the database is closed.
>>
>>
>>>
>>>
>>>> Outside testing, don't delete a database while running.
>>>>
>>>>
>>> So in tests there's apossibility I can remove the database files?
>>>
>>
>> Yes - the TDB test suite does - on Windows it creates a new directory for
>> each test using a unique name generator.  They go away when the JVM exits
>> but only then.
>>
>
> Seems to be the only way. Tried flushing the dataset, the graph, deleting
> all the triples yet the files remain in use.

Deleting the graph is removing from the quad table.  The database is 
separate from the view (graph).  With transactions changes, especially 
many small transactions, get batched up so that changes can actually 
happen faster than non-transaction mode.

	Andy

>
> Changing the filemode in a static initializer seems to do the trick. Not
> sure what the performance implications are. But for perfromance the
> SingleTdbDatasetTcProvider should be used anyway.
>
> Cheers,
> Reto