openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fernando Padilla <f...@alum.mit.edu>
Subject Re: slices hangs if run with Multithreaded=true
Date Fri, 12 Dec 2008 23:34:37 GMT
oh right. I forgot about the strict boundaries between EntityManagers.. 
darn.

So I suppose we just can't support ParallelExecutor, that's the easy 
fix, just not at all the most performant :(

We might be able to tweek things to remove the "query corruption" bug 
that I was seeing, but I suppose we can't guarantee no concurrency 
issues until Multithreaded=true.  And we can't do Multithreded=true 
until we make all locks a lot more granular, which will take some time.

So it looks like Slices is moving forward, but if it's going to be 2-3 
times slower than normal.. that's a pretty big price to pay for 
scalability.. hmmm



Unless we can do some acrobatics with locks around that code, so that 
the Parent Thread/DistributedStoreQuery could delegate the lock to the 
sub queries.. like can you do a selective release of locks to only a 
select few? :)



Pinaki Poddar wrote:
> <crazy-brainstorm>
> Another option for Slices, is to move up the 'distributed' level, so 
> that it maintains unique brokers or entity managers per database..   At 
> that point it could be a JPA Slices implementation, independent of 
> OpenJPA :)
> <>
> 
> This option had been considered. The architectural issue with this approach
> is the following:
> A JPA EntityManager represents a Persistence Context which not only has a
> unit of work semantics, it also displays a group like behavior. For example,
> an entity instance X managed by EntityManager E1 can not be simultaneously
> managed by another EntityManager E2. Also, X can not refer to an Y that is
> managed by E2. Hence, a design that spawns E1,E2,...,En connected to Slice
> S1, S2, ...Sn are essentially managing separate contexts with water-tight
> boundaries. 
> Fo example, under such a design, one can not issue "select p from Person p
> order by p.name" and get all the Persons from all the slices in the same
> persistence context.
> 
> 
> 
> 
> Fernando Padilla wrote:
>> yes I know.
>>
>> I was just walking through the code, to see what I could see, and all 
>> StateManagers use the global Broker lock.. so no attempt at granularity 
>> in many places :(
>>
>> Also, breaking the lock into Read/Write might give multithreaded use a 
>> little more performance as well..
>>
>> like you said, this is "battle tested code", which I am assuming is 
>> probably like 4 years old ( from the old kodo days ).  So I could 
>> understand the hesitation to not change it, but .... you know ... :)
>>
>>
>>
>> ps -
>> <crazy-brainstorm>
>> Another option for Slices, is to move up the 'distributed' level, so 
>> that it maintains unique brokers or entity managers per database..   At 
>> that point it could be a JPA Slices implementation, independent of 
>> OpenJPA :)
>>
>> It would be an DistributedEntityManager that would delegate to sub 
>> EntityManagers, and return objects managed by the 
>> DistributedEntityManager, but that behind the scenes would delegate to 
>> objects owned and managed by sub EntityManagers..
>>
>> So it would be like creating a JPAStoreManager that would store things 
>> into a sub entitymanager :) etc etc
>> </crazy-brainstorm>
>>
>>
>>
>> Pinaki Poddar wrote:
>>>> Since behind the scenes you are executing various different queries
>>>> using
>>> multiple 
>>>> threads, and they might be touching/using shared resources lower down. 
>>> Good point, and most probably you are right.
>>>
>>>> which is to fix up QueryImpl, not to do aggressive locking (at least in
>>>> the isUnique method), and I 
>>>> think it will at least remove the current blocking issue.
>>> So far, Slice has imposed almost no demand on existing kernel -- which is
>>> battle hardened code. So naturally, I am apprehensive of change in
>>> threading
>>> logic in kernel.QueryImpl. Any change of this nature *must* validate
>>> itself
>>> against a set of multithreaded tests (with or without Slice).
>>>
>>
> 

Mime
View raw message