openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fernando Padilla <f...@alum.mit.edu>
Subject Re: slices hangs if run with Multithreaded=true
Date Fri, 12 Dec 2008 21:21:25 GMT
Well, I was talking about SELECT queries, not even queries dealing with 
transactions.  So you might have come up with another issue with the 
ParallelExecutor.  And at the moment, I don't use any JTA tx, so I have 
no experience with this.  And my use of Slices is for 
performance/scale-out reason, so I will not be using distributed 
transactions, because that might negate the scalability gains.. (and 
most of the commits that I do will only involve one database, not many).


background:
In Slices, there is the Parent DistributedStoreQuery, which delegates to 
a number of sub Queries that are actually tied to real databases.  The 
ParallelExecutor executes the sub Queries using a Thread pool, allowing 
all queries to execute concurrently, which is a very good thing as the 
number of slices you're running against goes up..



David Jencks wrote:
> Not sure if this is relevant here.... for an enterprise environment 
> using JTA tx, would this multithreaded slice execution involve using the 
> same JTA tx on multiple threads?  This is generally not supported in 
> existing transaction managers although I think there's a possible 
> workaround/implementation using separate tx branches for each thread.  
> Talking with other tm designers there doesn't seem to be enough demand 
> for this multithreaded tx feature to support implementing it.
> 
> thanks
> david jencks
> 
> On Dec 12, 2008, at 12:34 PM, Fernando Padilla wrote:
> 
>> Alright Pinaki.
>>
>> So Slices Parallel Executor seems to be the root cause of the 
>> "corrupted queries" bug (OPENJPA-820).  My theory is that internally 
>> the Parallel Executor uses OpenJPA resources in a Multithreaded 
>> manner, but having Multithreaded=false might open it up to some 
>> concurrency issues..
>>
>> You already know that I tried to test it by turning on 
>> Multithreaded=true, but then ran into that hanging/blocking bug.
>>
>> So I simply changed the ExecutorService, to be a dumb one that only 
>> executes the tasks using the thread that submitted the task.. (so 
>> always uses the Parent Thread, so Reentrant Locks work properly) (I'll 
>> attach it so you see what I'm talking about ).
>>
>> And so far, I do not see any of the "corrupted" queries that I was 
>> talking about.  But also (it might be my imagination), but it seems 
>> much slower too. :)
>>
>>
>> So... do you agree?  Shall we look at dismantling the ParallelExecutor 
>> from Slices for now ( and take the slower performance )?  While we 
>> also look at getting ParallelExecutor working with 
>> Multithreaded=true.  And also look at making OpenJPA multithreaded 
>> support be more efficient, because it will have to be on for 
>> ParallelExecutor to work, etc etc..
>>
>>
>>
>> Pinaki Poddar wrote:
>>> This may be a tough one and thank you for sending the blocking thread 
>>> stack.
>>> The QueryContext is shared by the parallel queries. The stack points out
>>> that query context is deadlocked as Executors calling QueryContext 
>>> (which
>>> has its own lock) method on different threads.   Slice has not 
>>> imposed any major change on kernel's behavior (on threading
>>> or otherwise). But this current limitation might. The easy (and 
>>> escapist)
>>> way out is to fall back on sequential query execution when
>>> openjpa.Multithreaded is set.   Thoughts?
> 

Mime
View raw message