river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: com.sun.jini.thread lock contention
Date Sun, 06 Jun 2010 01:12:25 GMT
In summary, it is possible to create a new better TaskManger that takes 
advantage of new concurrency libraries, but it cannot be directly 
replaced by ThreadPoolExceutor, which is what Gregg is trying to say.

Fundamentally, we must learn everything about what we have and 
understand it, in order to improve or replace it.  I think we all are in 
the process of understanding, but we must be careful not to be too 
critical of each other, and be mindful not to hold our ground too 
steadfast either, to remain humble, so the list doesn't descend into 
argument.

I also think Chris has gotten to the heart of the matter:
> The big feature that TaskManager has that Java 1.5 lacks is the ability
> to reorder the tasks by asking each task if it wants to be before the
> others (Task.runAfter()).  That could be refactored into a custom
> BlockingQueue implementation, I suppose.
>
> Chris
It is interesting TPE doesn't create extra threads until the queue is 
full. 

Reading the source code from TaskManager it is apparent, maxThreads does 
in fact limit the total number of threads in TaskManager and I was 
previously mistaken.

This is taken directly from ThreadManager:

    /** Add a new task. */
    public synchronized void add(Task t) {
    tasks.add(t);
    boolean poke = true;
    while (threads.size() < maxThreads && needThread()) {
        Thread th;
        try {
        th = new TaskThread();
        th.start();

<comment> A new TaskThread (which is a Thread) is created every time the 
available threads is less than maxThreads AND a new thread is needed!  
So even if a new Thread is needed, we cannot have more than 
maxThreads</comment>

    /** Return true if a new thread should be created (ignoring 
maxThreads). */
    protected boolean needThread() {
    int bound = (int)(loadFactor * threads.size());
    int max = tasks.size();
    if (max < bound)
        return false;

<comment> Above: This first piece of code shows that the load factor 
influences the creation of new threads, in this case, the upper bound is 
the load factor * the number of threads currently in use.  So you must 
have more tasks than threads * load factor, to get past this check.  
This is similar to having the full Queue in TPE.</comment>

    max--;
    if (runAfter((Task)tasks.get(max), max))
        return false;

<comment> Above: If the last task must be run after some other task 
preceding it in the task list, return false.</comment>

    int ready = firstPending + 1;
    if (ready > bound)
        return true;

<comment> Above: First pending is incremented every time a task is found 
ready to run using "private boolean takeTask() " and it is decremented 
every time a task is "public void run()"  This may influence the 
creation of a new thread, if there are more ready tasks than total 
threads * loadFactor. </comment>

    for (int i = firstPending; i < max; i++) {
        if (!runAfter((Task)tasks.get(i), i)) {
        ready++;
        if (ready > bound)
            return true;
        }

<comment> Above this, piece of code loops over the tasks between 
firstPending and the maximum bound of the tasks List.  ready is 
incremented every time a ready task is found.  Then if there are more 
tasks ready than the total number of threads * loadFactor, a new thread 
will be created.

    }
    return false;
    }

Gregg Wonderly wrote:
> I am not against the thoughts here.  TaskManager has a very important 
> roll that has specific needs.  It is used to dispatch inbound calls 
> amongst other things.  It is essential that it never fail to create a 
> thread to handle an inbound call.  Without this behavior there is a 
> chance for distributed deadlock.
>
> The key clause in TPE is that it says the extra threads are not 
> created until the queue is full.  That is another form of "maximum".
>
> If it has a documented behavior that has no maximum, then I am okay 
> with using it.  I have just never seen it behave correctly based on 
> what I have experienced and so I am concerned about the maximum thread 
> issue.
>
> There are of course many types of applications that can use a ceiling 
> on maximum threads to throttle things.  But if you look at some of the 
> bug reports and discussions on the concurrency-interest list, there 
> are cases of this behavior popping up in the fork-join pool stuff too, 
> where the complexity of hoe work is divided and distributed creates 
> problems.
>
> Gregg Wonderly
>
> Sent from my iPhone
>
> On Jun 5, 2010, at 8:25 AM, Dennis Reedy <dennis.reedy@gmail.com> wrote:
>
>>
>> On Jun 5, 2010, at 609AM, Patrick Wright wrote:
>>
>>> One point I'd like to raise about using java.util.concurrent and TPE:
>>> I think that over the long term, it makes sense to (re)use existing
>>> utilities which are being maintained by domain experts rather than
>>> custom utilities you've written yourself. The concurrent libraries
>>> available since Java 5 were written and maintained by people widely
>>> recognized to be very, very good at a very hard problem. That doesn't
>>> mean they, or the library, is perfect, just that there is value in
>>> building on their work and letting them take care of the bugs and
>>> optimizations over time. The downside would be that if a River user
>>> was stuck with, say, Java 5, they couldn't take advantage of bugfixes
>>> or improvements in Java 6. On the other hand, that's true of the
>>> entire JDK.
>>>
>>> The max threads issue seems to me a non-issue. A JVM can allocate only
>>> so many native threads before it runs out of OS resources; that's a
>>> hard limit. You can set a max of Integer.MAX_VALUE but your VM would
>>> die long, long before it reached that.
>>>
>>> For me this is more of design policy decision. Re-use, intelligently
>>> and selectively, where possible, to reduce your project's workload.
>>>
>>>
>>> Patrick
>>
>> +1
>


Mime
View raw message