sis-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Desruisseaux (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SIS-76) Remove (for now) the internal ThreadPoolExecutor
Date Sun, 06 Jan 2013 16:24:12 GMT

    [ https://issues.apache.org/jira/browse/SIS-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13545432#comment-13545432
] 

Martin Desruisseaux commented on SIS-76:
----------------------------------------

The purpose of this task is mostly to keep trace of the commit revision numbers, so we know
where to find back the original code if we want to revert to executors:

* JDK7 branch: 1429547
* JDK6 branch: 1429549
* trunk: 1429550

Using the lightweight approach instead of executor, we observed approximately 20% of performance
increase in the {{CacheTest.stress()}} test. Before to revert to executor (if we want to do
that in a future version), we will need to benchmark the application in order to make sure
that there is really something to gain.

                
> Remove (for now) the internal ThreadPoolExecutor
> ------------------------------------------------
>
>                 Key: SIS-76
>                 URL: https://issues.apache.org/jira/browse/SIS-76
>             Project: Spatial Information Systems
>          Issue Type: Task
>          Components: Utilities
>            Reporter: Martin Desruisseaux
>            Assignee: Martin Desruisseaux
>            Priority: Minor
>
> SIS needs a thread for executing short tasks after some (potentially zero nanosecond)
delay. This thread is reserved to internal SIS usage, mostly resources disposal. We tried
to use the {{java.util.concurrent}} package in a previous version, especially {{ScheduledThreadPoolExecutor}},
but those executors seem more suitable to heavier tasks in applications controlling their
own executor. Indeed, the Javadoc of the {{ForkJoinPool}} executor said "_Normally a single
{{ForkJoinPool}} is used for all parallel task execution in a program or subsystem. Otherwise,
use would not usually outweigh the construction and bookkeeping overhead of creating a large
set of threads._" So we may be better to let users create their own executor, and potentially
leverage it in SIS (this strategy is yet to be determined).
> It seems difficult to instantiate an executor for casual SIS use without wasting resources.
For example {{ScheduledThreadPoolExecutor}} acts as a fixed-sized pool, thus forcing us to
use only one thread if we don't want to waste resources (profiling shows that even a single
thread has very low activity for the SIS tasks that we need to submit in background). The
{{ThreadPoolExecutor}} super-class is more flexible but still have a quite aggressive policy
on threads creation, and doesn't handle delayed tasks by itself.
> We could try to combine both worlds with a {{ThreadPoolExecutor}} using a {{DelayedQueue}},
but it forces us to declare a core pool size of 0 otherwise {{ThreadPoolExecutor}} tries to
execute the tasks immediately without queuing them. Combined with the {{DelayedQueue}} characteristics
(being an unbounded queue), this result in {{ThreadPoolExecutor}} never creating more than
one thread (because it waits for the queue to reject a task before to create more threads
than the pool size).
> Given that it seems difficult to configure {{(Scheduled)ThreadPoolExecutor}} in such
a way that two or more threads are created only when really needed, given that using those
thread pools seems an overkill when the pool size is fixed to one thread, given that our profiling
has show very low activity for that single thread anyway, and given that we do not need cancellation
and shutdown services for house keeping tasks since they are executed in a daemon thread,
a more lightweight solution seems acceptable here. Se we could remove the internal SIS executor,
use a single thread as we do for {{ReferenceQueueConsumer}}, and wait to see later if we need
an executor.
> h2. Future evolution
> We may revert to an executor in a future SIS evolution if we happen to need an executor
anyway. However it may be better to wait and see what are the executor needs. Setting up an
executor implies choosing many arbitrary parameter values like the number of core threads,
maximum threads, idle time, queue capacity, _etc._ Furthermore some platforms (e.g. MacOS)
provide OS-specific implementations integrating well in their environment. We may want to
let the user provides the executor of his choice, or we way want to have more profiling data
for choosing an appropriate executor. But we would need to find some way to give priority
to SIS tasks, since most of them are for releasing resources - in which case quick execution
probably help the system to run faster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message