incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paolo Castagna (JIRA)" <>
Subject [jira] Commented: (JENA-44) Support external sorting of bindings in ARQ
Date Tue, 15 Feb 2011 15:56:57 GMT


Paolo Castagna commented on JENA-44:

Indeed, it would be beneficial to be able to cancel a large sort (even when it is an external
JENA-29 adds cancel, but if cancel is called while ARQ is performing a large in-memory sort,
the cancel will not work until the sort has finished (or the machine has crashed or become
unresponsive doing GC). We have experience these sort of problems.
Therefore, JENA-29 and JENA-44 are complementary and they could benefit from each others.

We are already seeing the benefit from JENA-44, since it put a cap, in practice, on the amount
of memory used by ARQ to do large sorts.
Moreover, JENA-44 could use same mechanism to serialize/deserialize bindings from JENA-45,
if that is going to go intro trunk before JENA-44.

> Support external sorting of bindings in ARQ
> -------------------------------------------
>                 Key: JENA-44
>                 URL:
>             Project: Jena
>          Issue Type: New Feature
>          Components: ARQ
>            Reporter: Sam Tunnicliffe
>            Priority: Minor
>         Attachments: JENA-44-0.patch
> In QueryIterSort, the sorting of the contents of an Iterator<Binding> is done in
memory, using Arrays.sort. This can be problematic where the set to be sorted is large. A
possible solution could be to use an external, disk-backed algorithm. A hybrid approach may
be better, whereby we attempt the in-memory sort, but when the number of bindings encountered
goes over a certain number, resort to the disk-backed variant.

This message is automatically generated by JIRA.
For more information on JIRA, see:


View raw message