incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Seaborne <>
Subject Re: JENA-44, JENA-45 etc - common Binding I/O
Date Thu, 23 Jun 2011 16:59:43 GMT

On 23/06/11 17:07, Paolo Castagna wrote:
> Hi Andy,
> first of all, thanks for this.
> Re: JENA-44... what is blocking JENA-44 going into trunk is just the
> lack of a
> common way to serialize binding. By the way, we are using a patched
> version of
> ARQ on some of our servers (with no problem and improvements in terms of
> stability, RAM consumption in particular when users submit queries which
> need
> to sort large resultsets and they timeout).
> So, all this is more than welcome from my point of view (i.e. one patch
> less
> to manage).

Have you looked at the DeferredFileQueue / ThresholdPolicy code in 
JENA-45?  This is another area of commonality.  Any thoughts about 
DataBag from Pig?  (JENA-44, comment 24/May/11, pt 3 - this mikght be 
too much for this round).

There are various settable paramters - what makes a difference? 
especially writeBufferSize.

I didn't notice how cancellation would stop executors, only clear up 
afterwards.  What about a volatile flag?

>> VARS ?x ?y .
>> Set the variables in force for subsequent rows,
>> until the next VARS directive.
>> We need VARS because it's not always possible to determine all
>> the possible variables before starting to write out bindings.
> This is not completely clear to me. An example of when it's not possible
> to determine all the possible variables before starting to write out
> binding
> will probably convince me and help me to clarify.

Support you have an Iterator<Binding> from a LeftJoin or a Union.  One 
way is to statically determine the variables, the other is to be relaxed 
and output based on the Bindings seen.  Static analysis requires the 
info to be passed from query execution into, for example, the heart of

The first might have ?x, ?z, the second ?x, ?y, ?z from an OPTIONAL. 
The separation of the code from the static analysis

If you set it once at the start, that also works.

And you can concat streams.


View raw message