river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Firmstone <j...@zeus.net.au>
Subject Re: Space/outrigger suggestions (remote iterator vs. collection)
Date Wed, 19 Jan 2011 11:17:31 GMT
Then you end up with something like MatchSet, so it would seem someone 
already solved the problem.  Perhaps the name "Set" might have been a 
confusing choice.

Just think of MatchSet like an Iterator, without the hasNext() call.

The caller can also terminate when you've found the result you want.



Peter Firmstone wrote:
> Dan Creswell wrote:
>>> When access large disk records and pulling these into memory, you 
>>> typically
>>> only take a small chunk, process it and move on, so the behaviour is 
>>> more
>>> stream like, rather than iterator, I created an interface called
>>> ResultStream (yes it supports Generics, but beware the compilation
>>> boundaries that haven't been checked by the compiler), result stream is
>>> terminated by a null value.
>> That's generally similar to how MatchSet works under the covers....
>>  <snip>
>>> package org.apache.river.api.util;
>>> /**
>>> * This interface is similar to an Enumerator, it is designed to return
>>> * results incrementally in loops, however unlike an Enumerator, 
>>> there is no
>>> * check first operation as implementors must return a null value after
>>> * the backing data source has been exhausted. So this terminates like a
>>> stream
>>> * by returning a null value.
>>> *
>>> * @author Peter Firmstone
>>> */
>>> public interface ResultStream<T> {
>>>   /**
>>>    * Get next T, call from a loop until T is null;
>>>    * @return T unless end of stream in which case null is returned.
>>>    */
>>>   public T get();
>> So this overall, isn't far from where MatchSet goes. For those following
>> along, remoteness issues are such that we can't reliably return null to
>> indicate "end of stream". One may never reach the end of the stream if
>> there's a failure at the back-end. One could counter that with a  
>> "wait for
>> the failure to go away" scenario but that assumes the failure does go 
>> away
>> and it might not (and whilst you're figuring all that out your thread is
>> blocked and can't say much to your upstream users).
>> In essence, then returning null means "we successfully reached the 
>> end of
>> the stream". And we need some other mechanism to say "we didn't reach 
>> the
>> end of the stream for some reason". Typically the mechanism is an
>> exception....
> Yeah, you'd need to throw an IOException, I've pondered adding it, so 
> it can be used over remote connections, that way another interface can 
> implement both Remote and ResultStream.
> Thanks, I think that was all the convincing I needed.
> ResultStream was originally created for Parallel iteration, multiple 
> threads calling get, you can't check first, you have to check after ;)
> Cheers,
> Peter.

View raw message