streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Douglas <>
Subject Re: LocalStreamBuilder#stop()
Date Wed, 08 Oct 2014 15:55:49 GMT
Hi Ryan,

I think the most important thing is to ensure that anytime "stop()" is
called, the behavior is deterministic. When that method is called, we need
a guarantee that things will completely shut down in a reasonable amount of
time. However, I don't like the idea of just abandoning datums that are
already in the pipeline. If it's possible, I'd prefer to go with an
approach similar to option 2b. Completely shutdown the providers, and give
the Stream some time to attempt to process all currently queued datums.
However, after a certain period of time, I think that a forceful shutdown
needs to be executed regardless of the state of the pipeline.

-- Robert Douglas

On Wed, Oct 8, 2014 at 10:31 AM, Ryan Ebanks <> wrote:

> I've been added tests and fixing some issues in the local stream builder
> class.  I need some input on how the stop() method should work so I can
> write the appropriate tests.
> Option 1.) Gentle Shutdown
> Immediately stop the providers from producing more data, but allow for all
> data currently in the stream to finish processing.  Shutdown time would be
> unknown as a result.
> Option 2.) Forceful Shutdown
> Immediately stop the providers from producing more data, and stop all
> processors/writers.  Data will be left in the pipeline will be discarded.
> shutdown will be immediate
> Option 2b) Forceful Shutdown after small wait
> Immediately stop providers from producing more data.  Give the
> processors/writers a small window to try to finish processing (< 10secs),
> and if not finished, forcefully shutdown all processors/writers.
> These options can be changed in the future as more features are added.  For
> example once datum tracking is implemented, we will probably want a
> combination of option 1 and 2b.  But we need a defined way for it work now,
> so tests are testing the correct functionality.
> -Ryan Ebanks

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message