apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Deshmukh <sand...@datatorrent.com>
Subject Re: Best way to trigger start reading directory
Date Thu, 03 Dec 2015 03:19:22 GMT
You use StatsListener shared between two operators to trigger this.

Regards,
Sandeep

On Thu, Dec 3, 2015 at 8:08 AM, Isha Arkatkar <isha@datatorrent.com> wrote:

> Hi,
>
>     Yes, that should work. But suppose first operator rolled back to
> previous checkpoint due to some fail over, the state of application would
> be reset. Except the part that the 2nd file reader which was not emitting
> anything in those windows, now will continue to emit tuples. That will make
> the state inconsistent.
> May be I can create the link from downstream operator after first reader is
> done to handle that.
>
> Thanks,
> Isha
>
> On Wed, Dec 2, 2015 at 5:02 PM, Munagala Ramanath <ram@datatorrent.com>
> wrote:
>
> > One way (a bit hacky) is to have the 2nd operator monitor an empty
> > directory. Then, when the
> > 1st is done reading, it sends a control tuple to a "file-link" operator;
> > that operator creates
> > symbolic links from the directory monitored by the 2nd operator to the
> > actual input files.
> > That should then trigger the 2nd input operator to do its thing.
> >
> > Ram
> >
> > On Wed, Dec 2, 2015 at 4:43 PM, Isha Arkatkar <isha@datatorrent.com>
> > wrote:
> >
> > > Hi all,
> > >
> > >    I have an application with 2 input file reader operators. In this
> > case,
> > > want to trigger start reading from 2nd input location, only after 1st
> > > operator is done reading. What is the best way to do this?
> > >
> > > Thanks!
> > > Isha
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message