nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: Reg: Tailing a remote file
Date Thu, 21 Apr 2016 12:48:23 GMT
Sourav,

If what is behind your '*' has some format logic (like dates), you could
try using expression language.
Otherwise there is an open JIRA for this: NIFI-1170.



2016-04-21 14:42 GMT+02:00 Sourav Gulati <sourav.gulati@impetus.co.in>:

> Simon,
>
> I see an issue with tail processor .
>
> I have to give the exact name of the file to tail. I can give something
> like /home/user/abc*.log
>
> Is there anyway I can do this?
>
> Regards,
> Sourav Gulati
>
> -----Original Message-----
> From: Simon Ball [mailto:sball@hortonworks.com]
> Sent: Thursday, April 21, 2016 3:40 PM
> To: dev@nifi.apache.org
> Subject: Re: Reg: Tailing a remote file
>
> If you're using a rolling file name pattern you will want to tail the
> current log file, and essentially ignore the rolled history, unless you
> need to represses or miss data.
>
> TailFile is a local processor, so you would run a nifi agent on the box
> that has the files on, use tail file -> remote process group -> input port
> on core nifi instance -> wherever your data is going next.
>
> The local/remote setting comes up when clustering nifi and using something
> like an NFS mount to get to the files being tailed, which is a different
> way of doing this. The remote will lead to state about where you've gotten
> to being stored differently to support the cluster continuing to tail from
> alternative nifi cluster nodes.
>
> Simon
>
>
> > On 21 Apr 2016, at 11:38, Sourav Gulati <sourav.gulati@impetus.co.in>
> wrote:
> >
> > Hi Simon/Team.
> >
> > Please provide some good example of tail processor.
> >
> > I need some example for " Rolling Filename Pattern".
> >
> > Also, as per my knowledge TailFile processor is used for tailing local
> file then what does location attribute signifies, if I set it to remote.
> >
> > Regards,
> > Sourav Gulati
> >
> > -----Original Message-----
> > From: Simon Ball [mailto:sball@hortonworks.com]
> > Sent: Wednesday, April 20, 2016 2:27 PM
> > To: dev@nifi.apache.org
> > Subject: Re: Reg: Tailing a remote file
> >
> > Site-to-site is in pretty much every version of nifi released under
> apache.
> >
> >> On 20 Apr 2016, at 10:48, Sourav Gulati <sourav.gulati@impetus.co.in>
> wrote:
> >>
> >> Thanks Simon. Which version of Nifi support site to site?
> >>
> >> Regards,
> >> Sourav Gulati
> >>
> >> -----Original Message-----
> >> From: Simon Ball [mailto:sball@hortonworks.com]
> >> Sent: Wednesday, April 20, 2016 1:28 PM
> >> To: dev@nifi.apache.org
> >> Subject: Re: Reg: Tailing a remote file
> >>
> >> The best way to do this would be to run a nifi agent on the remote
> system, using the TailFile processor, and then use site-to-site to get that
> into your core nifi.
> >>
> >> Simon
> >>
> >>
> >>> On 20 Apr 2016, at 09:49, Sourav Gulati <sourav.gulati@impetus.co.in>
> wrote:
> >>>
> >>> Hi Team,
> >>>
> >>> We want to process logs from a file which is getting updated
> continuously on remote system.
> >>>
> >>> What is the best way to do that in NiFi?
> >>>
> >>> Regards,
> >>> Sourav Gulati
> >>>
> >>>
> >>> ________________________________
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
> >>
> >> ________________________________
> >>
> >>
> >>
> >>
> >>
> >>
> >> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
> >
> >
> > ________________________________
> >
> >
> >
> >
> >
> >
> > NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
> >
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message