nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Kawamura <ijokaruma...@gmail.com>
Subject Re: Re: A newcomer Question about nifi use
Date Tue, 27 Jun 2017 05:56:22 GMT
Hi Hollis,

If you haven't done yet, I'd strongly recommend to take a look on the NiFi
User Guid doc [1] to get familiar with how to build a NiFi flow.
In case you prefer video, there's a great series of how-to [2].

Then you'll understand how to address those warning messages shown on NiFi
UI.

Once you got basic idea how NiFi flow works, you will need to consider
"When the remote shell should be executed", because NiFi is more of a
streaming data processing application rather than an batch oriented
scheduled ETL tool.
For example, if you'd like to call the remote script after putting certain
files from local to HDFS, then you can build a NiFi flow looks like:
GetFile -> (MergeContent, optionally to create larger files) -> PutHDFS ->
ExecuteStreamCommand

There are different processors those can be used to trigger actions
(ListXXX, ListenXXX, ConsumeXXX ... etc) for different use-cases.

[1]
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#building-dataflow
[2]
https://www.youtube.com/watch?v=Y5znvcJ_NWo&list=PLHre9pIBAgc4e-tiq9OIXkWJX8bVXuqlG

Thanks,
Koji


On Tue, Jun 27, 2017 at 12:45 PM, huipanc@163.com <huipanc@163.com> wrote:

> Hi Koji,
>        Thank you much very. I use ExecuteStreamCommand to execute remote
> ssh shell. But I don't know how to run a full  flow. My problem is
> described as below. Hence what is 'Relationship output strem',
> 'Relationship original' and 'upstream Connections'. I cannot find any
> reference in nifi api document till now.
>        My aim is to execute ssh shell  in my hdfs machine.Can you give me
> a fully example. Thanks.
>
>
> ------------------------------
> Thanks,
>   Hollis
>
> *From:* Koji Kawamura <ijokarumawak@gmail.com>
> *Date:* 2017-06-26 16:49
> *To:* users <users@nifi.apache.org>
> *Subject:* Re: A newcomer Question about nifi use
> Hi Hollis,
>
> To execute a remote shell using ssh, I'd use 'sh' command and its -c
> option with ExecuteStreamCommand or ExecuteProcess.
>
> For example, ExecuteStreamCommand configuration would look like:
> - Command Path: sh
> - Command Arguments: -c;ssh user@host.example.com
> command-to-execute-on-remote
> - Argument Delimiter: ;
>
> Thanks,
> Koji
>
> On Mon, Jun 26, 2017 at 5:20 PM, huipanc@163.com <huipanc@163.com> wrote:
> > Hi,
> >      I am a newcomer of using nifi. I mean to use nifi to execute remote
> ssh
> > shell. So I want to query there is a possibilty to use nifi to execute
> > remote ssh shell (just like using oozie ssh to excute remote ssh shell).
> >
> > ________________________________
> > Thanks,
> > Hollis
>
>

Mime
View raw message