nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Balaji K Hari <balaji.h...@capgemini.com>
Subject Clarifications/Suggestions on Using NIFI.
Date Thu, 21 Apr 2016 12:40:34 GMT
Hi Joe/Team,

As per your suggestion, we have one specific use case to be developed using NIFI and need
your assistance on this requirement to be achieved.


1)     We are trying to pull the data from multiple RDBMS databases and  putting in to Amazon
S3 using NIFI. When we have done this, we found that the database data would be put into NIFI
local first and then put the same into S3.

However we want to know the direct way, where the data would directly pulled from database
and put into S3. So could you please suggest any processor where we can achieve this. Please
find the diagram attached.


2)     Also when we configure the NIFI, we observed that it is a Single User instance i.e.
only 1 user can work on NIFI and if any other users tries to do anything it will reflects
in the initial user’s instance. Can NIFI be configured as a multi-user instance where all
the users can work on it at a time?



3)     I am trying to email notifications using “PutEmail” processor Please explain the
parameters to be put in the “PutEmail” Processor. The first parameter it asks is “SMTP
hostname”. If I want to send email from my gmail id to yahoo id. Should I put SMTP hostname
of gmail or Yahoo ? Also where can I get details of SMTP hostnames and port number if this
processor is been executed from the client network?

Looking forward for your valuable inputs/suggestions.

Regards,
_______________________________________________________________________
[Email_CBE.gif]Balaji KNV_Hari
Technical Architect

From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Thursday, April 14, 2016 7:40 PM
To: dev@nifi.apache.org; Balaji K Hari
Subject: Re: Clarifications/Suggestions on Using NIFI.



On Thu, Apr 14, 2016 at 10:09 AM, Joe Witt <joe.witt@gmail.com<mailto:joe.witt@gmail.com>>
wrote:
Hello

You received a near immediate response which you can find here [1].  But since you are not
subscribed to the mailing list you did not receive the response.  Please subscribe here [2].

[1] http://mail-archives.apache.org/mod_mbox/nifi-dev/201604.mbox/%3CCALJK9a6h0qv14%3DyqN4mqRDJQYaimwO1V1DviqFntt0Xy2dTYQQ%40mail.gmail.com%3E
[2] https://nifi.apache.org/mailing_lists.html

Thanks
Joe

On Thu, Apr 14, 2016 at 10:03 AM, Balaji K Hari <balaji.hari@capgemini.com<mailto:balaji.hari@capgemini.com>>
wrote:
Hi Dev Team,

Good Morning!!

I hope you are trying hard by taking time from your busy schedule to provide the inputs for
the below questions.

It would be really helpful, if you can answer these at earliest as based on this we want to
try with other options to achieve the functionalities as per requirements.

Thanks a lot for your assistance… Looking forward for your reply.

Regards,
_______________________________________________________________________
[Email_CBE.gif]Balaji KNV_Hari
Technical Architect

From: Balaji K Hari
Sent: Tuesday, April 12, 2016 7:57 PM
To: 'dev@nifi.incubator.apache.org<mailto:dev@nifi.incubator.apache.org>'
Subject: Clarifications/Suggestions on Using NIFI.
Importance: High

Hi Team,

Based on the project requirements, I was looking at different features included in Apache
NIFI and found that this would be the good way to interact with the Development team who have
developed NIFI and are looking for suggestions/inputs from the User community to improvise
the product and also it is a great medium where the users who are using this NIFI would get
the valuable inputs from the developers for their requirements.

Need your assistance/inputs on the below requirements and how these can be implemented in
NIFI to achieve the solution.


==>  I have observed that, Event Based Scheduling/Any Trigger Based Scheduling is yet to
be included in the latest NIFI product. Any workarounds/alternatives to achieve this?

==>  Can Spark/Hive Jobs can be scheduled on time basis and also executed through NIFI?
If Yes, please suggest how can we do this?

==>  Can we get the data from multiple tables of Oracle/SQL Server/Teradata and put directly
in S3/HDFS and also directly to RedShift/Any database? If Yes, please suggest how can we do
this?

==>  Also can we do the transformations/manipulations on the data while moving it to S3/HDFS
from RDBMS databases? If Yes, please suggest how can we do this?

==>  Can we do the validations and also find the duplicate data/records before you put
the data into S3/HDFS. For example, I have moved the data from RDBMS tables into S3 and as
part of daily loads, I need to check whether any duplicate records are present in the new
load and need to remove those records while data movement itself. Please provide your inputs
how can we do this?

==>  Also can you provide valuable inputs on how can we achieve the workflow execution
 dependency i.e. For example, I have designed one workflow and based on this 1st workflow
execution completion, I need to start the second workflow else need to start another workflow.
Can this be achieved in NIFI?

It would be really helpful and appreciated on the above inputs, as you would be the best team
who can help the us the solutions/workarounds in using the NIFI product as it is been identified
as a good user friendly product for Data Ingestion/movement.

Looking forward for your reply with the requested suggestions and solutions. Thanks in Advance!!!!
☺☺

Regards,
_______________________________________________________________________
Balaji KNV_Hari
Technical Architect


This message contains information that may be privileged or confidential and is the property
of the Capgemini Group. It is intended only for the person to whom it is addressed. If you
are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate,
distribute, or use this message or any part thereof. If you receive this message in error,
please notify the sender immediately and delete all copies of this message.


Mime
View raw message