nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Bateman <r...@windofkeltia.com>
Subject Re: Transformations using Nifi
Date Thu, 12 Oct 2017 14:50:35 GMT
Aruna,

I don't think there is any generalized NiFi training course yet. I 
started writing a book a year ago, but it's a pretty thankless task and 
takes a lot of time. I mostly write custom processors so I tend to see 
the world differently from most "user" users of NiFi. When I answer 
questions, which I don't do too much, I often tell an impractical story 
that doesn't answer the question asked.

Now, I haven't done much database work at all, so I'm not going to be 
too much help to you. However, looking at your questions, particularly 
the one about obtaining the current date when the flow is run (#2) and 
also #3, I would suggest that you study the NiFi Expression Language. 
This would allow you to insert the "now" date into your flow at some point.

The other suggestions I have are to experiment, which I'm sure you're 
doing, and Google hard for help. For this forum, which is filled with 
very nice and helpful people, you'll tend to get a lot more and better 
help if you come in with a very specific question rather than a list of 
things or a general, "help me" sort of plea.

Cheers,

Russ

P.S. NiFi absolutely rocks, which you'll see as soon as you get over 
your initial hump here. But, you're on the right track.


On 10/12/2017 08:16 AM, Aruna Sankaralingam wrote:
>
> Hi,
>
> Could you someone please help me with these requirements in the email 
> below?
>
> Thanks
>
> Aruna
>
> *From:*Aruna Sankaralingam [mailto:Aruna.Sankaralingam@Cormac-Corp.com]
> *Sent:* Wednesday, October 11, 2017 11:26 AM
> *To:* users@nifi.apache.org
> *Subject:* Transformations using Nifi
>
> I am trying to see what kind of transformations that can be done in 
> nifi and how.
>
> Now I have a basic flow that takes CSV from the local dir and puts 
> into s3 and loads into postgres database.
>
> There are 4 columns in my test file 3 of which are string and one is 
> an integer field. I would like to do the following before I load the 
> data into postgres. If someone can help me on how to go about these, 
> it will be great.
>
> 1.Convert one of the string columns to upper case
>
> /For converting to upper case, I was told to use the Update Record 
> processor but  my version is 1.2.0 and the update record processor is 
> not available. /
>
> 2.Postgres has an extra column called “Load_Date” in which I would 
> like to load the current date with timestamp when the flow is run
>
> 3.If the integer column has more than 5 digits, I would like to take 
> only the first 5 digits and load to the table
>
> 4.There is a look up table in postgres. I would like to check if the 
> first column value is present in the look up table and if yes, proceed 
> ahead and if not ignore the record
>
> I am trying to learn nifi so I would really appreciate any kind of 
> help here. Is there any training available online that I can take in 
> order to understand and do all these?
>


Mime
View raw message