nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Farbota <jfarb...@payoff.com>
Subject Re: Encryption and Hashing in Nifi
Date Tue, 27 Jun 2017 18:11:02 GMT
Andy,

Just wanted to note that this would be really useful for us if this was in
a processor as well.

[image: Payoff, Inc.]
*Jeremy Farbota*
Software Engineer, Data
Payoff, Inc.

jfarbota@payoff.com
(217) 898-8110 <+2178988110>

On Tue, Jun 27, 2017 at 10:29 AM, Andy LoPresto <alopresto@apache.org>
wrote:

> And to circle back to Raman’s original question, I did find this example
> script [1] I wrote to help someone with a similar problem (encrypting
> flowfile attributes rather than JSON fields, but it can easily be
> modified).
>
> [1] https://stackoverflow.com/a/40295725/70465
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 27, 2017, at 10:28 AM, Andy LoPresto <alopresto@apache.org> wrote:
>
> James,
>
> The Developer Guide [1] is not a step-by-step tutorial but does include
> some excellent information. Here are some other resources that may be
> helpful but are not official Apache NiFi documents:
>
> * Step-by-step guide to building custom processor (1.1.0) -
> http://www.nifi.rocks/developing-a-custom-apache-nifi-processor-json/
> * Bryan Bende’s guide (0.0.2-incubating) - http://bryanbende.com/
> development/2015/02/04/custom-processors-for-apache-nifi
> * HCC article on building custom processors - https://community.
> hortonworks.com/articles/4318/build-custom-nifi-processor.html
>
> Hopefully those are sufficient to get started/familiar with the process.
>
>
> [1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 27, 2017, at 9:52 AM, James McMahon <jsmcmahon3@gmail.com> wrote:
>
> Andy and Joe, would you remind those of us who have not done it before
> with a link to an example showing how to  build a Java implementation of a
> processor? I recall looking into that at one time and discovering that it
> can be a nontrivial effort.
>
> On Tue, Jun 27, 2017 at 12:42 PM, Andy LoPresto <alopresto@apache.org>
> wrote:
>
>> Hi Raman,
>>
>> I agree with Joe that you can use the ExecuteScript processor to
>> prototype what you are looking for (and if you need any help with that, let
>> me know). I also wanted to note that there are some outstanding Jiras which
>> will hopefully make this type of activity easier in the near future,
>> specifically NIFI-4132:
>>
>> * NIFI-3890 - Create Key Management Controller Service [1]
>> * NIFI-3889 - EncryptContent processor should add encryption metadata as
>> attributes [2]
>> * NIFI-3834 - Implement encrypted content repository [3]
>> * NIFI-1885 - Extend HashAttribute to provide multiple algorithms [4]
>> * NIFI-3929 - Allow external key management for EncryptContent processor
>> [5]
>> * NIFI-2961 - Create EncryptAttribute processor [6]
>> * NIFI-4132 - Create EncryptRecord controller service and processor [7]
>>
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-3890
>> [2] https://issues.apache.org/jira/browse/NIFI-3889
>> [3] https://issues.apache.org/jira/browse/NIFI-3834
>> [4] https://issues.apache.org/jira/browse/NIFI-1885
>> [5] https://issues.apache.org/jira/browse/NIFI-3929
>> [6] https://issues.apache.org/jira/browse/NIFI-2961
>> [7] https://issues.apache.org/jira/browse/NIFI-4132
>>
>>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Jun 27, 2017, at 4:45 AM, Ramaninder Singh Jhajj <
>> jhajj.raman094@gmail.com> wrote:
>>
>> Thanks a lot Joe for the quick response.
>>
>> That seems like a good solution. Will dig deep into it.
>>
>> Thanks a lot.
>>
>> On Tue, Jun 27, 2017 at 1:40 PM, Joe Witt <joe.witt@gmail.com> wrote:
>>
>>> Raman,
>>>
>>> There are no out of the box apache nifi processors to encrypt specific
>>> fields of a given record.  With the record reader/writer capabilities
>>> that came out in 1.3.0 we can probably much more easily provide one
>>> but in the mean time it is a pretty straight forward extension to
>>> build.  You could use execute script to nail down requirements and
>>> understand throughput needs then build a Java implementation of the
>>> processor to optimize performance.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Tue, Jun 27, 2017 at 7:33 AM, Ramaninder Singh Jhajj
>>> <jhajj.raman094@gmail.com> wrote:
>>> > Hello Everyone,
>>> >
>>> > I need some help with Nifi. I have a requirement where we need to
>>> encrypt
>>> > and hash some of the data in a flowfile instead of the whole flowfile.
>>> The
>>> > situation is, we receive customer information in a flowfile in XML
>>> format, I
>>> > do some cleanup and tranform the flowfile content in JSON content. Now
>>> the
>>> > requirement if we need to encrypt some of the sensitive data of
>>> > customer(only some attributes of the JSON flowfile). So if customer
>>> JSON
>>> > contains name, phoneno, email, SSN etc. we just want to encrypt email
>>> and
>>> > SSN and not the whole file.
>>> >
>>> > I know there is a processor for encryption but that encrypts the whole
>>> > flowfile. Is there a processor or a way to encrypt some of the
>>> attributes of
>>> > JSON file. We do not want to use "ExecuteScript" processor as it can
>>> not be
>>> > parallelised and will run the script in linear fashion (let me know if
>>> I am
>>> > wrong here).
>>> >
>>> >
>>> > Kind Regards,
>>> > Raman
>>>
>>
>>
>>
>
>
>

Mime
View raw message