nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prabhu Mahendran <prabhuu161...@gmail.com>
Subject Re: Sqoop Support in NIFI
Date Tue, 29 Mar 2016 08:06:34 GMT
Hi,

Yes, In my case i have created the Custom processor with Sqoop API which
accommodates complete functionality of sqoop.
As per you concern we have able to move the data only from HDFS to SQl or
Vice versa, But sqoop having more functionality which we can achieve it by
Sqoop.RunTool() in org.apache.sqoop.sqoop. The Sqoop Java client will works
well and Implement that API into new Sqoop NIFI processor Doesn't work!

On Tue, Mar 29, 2016 at 12:49 PM, Conrad Crampton <
conrad.crampton@secdata.com> wrote:

> Hi,
> If you could explain exactly what you are trying to achieve I.e. What part
> of the data pipeline you are looking to use NiFi for and where you wish to
> retain Sqoop I could perhaps have a more informed input (although I have
> only been using NiFi myself for a few weeks). Sqoop obviously can move the
> data from RDBM systems through to HDFS (and vice versa) as can NiFi, not
> sure why you would want the mix (or at least I can’t see it from the
> description you have provided thus far).
> I have limited knowledge of Sqoop, but either way, I am sure you could
> ‘drive’ Sqoop from a custom NiFi processor if you so choose, and you can
> ‘drive’ NiFi externally (using the REST api) - if Sqoop can consume it.
> Regards
> Conrad
>
>
> From: prabhu Mahendran <prabhuu161994@gmail.com>
> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
> Date: Tuesday, 29 March 2016 at 07:55
> To: "users@nifi.apache.org" <users@nifi.apache.org>
> Subject: Re: Sqoop Support in NIFI
>
> Hi Conrad,
>
> Thanks for Quick Response.
>
> Yeah.Combination of Execute SQL and Put HDFS works well instead
> of Sqoop.But is there any possible to use Sqoop(client) to do like this?
>
> Prabhu Mahendran
>
> On Tue, Mar 29, 2016 at 12:04 PM, Conrad Crampton <
> conrad.crampton@secdata.com> wrote:
>
>> Hi,
>> Why use sqoop at all? Use a combination of ExecuteSQL [1] and PutHDFS [2].
>> I have just replace the use of Flume using a combination of ListenSyslog
>> and PutHDFS which I guess is a similar architectural pattern.
>> HTH
>> Conrad
>>
>>
>>
>> http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExecuteSQL/index.html
>>  [1]
>>
>> http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.PutHDFS/index.html
>>  [2]
>>
>> From: prabhu Mahendran <prabhuu161994@gmail.com>
>> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org>
>> Date: Tuesday, 29 March 2016 at 07:27
>> To: "users@nifi.apache.org" <users@nifi.apache.org>
>> Subject: Sqoop Support in NIFI
>>
>> Hi,
>>
>> I am new to nifi.
>>
>>        I have to know that  "Is there is any Support for Sqoop with help
>> of NIFI Processors?."
>>
>> And in which way to done the following case with help of Sqoop.
>>
>>     Move data from oracle,SqlServer,MySql into HDFS and vice versa.
>>
>>
>> Thanks,
>> Prabhu Mahendran
>>
>>
>>
>>
>> ***This email originated outside SecureData***
>>
>> Click here <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==> to
>> report this email as spam.
>>
>>
>> SecureData, combating cyber threats
>>
>> ------------------------------
>>
>> The information contained in this message or any of its attachments may
>> be privileged and confidential and intended for the exclusive use of the
>> intended recipient. If you are not the intended recipient any disclosure,
>> reproduction, distribution or other dissemination or use of this
>> communications is strictly prohibited. The views expressed in this email
>> are those of the individual and not necessarily of SecureData Europe Ltd.
>> Any prices quoted are only valid if followed up by a formal written quote.
>>
>> SecureData Europe Limited. Registered in England & Wales 04365896.
>> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
>> Maidstone, Kent, ME16 9NT
>>
>
>

Mime
View raw message