nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: NiFi Spark Processor
Date Fri, 18 Nov 2016 14:11:48 GMT
Shankha

I know it may not always be all that obvious, but have you started the OutputPort? Also, have
you defined ‘nifi.remote.input.socket.port’ property in wifi.properties?
Also, unless you’ve already discovered it there is an excellent blog (https://blogs.apache.org/nifi/entry/stream_processing_nifi_and_spark)
from Mark on this which covers every little detail, so hope that will help.

Cheers
Oleg

On Nov 18, 2016, at 12:29 AM, shankhamajumdar <shankha.majumdar@lexmark.com<mailto:shankha.majumdar@lexmark.com>>
wrote:

Hi joey,

I am trying the POC in different way now.

1. GetFile processor which is getting the files from the local drive.

2. Connecting the processor to the NiFi Output Port.

3. Written a Spark streaming Job which is connecting to the NiFi Output Port
to stream the data.

The Spark Job is able to connect the NiFi Output Port successfully. But not
able to get any data in spark though in the NiFi the data has been queued
from GetFile processor. I have checked the NiFi Data Provenance of the NiFi
Output Port but no data is showing there as well where as GetFile to Output
Port connector is showing the data has been queued.

Regards,
Shankha



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/NiFi-Spark-Processor-tp13899p13913.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com<http://nabble.com>.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message