nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Geercken" <uwe.geerc...@web.de>
Subject Site-to-site not working
Date Sat, 10 Sep 2016 10:02:27 GMT
Hello,

I have used Nifi for a while on my laptop. Now I wanted to setup a second machine communicating
with the first one - either using Nifi oder Minifi. For the moment I have Nifi 1.0.0 running
on both and I followed the instructions of how to setup Nifi. I read documentation, but there
must be something I have overlooked I guess. There is some documentation on site-to-site but
nothing concrete - I assume that is what I have to setup. I miss e.g. the concrete info what
exact port the two instances will use to communicate, so I fetched that from a website but
I am not sure if that is correct.

I just won't get it working and would need some help or advice. I don't know if it is misconfiguration
or maybe a security/firewall problem.

The setup is as follows:

Computer 1: 
- Fedora 24
- Nifi 1.0.0
- running a seperate zookeeper insatnce
- Java OpenJdk 1.8.0
- IP 192.168.2.11 (WLAN)
- I added the following to the nifi.properties file:
# Site to Site properties
nifi.remote.input.socket.host=192.168.2.17
nifi.remote.input.socket.port=10000
nifi.remote.input.secure=false

Computer 2:
- Fedora 24
- Nifi 1.0.0
- running embedded zookeeper
- Java OpenJdk 1.8.0
- IP 192.168.2.17 (LAN)
- I did not change anything in the configuration except for the embedded zookeeper

On both computers Nifi runs and I can successfully build data flows. I can ping computer 1
to computer 2 and vice versa. I have built following flows:

Computer 1:
- input port
- putfiles processor, storing the files on the local disk

Computer 2:
- generate flowfiles processor
- remote processor group pointing to: http://192.168.2.11:8080/nifi.

Now the remote processor group and the logs show following error:

ERROR [Timer-Driven Process Thread-8] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=in1,target=http://192.168.2.11:8080/nifi]
failed to communicate with http://192.168.2.11:8080/nifi due to java.net.ConnectException:
Connection refused

If I use http in the remote processor group, I get following error:

WARN [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector org.apache.nifi.remote.client.PeerSelector@335fb7ad
Unable to refresh Remote Group's peers due to Remote instance of NiFi is not configured to
allow HTTP site-to-site communications

So I don't undestand what the real problem is. Can somebody please help?

Rgds,

Uwe



> Gesendet: Freitag, 09. September 2016 um 22:49 Uhr
> Von: "Joe Skora" <jskora@gmail.com>
> An: dev@nifi.apache.org
> Betreff: Re: Nifi Debugging AWS PutS3 processor
>
> Julie,
> 
> As James mentioned, it doesn't seem to be an endpoint issue.  Normally the
> library can find them, but if you are using an S3-compatible service
> instead of AWS S3 proper you might need to configure the Endpoint Override
> property.
> 
> Firewall issues could also be blocking the traffic.
> 
> Can you access the buckets with AWS Command Line Interface
> <https://aws.amazon.com/cli/> tools?
> 
> Regards,
> Joe
> 
> On Fri, Sep 9, 2016 at 4:39 PM, James Wing <jvwing@gmail.com> wrote:
> 
> > I'm sorry to hear you're having trouble with PutS3.  A few questions:
> >
> >    - Which version of NiFi are you using?
> >    - Have you configured the Proxy Host/Port or Endpoint Override URL
> >    settings on your PutS3 processor?
> >    - Are there larger exception stack traces in the nifi-app.log files you
> >    can share?
> >    - Some of the errors mention Multi-part Upload.  Are large flowfiles
> >    failing, but smaller files successful?
> >    - The errors in the screenshot reflect two IP addresses, but it may just
> >    be a small sample.  Are the errors evenly distributed across all the
> >    cluster nodes?
> >
> > Thanks,
> >
> >
> > James
> >
> >
> > On Fri, Sep 9, 2016 at 11:19 AM, jmurakami <julie.murakami@capitalone.com>
> > wrote:
> >
> > > Hi. I am currently using the putS3 processor and getting an
> > > AmazonClientException error: Connection Timed Out. We are running the
> > flow
> > > on a Nifi cluster (of 3 instances and a cluster manager). We see the
> > error
> > > attached below and about a third of the data not being put into the
> > bucket.
> > >
> > > We were able to use the aws cli in an instance in our Nifi cluster and
> > > successfully put data into our S3 bucket, so the security groups, Nacl,
> > > subnets, and bucket policies should be okay. I don't believe it is a
> > > credential or key error since some of the data was put into the bucket
> > > already. We are using IAM roles to allow access to the S3 bucket.
> > >
> > > Has anyone else encountered this error?
> > > <http://apache-nifi-developer-list.39713.n7.nabble.com/file/
> > > n13311/Screen_Shot_2016-09-06_at_10.png>
> > >
> > > Or any advice about how to go about this error would help. Thanks!
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-nifi-developer-
> > > list.39713.n7.nabble.com/Nifi-Debugging-AWS-PutS3-processor-tp13311.html
> > > Sent from the Apache NiFi Developer List mailing list archive at
> > > Nabble.com.
> > >
> >
> 

Mime
View raw message