nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Issue with AWS ELB on secure nifi-registry
Date Wed, 21 Mar 2018 16:33:14 GMT
There only needs to be W to /proxy so that part should be fine.

After you edited the Node Identities, did you delete users.xml and
authorizations.xml?

You would have to do that for those changes to take effect. You can
look in users.xml and see if you still have the user identities
without whitespace.

On Wed, Mar 21, 2018 at 12:20 PM, Scott Howell <scotthowell@mobilgov.com> wrote:
> One other thing I am seeing and I don’t know if this is an issue or not in my authorizations.xml
I do not have a policy for /proxy with action=“R” only action=“W”.
>
>> On Mar 21, 2018, at 11:03 AM, Scott Howell <scotthowell@mobilgov.com> wrote:
>>
>> Thanks for that. I am still getting this error in my nifi-user.log
>>
>> o.a.n.w.s.NiFiAuthenticationFilter Rejecting access to web api: Untrusted proxy CN=*.{redacted}.com,
OU={redacted}, O={redacted}, L=Kansas City, ST=Missouri, C=US
>>
>> Is there an issue with using a wildcard cert?
>>
>>
>>> On Mar 21, 2018, at 10:23 AM, Bryan Bende <bbende@gmail.com> wrote:
>>>
>>> All identity strings are case & whitespace sensitive.
>>>
>>> The node identities in your authorizers.xml have no whitespace, and
>>> the identity showing in the logs does.
>>>
>>> On Wed, Mar 21, 2018 at 11:05 AM, Scott Howell <scotthowell@mobilgov.com>
wrote:
>>>> Thanks for all of the help with this. I have the cluster up and running.
The
>>>> logs look great everything seems to be working but I cannot login into the
>>>> UI. I am using a wildcard self-signed certificate. The /proxy is in
>>>> authorizations.xml with the correct users for the nodes.
>>>>
>>>> The error I see with the UI :
>>>> is Untrusted proxy CN=*.{redacted}.com, OU={redacted}, O={redacted},
>>>> L=Kansas City, ST=Missouri, C=US
>>>>
>>>> I haven’t had much luck finding a lot of documentation or forum questions
>>>> with this kind of issue.
>>>>
>>>> My authorizers.xml looks like this
>>>> <authorizers>
>>>> <authorizer>
>>>>   <identifier>file-provider</identifier>
>>>>   <class>org.apache.nifi.authorization.FileAuthorizer</class>
>>>>   <property name="Authorizations
>>>> File">/opt/config/authorizations.xml</property>
>>>>   <property name="Users File">/opt/config/users.xml</property>
>>>>   <property name="Initial Admin
>>>> Identity">uid=scott,ou=users,dc={redacted},dc=com</property>
>>>>   <property name="Legacy Authorized Users File"></property>
>>>>
>>>>   <property name="Node Identity
>>>> 1">CN=node-1-nifi-dev.{redacted}.com,OU={redacted},O={redacted},L=Kansas
>>>> City,ST=Missouri,C=US</property>
>>>>   <property name="Node Identity
>>>> 2">CN=node-2-nifi-dev.{redacted}.com,OU={redacted},O={redacted},L=Kansas
>>>> City,ST=Missouri,C=US</property>
>>>> </authorizer>
>>>> </authorizers>
>>>>
>>>> Thanks,
>>>>
>>>> Scott
>>>>
>>>> On Mar 20, 2018, at 1:15 PM, Andy LoPresto <alopresto@apache.org> wrote:
>>>>
>>>> Scott,
>>>>
>>>> The original exception is "nested exception is
>>>> java.security.KeyStoreException:  not found”. Can you verify that the
>>>> keystore you’ve provided is valid using the “keytool” command? In addition,
>>>> you will need a truststore as well. Try following Pierre's [1] or Bryan’s
>>>> [2] instructions for setting up a secure cluster.
>>>>
>>>> [1]
>>>> https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secured-cluster-setup/
>>>> [2]
>>>> https://bryanbende.com/development/2016/08/17/apache-nifi-1-0-0-authorization-and-multi-tenancy
>>>>
>>>>
>>>> Andy LoPresto
>>>> alopresto@apache.org
>>>> alopresto.apache@gmail.com
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>> On Mar 20, 2018, at 11:05 AM, Scott Howell <scotthowell@mobilgov.com>
wrote:
>>>>
>>>> Thanks for all of the help yesterday I was able to get a secure nifi and
>>>> nifi-registry up and communicating. I am now trying to figure out how to
>>>> create a secure cluster. I am currently getting this error when I start up
>>>> nifi.
>>>>
>>>> tion; nested exception is
>>>> org.springframework.beans.factory.BeanCreationException: Error creating bean
>>>> with name 'clusterCoordinationProtocolSenderListener' defined in class path
>>>> resource [nifi-cluster-protocol-context.xml]: Cannot resolve reference to
>>>> bean 'clusterCoordinationProtocolSender' while setting constructor argument;
>>>> nested exception is org.springframework.beans.factory.BeanCreationException:
>>>> Error creating bean with name 'clusterCoordinationProtocolSender' defined
in
>>>> class path resource [nifi-cluster-protocol-context.xml]: Cannot resolve
>>>> reference to bean 'protocolSocketConfiguration' while setting constructor
>>>> argument; nested exception is
>>>> org.springframework.beans.factory.BeanCreationException: Error creating bean
>>>> with name 'protocolSocketConfiguration': FactoryBean threw exception on
>>>> object creation; nested exception is java.security.KeyStoreException:  not
>>>> found
>>>>      at
>>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
>>>>      at
>>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1634)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
>>>>      ... 50 common frames omitted
>>>> Caused by: org.springframework.beans.factory.BeanCreationException: Error
>>>> creating bean with name 'clusterCoordinationProtocolSenderListener' defined
>>>> in class path resource [nifi-cluster-protocol-context.xml]: Cannot resolve
>>>> reference to bean 'clusterCoordinationProtocolSender' while setting
>>>> constructor argument; nested exception is
>>>> org.springframework.beans.factory.BeanCreationException: Error creating bean
>>>> with name 'clusterCoordinationProtocolSender' defined in class path resource
>>>> [nifi-cluster-protocol-context.xml]: Cannot resolve reference to bean
>>>> 'protocolSocketConfiguration' while setting constructor argument; nested
>>>> exception is org.springframework.beans.factory.BeanCreationException: Error
>>>> creating bean with name 'protocolSocketConfiguration': FactoryBean threw
>>>> exception on object creation; nested exception is
>>>> java.security.KeyStoreException:  not found
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
>>>>      at
>>>> org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:648)
>>>>      at
>>>> org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:145)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1193)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1095)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
>>>>      at
>>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
>>>>      at
>>>> org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1084)
>>>>      at
>>>> org.apache.nifi.cluster.spring.NodeClusterCoordinatorFactoryBean.getObject(NodeClusterCoordinatorFactoryBean.java:44)
>>>>      at
>>>> org.apache.nifi.cluster.spring.NodeClusterCoordinatorFactoryBean.getObject(NodeClusterCoordinatorFactoryBean.java:34)
>>>>      at
>>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
>>>>      ... 55 common frames omitted
>>>> Caused by: org.springframework.beans.factory.BeanCreationException: Error
>>>> creating bean with name 'clusterCoordinationProtocolSender' defined in class
>>>> path resource [nifi-cluster-protocol-context.xml]: Cannot resolve reference
>>>> to bean 'protocolSocketConfiguration' while setting constructor argument;
>>>> nested exception is org.springframework.beans.factory.BeanCreationException:
>>>> Error creating bean with name 'protocolSocketConfiguration': FactoryBean
>>>> threw exception on object creation; nested exception is
>>>> java.security.KeyStoreException:  not found
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
>>>>      at
>>>> org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:648)
>>>>      at
>>>> org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:145)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1193)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1095)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
>>>>      at
>>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
>>>>      ... 70 common frames omitted
>>>> Caused by: org.springframework.beans.factory.BeanCreationException: Error
>>>> creating bean with name 'protocolSocketConfiguration': FactoryBean threw
>>>> exception on object creation; nested exception is
>>>> java.security.KeyStoreException:  not found
>>>>      at
>>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
>>>>      at
>>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1634)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
>>>>      at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
>>>>      at
>>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
>>>>      ... 82 common frames omitted
>>>> Caused by: java.security.KeyStoreException:  not found
>>>>      at java.security.KeyStore.getInstance(KeyStore.java:851)
>>>>      at
>>>> org.apache.nifi.security.util.KeyStoreUtils.getKeyStore(KeyStoreUtils.java:66)
>>>>      at
>>>> org.apache.nifi.security.util.KeyStoreUtils.getTrustStore(KeyStoreUtils.java:80)
>>>>      at
>>>> org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:73)
>>>>      at
>>>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:45)
>>>>      at
>>>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:30)
>>>>      at
>>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
>>>>      ... 87 common frames omitted
>>>> Caused by: java.security.NoSuchAlgorithmException:  KeyStore not available
>>>>      at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
>>>>      at java.security.Security.getImpl(Security.java:695)
>>>>      at java.security.KeyStore.getInstance(KeyStore.java:848)
>>>>      ... 93 common frames omitted
>>>>
>>>> My nifi.properties file is.
>>>>
>>>> # Licensed to the Apache Software Foundation (ASF) under one or more
>>>> # contributor license agreements.  See the NOTICE file distributed with
>>>> # this work for additional information regarding copyright ownership.
>>>> # The ASF licenses this file to You under the Apache License, Version 2.0
>>>> # (the "License"); you may not use this file except in compliance with
>>>> # the License.  You may obtain a copy of the License at
>>>> #
>>>> #     http://www.apache.org/licenses/LICENSE-2.0
>>>> #
>>>> # Unless required by applicable law or agreed to in writing, software
>>>> # distributed under the License is distributed on an "AS IS" BASIS,
>>>> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> # See the License for the specific language governing permissions and
>>>> # limitations under the License.
>>>>
>>>> # Core Properties #
>>>> nifi.version={{nifi_version}}
>>>> nifi.flow.configuration.file=/opt/config/flow.xml.gz
>>>> nifi.flow.configuration.archive.enabled=true
>>>> nifi.flow.configuration.archive.dir=/opt/config/archive/
>>>> nifi.flow.configuration.archive.max.time=30 days
>>>> nifi.flow.configuration.archive.max.storage=500 MB
>>>> nifi.flowcontroller.autoResumeState=true
>>>> nifi.flowcontroller.graceful.shutdown.period=10 sec
>>>> nifi.flowservice.writedelay.interval=500 ms
>>>> nifi.administrative.yield.duration=30 sec
>>>> # If a component has no work to do (is "bored"), how long should we wait
>>>> before checking again for work?
>>>> nifi.bored.yield.duration=10 millis
>>>>
>>>>
>>>> nifi.authorizer.configuration.file=/opt/config/authorizers.xml
>>>> nifi.login.identity.provider.configuration.file=/opt/config/login-identity-providers.xml
>>>> nifi.templates.directory=/opt/config/templates
>>>> nifi.ui.banner.text=
>>>> nifi.ui.autorefresh.interval=30 sec
>>>> nifi.nar.library.directory=/opt/nifi/lib
>>>> nifi.nar.library.directory.custom=/opt/config/processors
>>>> nifi.nar.working.directory=/opt/nifi/work/nar/
>>>> nifi.documentation.working.directory=./work/docs/components
>>>>
>>>> ####################
>>>> # State Management #
>>>> ####################
>>>> nifi.state.management.configuration.file=/opt/config/state-management.xml
>>>> # The ID of the local state provider
>>>> nifi.state.management.provider.local=local-provider
>>>> # The ID of the cluster-wide state provider. This will be ignored if NiFi
is
>>>> not clustered but must be populated if running in a cluster.
>>>> nifi.state.management.provider.cluster=zk-provider
>>>> # Specifies whether or not this instance of NiFi should run an embedded
>>>> ZooKeeper server
>>>> nifi.state.management.embedded.zookeeper.start=false
>>>> # Properties file that provides the ZooKeeper properties to use if
>>>> <nifi.state.management.embedded.zookeeper.start> is set to true
>>>> nifi.state.management.embedded.zookeeper.properties=/opt/config/zookeeper.properties
>>>>
>>>>
>>>> # H2 Settings
>>>> nifi.database.directory=/opt/database_repository
>>>> nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
>>>>
>>>> # FlowFile Repository
>>>> nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
>>>> nifi.flowfile.repository.directory=/opt/flowfile_repository
>>>> nifi.flowfile.repository.partitions=256
>>>> nifi.flowfile.repository.checkpoint.interval=2 mins
>>>> nifi.flowfile.repository.always.sync=false
>>>>
>>>> nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
>>>> nifi.queue.swap.threshold=20000
>>>> nifi.swap.in.period=5 sec
>>>> nifi.swap.in.threads=1
>>>> nifi.swap.out.period=5 sec
>>>> nifi.swap.out.threads=4
>>>>
>>>> # Content Repository
>>>> nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
>>>> nifi.content.claim.max.appendable.size=10 MB
>>>> nifi.content.claim.max.flow.files=100
>>>> nifi.content.repository.directory.default=/opt/content_repository
>>>> nifi.content.repository.archive.max.retention.period=12 hours
>>>> nifi.content.repository.archive.max.usage.percentage=50%
>>>> nifi.content.repository.archive.enabled=true
>>>> nifi.content.repository.always.sync=false
>>>> nifi.content.viewer.url=/nifi-content-viewer/
>>>>
>>>> # Provenance Repository Properties
>>>> nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository
>>>>
>>>> # Persistent Provenance Repository Properties
>>>> nifi.provenance.repository.directory.default=/opt/provenance_repository
>>>> nifi.provenance.repository.max.storage.time=24 hours
>>>> nifi.provenance.repository.max.storage.size=1 GB
>>>> nifi.provenance.repository.rollover.time=30 secs
>>>> nifi.provenance.repository.rollover.size=100 MB
>>>> nifi.provenance.repository.query.threads=2
>>>> nifi.provenance.repository.index.threads=1
>>>> nifi.provenance.repository.compress.on.rollover=true
>>>> nifi.provenance.repository.always.sync=false
>>>> nifi.provenance.repository.journal.count=16
>>>> # Comma-separated list of fields. Fields that are not indexed will not be
>>>> searchable. Valid fields are:
>>>> # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID,
>>>> AlternateIdentifierURI, Relationship, Details
>>>> nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename,
>>>> ProcessorID, Relationship
>>>> # FlowFile Attributes that should be indexed and made searchable.  Some
>>>> examples to consider are filename, uuid, mime.type
>>>> nifi.provenance.repository.indexed.attributes=
>>>> # Large values for the shard size will result in more Java heap usage when
>>>> searching the Provenance Repository
>>>> # but should provide better performance
>>>> nifi.provenance.repository.index.shard.size=500 MB
>>>> # Indicates the maximum length that a FlowFile attribute can be when
>>>> retrieving a Provenance Event from
>>>> # the repository. If the length of any attribute exceeds this value, it will
>>>> be truncated when the event is retrieved.
>>>> nifi.provenance.repository.max.attribute.length=65536
>>>>
>>>> # Volatile Provenance Respository Properties
>>>> nifi.provenance.repository.buffer.size=100000
>>>>
>>>> # Component Status Repository
>>>> nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository
>>>> nifi.components.status.repository.buffer.size=1440
>>>> nifi.components.status.snapshot.frequency=1 min
>>>>
>>>> # Site to Site properties
>>>> nifi.remote.input.host=
>>>> nifi.remote.input.secure=false
>>>> nifi.remote.input.socket.port=9998
>>>> nifi.remote.input.http.enabled=false
>>>> nifi.remote.input.http.transaction.ttl=30 sec
>>>>
>>>> # web properties #
>>>> nifi.web.war.directory=/opt/nifi/lib
>>>> nifi.web.http.host=
>>>> nifi.web.http.port=
>>>> nifi.web.https.host={{redacted}}
>>>> nifi.web.https.port=8443
>>>> nifi.web.jetty.working.directory=/opt/nifi/work/jetty
>>>> nifi.web.jetty.threads=200
>>>>
>>>> # security properties #
>>>> nifi.sensitive.props.key=x0KDgO9L8lAhFGLdvu2VEjFVGc6Kg3V0R5I4bYwoqdgL47moo0wApKQtAVu1BvD
>>>> nifi.sensitive.props.key.protected=
>>>> nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>>>> nifi.sensitive.props.provider=BC
>>>> nifi.sensitive.props.additional.keys=
>>>>
>>>> nifi.security.keystore=/opt/certs/payit_keystore
>>>> nifi.security.keystoreType=JKS
>>>> nifi.security.keystorePasswd={{keystore_password}}
>>>> nifi.security.keyPasswd=
>>>> nifi.security.truststore=
>>>> nifi.security.truststoreType=
>>>> nifi.security.truststorePasswd=
>>>> nifi.security.needClientAuth=false
>>>> nifi.security.user.authorizer=file-provider
>>>> nifi.security.user.login.identity.provider=ldap-provider
>>>> nifi.security.ocsp.responder.url=
>>>> nifi.security.ocsp.responder.certificate=
>>>>
>>>> # Identity Mapping Properties #
>>>> # These properties allow normalizing user identities such that identities
>>>> coming from different identity providers
>>>> # (certificates, LDAP, Kerberos) can be treated the same internally in NiFi.
>>>> The following example demonstrates normalizing
>>>> # DNs from certificates and principals from Kerberos into a common identity
>>>> string:
>>>> #
>>>> #nifi.security.identity.mapping.pattern.dn=^cn=(.*?),ou=(.*?),dc=(.*?),dc=(.*?),dc=(.*?)$
>>>> #nifi.security.identity.mapping.value.dn=$1
>>>> # nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
>>>> # nifi.security.identity.mapping.value.kerb=$1@$2
>>>>
>>>> # cluster common properties (all nodes must have same values) #
>>>> nifi.cluster.protocol.heartbeat.interval=5 sec
>>>> nifi.cluster.protocol.is.secure=true
>>>>
>>>> # cluster node properties (only configure for cluster nodes) #
>>>> nifi.cluster.is.node=true
>>>> nifi.cluster.node.address=nifi-dev.mobilgov.com
>>>> nifi.cluster.node.protocol.port=9999
>>>> nifi.cluster.node.protocol.threads=10
>>>> nifi.cluster.node.event.history.size=25
>>>> nifi.cluster.node.connection.timeout=5 sec
>>>> nifi.cluster.node.read.timeout=5 sec
>>>> nifi.cluster.firewall.file=
>>>>
>>>>
>>>> # zookeeper properties, used for cluster management #
>>>> nifi.zookeeper.connect.string=internal-etcd-dev-EtcdLoadB-3RWA2WEFBBT8-2068560477.us-east-2.elb.amazonaws.com:2181,internal-etcd-dev-EtcdLoadB-3RWA2WEFBBT8-2068560477.us-east-2.elb.amazonaws.com2182,internal-etcd-dev-EtcdLoadB-3RWA2WEFBBT8-2068560477.us-east-2.elb.amazonaws.com:2183
>>>> nifi.zookeeper.connect.timeout=3 secs
>>>> nifi.zookeeper.session.timeout=3 secs
>>>> nifi.zookeeper.root.node=/nifi
>>>>
>>>> # kerberos #
>>>> nifi.kerberos.krb5.file=
>>>>
>>>> # kerberos service principle #
>>>> nifi.kerberos.service.principal=
>>>> nifi.kerberos.service.keytab.location=
>>>>
>>>> # kerberos spnego principle #
>>>> nifi.kerberos.spnego.principal=
>>>> nifi.kerberos.spnego.keytab.location=
>>>> nifi.kerberos.spnego.authentication.expiration=12 hours
>>>>
>>>> # external properties files for variable registry
>>>> # supports a comma delimited list of file locations
>>>> nifi.variable.registry.properties=
>>>>
>>>> I think I have everything set correctly but I have not been able to start
an
>>>> instances up.
>>>>
>>>> Thanks,
>>>>
>>>> Scott
>>>>
>>>> On Mar 19, 2018, at 4:35 PM, Bryan Bende <bbende@gmail.com> wrote:
>>>>
>>>> The base file is here for comparison:
>>>>
>>>> https://github.com/apache/nifi-registry/blob/master/nifi-registry-resources/src/main/resources/conf/identity-providers.xml#L23
>>>>
>>>> On Mon, Mar 19, 2018 at 5:34 PM, Bryan Bende <bbende@gmail.com> wrote:
>>>>
>>>> For your first file, is what you showed there actually wrapped in
>>>> <identityProviders> </identityProviders> or is it exactly what
you
>>>> showed?
>>>>
>>>> It may just be that you only copied/pasted the one provider, but the
>>>> root element is not <provider>, so as it is shown there it would not
>>>> parse.
>>>>
>>>> On Mon, Mar 19, 2018 at 2:54 PM, Scott Howell <scotthowell@mobilgov.com>
>>>> wrote:
>>>>
>>>> Here is my file
>>>>
>>>> <provider>
>>>>     <identifier>ldap-identity-provider</identifier>
>>>>     <class>org.apache.nifi.registry.security.ldap.LdapProvider</class>
>>>>     <property name="Authentication Strategy">SIMPLE</property>
>>>>
>>>>     <property name="Manager DN">cn=Manager,dc=mobilgov,dc=com</property>
>>>>     <property name="Manager Password”>redacted</property>
>>>>
>>>>
>>>>     <property name="Referral Strategy">FOLLOW</property>
>>>>     <property name="Connect Timeout">10 secs</property>
>>>>     <property name="Read Timeout">10 secs</property>
>>>>
>>>>     <property name="Url”>redacted</property>
>>>>     <property name="User Search
>>>> Base">ou=users,dc=mobilgov,dc=com</property>
>>>>     <property name="User Search Filter">uid={0}</property>
>>>>
>>>>     <property name="Identity Strategy">USE_DN</property>
>>>>     <property name="Authentication Expiration">12 hours</property>
>>>> </provider>
>>>>
>>>> Here is my authorizers.xml
>>>>
>>>> <authorizers>
>>>>
>>>> <userGroupProvider>
>>>>     <identifier>file-user-group-provider</identifier>
>>>>
>>>> <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
>>>>     <property name="Users File">conf/users.xml</property>
>>>>     <property name="Legacy Authorized Users File"></property>
>>>>     <property name="Initial User Identity 1”>redacted</property>
>>>> </userGroupProvider>
>>>>
>>>> <accessPolicyProvider>
>>>>     <identifier>file-access-policy-provider</identifier>
>>>>
>>>> <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
>>>>     <property name="User Group
>>>> Provider">file-user-group-provider</property>
>>>>     <property name="Authorizations
>>>> File">conf/authorizations.xml</property>
>>>>     <property name="Initial Admin Identity”>redacted</property>
>>>>     <property name="NiFi Identity 1"></property>
>>>> </accessPolicyProvider>
>>>>
>>>> <authorizer>
>>>>     <identifier>managed-authorizer</identifier>
>>>>
>>>> <class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class>
>>>>     <property name="Access Policy
>>>> Provider">file-access-policy-provider</property>
>>>> </authorizer>
>>>> </authorizers>
>>>>
>>>> On Mar 19, 2018, at 12:59 PM, Bryan Bende <bbende@gmail.com> wrote:
>>>>
>>>> It looks like that error would happen if your identity-providers.xml
>>>> contained invalid XML.
>>>>
>>>> Did you start by modifying the identity-providers.xml file that was
>>>> already there? Can you share the file, or the contents (removing
>>>> anything sensitive)?
>>>>
>>>> On Mon, Mar 19, 2018 at 1:09 PM, Scott Howell <scotthowell@mobilgov.com>
>>>> wrote:
>>>>
>>>> So I was able to get the UI pulled up but now I am hitting a roadblock with
>>>> my identity-provider.xml.
>>>>
>>>> I am getting  a number of errors like this:
>>>>
>>>> Caused by: org.springframework.beans.factory.BeanCreationException: Error
>>>> creating bean with name 'getIdentityProvider' defined in class path resource
>>>> [org/apache/nifi/registry/security/authentication/IdentityProviderFactory.class]:
>>>> Bean instantiation via factory method failed; nested exception is
>>>> org.springframework.beans.BeanInstantiationException: Failed to instantiate
>>>> [org.apache.nifi.registry.security.authentication.IdentityProvider]: Factory
>>>> method 'getIdentityProvider' threw exception; nested exception is
>>>> java.lang.Exception: Unable to load the login identity provider
>>>> configuration file at: /opt/nifi-registry-0.1.0/conf/identity-providers.xml
>>>>    at
>>>> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:587)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1250)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1099)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:502)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:312)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:251)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1135)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1062)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:815)
>>>> ~[na:na]
>>>>    at
>>>> org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:721)
>>>> ~[na:na]
>>>>    ... 43 common frames omitted
>>>>
>>>> I know it has to do with the identity-provider.xml but I have my setup just
>>>> like the documentation ask for. I turned on debug but was not able to see
>>>> anything different or better explanation from it.
>>>>
>>>>
>>>> On Mar 19, 2018, at 10:06 AM, Kevin Doran <kdoran@apache.org> wrote:
>>>>
>>>> Ok, that use case should be fine.
>>>>
>>>> If it were an authorization issue you would see something in the logs saying
>>>> that an authorization attempt failed and the server is responding with a
>>>> 403.  Just to be sure, can you enable debug logging if you haven't already,
>>>> i.e., in your nifi-registry/conf/logback.xml file, change
>>>> 'org.apache.nifif.registry' to debug:
>>>>
>>>> <!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->
>>>> <logger name="org.apache.nifi.registry" level="DEBUG"/>
>>>>
>>>> If there is nothing being written to nifi-registry-app.log, it points
>>>> towards a connection issue, so I would double check your host, port, and
TLS
>>>> settings. You'll have to get an HTTPS cert from a root CA or configure your
>>>> ELB to trust your company's self-signed cert (again, not sure if/how to do
>>>> this, but I assume there should be some way to configure it. It might
>>>> require settings not exposed in the AWS web console.)
>>>>
>>>> On 3/19/18, 10:51, "Scott Howell" <scotthowell@mobilgov.com> wrote:
>>>>
>>>> Thanks Kevin,
>>>>
>>>> I am just using the ELB to go from the public subnet to the private subnet.
>>>> I will not have multiple instances running of registry.
>>>>
>>>> I will say on my authorizers.xml there is one difference between my nifi
>>>> instance. On my nifi instance I am using file-provider for
>>>> nifi.security.user.authorizer in my nifi.properties. I don’t think from
>>>> reading the documents for nifi-registry that I can use that. If there is
a
>>>> way that might be my problem. I was running into some issues with my nifi
>>>> instance when I was using managed-authorizers instead of file-provider.
>>>>
>>>>
>>>>
>>>> On Mar 19, 2018, at 9:35 AM, Kevin Doran <kdoran@apache.org> wrote:
>>>>
>>>> Hey Scott,
>>>>
>>>> Assuming you are using two-way TLS with client certificates for
>>>> authentication, I recommend configuring your ELB for TCP passthrough so that
>>>> the TLS handshake is between the end-client and the NiFi Registry Server
(in
>>>> other words, no decryption/termination of the TLS connection happens in the
>>>> ELB). If you are using some other form of authentication (e.g., LDAP), you
>>>> will need to configure your ELB to trust the self-signed key NiFi Registry
>>>> is using. I'm not sure how to do that as I've never run an ELB with that
>>>> configuration before.
>>>>
>>>> Also, just a note about using an ELB with NiFi Registry:
>>>>
>>>> NiFi Registry is currently only supports single-instance use as persisted
>>>> data and in-memory state is not synced between multiple instances. Are you
>>>> hoping to use the ELB for actual load balancing, or is it just to take
>>>> advantage of other ELB features, such as forwarding and security group
>>>> rules? If the plan is to load balance multiple Registry instances, just be
>>>> aware that you will probably run into some unexpected behavior. (As you
>>>> mentioned using authorization, that is one case where I know the in-memory
>>>> cache of the persisted data will not refresh across instances, so even if
>>>> you were using some sort of shared network file system attached to multiple
>>>> Registry instances, such as EFS, it would not work the way you hope.)
>>>>
>>>> Hope this helps,
>>>> Kevin
>>>>
>>>> On 3/19/18, 10:20, "Scott Howell" <scotthowell@mobilgov.com> wrote:
>>>>
>>>> Thanks for the quick response.
>>>>
>>>> A couple of things I am seeing.
>>>>
>>>> 1. There is no error, I don’t see anything in the logs once the service
>>>> comes up. This is because the health check is not even hitting the instance
>>>> when secure.
>>>>
>>>> 2. Nothing interesting in the nifi-registry-app.logs. That was my concern
>>>> because on my nifi instance I can see the health check hitting the instance
>>>> from the ELB. This does not happen on the nifi-registry instance.  I see
the
>>>> service startup and it tells me what domain and port I can access the UI
but
>>>> nothing else after that.
>>>>
>>>> 3. When I am on an instances in the same private subnet I am able to curl
to
>>>> the instance I get the TLS SSL which tells me the keystore is on the server.
>>>> I am using a JKS keystore that is self-signed by the company I work for.
>>>>
>>>> On Mar 19, 2018, at 9:10 AM, Bryan Bende <bbende@gmail.com> wrote:
>>>>
>>>> Hello,
>>>>
>>>> What error are you getting when you cannot access the UI?
>>>>
>>>> Is there anything interesting in nifi-registry-app.log regarding
>>>> authentication/authorization when this happens?
>>>>
>>>> Can you access the UI securely without going through the ELB?
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>>
>>>> On Mon, Mar 19, 2018 at 10:05 AM, Scott Howell <scotthowell@mobilgov.com>
>>>> wrote:
>>>>
>>>> I was able to stand up nifi-registry behind an AWS ELB non-secure.
>>>> Everything was working great and was able to access the UI anonymously. I
>>>> set up the authorization just like on my nifi instances along with the
>>>> authorizers and identity-provider. The service comes up without errors and
>>>> everything looks good but the health check does not pass and I cannot access
>>>> the UI to login. I was wondering if anyone else has ran into this issue
>>>> using nifi-registry.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>


Mime
View raw message