uima-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaroslaw Cwiklik <cwik...@apache.org>
Subject Re: DUCC Security Model for Service Deployment
Date Wed, 19 Jul 2017 15:16:34 GMT
John, you can use the broker shipped with UIMA-AS installation for your
services.

Jerry

On Wed, Jul 19, 2017 at 10:55 AM, Osborne, John David (Campus) <
ozborn@uab.edu> wrote:

> Jerry, my services would span across multiple jobs so a service model is
> preferable for me.
>
> With regards to a secondary broker - I'm thinking to use a UIMA-AS
> installation on the same machine rather than download the full Apache AMQ
> runtime standalone. I'm worried about potential compatibility issues
> between DUCC broker and downloaded broker (unfounded concern?) and I am
> unfamiliar with AMQ in isolation.  It should also allow for comparison
> between AS and DUCC. Is this a reasonable approach?
>
> Thanks for all your help so far,
>
>  -John
>
>
> -----Original Message-----
> From: uimaee@gmail.com [mailto:uimaee@gmail.com] On Behalf Of Jaroslaw
> Cwiklik
> Sent: Tuesday, July 18, 2017 3:40 PM
> To: user@uima.apache.org
> Subject: Re: DUCC Security Model for Service Deployment
>
> John, services are typically long running and independent of jobs. They
> provide a specific function and may take a long time to initialize. They
> are meant to be reusable across multiple jobs. If your analytics are fast
> initializing and not meant to be shared you can just use jobs.
>
> We should probably modify the documentation to talk about recommending a
> secondary broker for services. My recommendation is to download and install
> the full AMQ runtime for your services. DUCC only bundles essential parts
> of AMQ distro.
>
> You decide about how secure your DUCC should be. We lock DUCC's broker by
> default to protect it from a malicious user code. If this is not your
> concern you can use DUCC's broker if you want. Keep in mind that if you use
> DUCC's broker for services its load will increase and if this is high
> enough it may effect how DUCC responds. Another benefit of having a
> secondary broker is that you can keep your services up while DUCC is being
> shutdown for update or maintenance. This is especially important if your
> services take hours to initialize.
>
> Jerry
>
> On Tue, Jul 18, 2017 at 4:07 PM, Osborne, John David (Campus) <
> ozborn@uab.edu> wrote:
>
> > Thanks Jerry!
> >
> > I can run it using the insecure option (just tried it - works great!)
> > - the broker port is closed to all but localhost as another
> > application feeds requests to DUCC. From a non-security operational
> > point of few though I am violating your recommendation to use a
> > secondary broker... I hope this won't get me into too much trouble...
> >
> > From the documentation, I had no idea there was a need for a secondary
> > broker to run services. If I wanted to do this, is this something that
> > can be set up in the current ducc distribution or would it be better
> > to go ahead and run my legacy services independently? To do this,  I'm
> > presuming I would instantiate a secondary broker (presumably still
> > 61616) but submit the deployment descriptor to the DUCC broker (61617)
> > but with the inputQueue brokerURL pointing to the secondary (61616)
> broker?
> >
> > I was hoping to make a clean migration to DUCC - maybe it would be
> > easier to get rid of services altogether and just submit jobs? Is this
> > the current trend or are a lot of folks running secondary brokers and
> services?
> >
> >  -John
> >
> >
> >
> > -----Original Message-----
> > From: uimaee@gmail.com [mailto:uimaee@gmail.com] On Behalf Of Jaroslaw
> > Cwiklik
> > Sent: Tuesday, July 18, 2017 2:19 PM
> > To: user@uima.apache.org
> > Subject: Re: DUCC Security Model for Service Deployment
> >
> > Hi, as Lou stated its recommended to use a secondary broker for
> > services to keep this separate from a DUCC broker. The DUCC broker is
> > protected by default as it is used for internal communication.
> >
> > If you want to run without broker credentials change this setting in
> > default.ducc.properties
> >
> > ducc.broker.configuration = conf/activemq-ducc-unsecure.xml
> >
> > Jerry
> >
> >
> >
> > On Tue, Jul 18, 2017 at 3:05 PM, Osborne, John David (Campus) <
> > ozborn@uab.edu> wrote:
> >
> > > Thanks Lou,
> > >
> > > I'm still not totally following.
> > >
> > > In UIMA-AS, I could deploy services (as the administrator) using:
> > > deployAsyncService.sh $SOME_DEPLOYMENT_DESCRIPTOR $BROKER_URL
> > >
> > > I thought the equivalent in DUCC was (for example):  ducc_services
> > > --register  --process_descriptor $SOME_DEPLOYMENT_DESCRIPTOR
> > > --classpath $CLASSPATH --autostart true
> > >
> > > In the first case, I don't recall worrying about passwords - I
> > > presume deployAsyncService was privileged and knew any broker
> > > credentials. Are you saying the command line utility ducc_services
> > > is unaware of ducc-broker-credentionals.properties and there is no
> > > other way for the ducc user to pass these credentionals to the
> > > broker when registering/auto-start services?
> > >
> > > Sorry for the confusion, new to DUCC. I can see the
> > > $DUCC_HOME/resources.private/ducc-broker-credentials.properties - I
> > > just don't know how to use them when deploying (same thing as
> > > registering and
> > > starting?) a service.
> > >
> > >  -John
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Lou DeGenaro [mailto:lou.degenaro@gmail.com]
> > > Sent: Tuesday, July 18, 2017 1:34 PM
> > > To: user@uima.apache.org
> > > Subject: Re: DUCC Security Model for Service Deployment
> > >
> > > The DUCC broker is for DUCC.  User services should employ a separate
> > > user broker, not the DUCC broker.
> > >
> > > That said, with proper permissions you can discover the DUCC broker
> > > user and pw in
> > > $DUCC_HOME/resources.private/ducc-broker-credentials.properties
> > >
> > > Lou.
> > >
> > > On Tue, Jul 18, 2017 at 2:14 PM, Osborne, John David (Campus) <
> > > ozborn@uab.edu> wrote:
> > >
> > > > Yes - the idea (based on a previous UIMA-AS workflow) was to
> > > > attached services to the broker and have other application/s call
> > > > them
> > as needed.
> > > >
> > > >
> > > > I thought this was still done with DUCC?
> > > >
> > > >
> > > > I see what I *think* is the active security configuration in
> > > > activemq.xml
> > > > below:
> > > >
> > > >
> > > > <simpleAuthenticationPlugin anonymousAccessAllowed="false">
> > > >             <users>
> > > >                 <authenticationUser username="${ducc.broker.admin.
> > > username}"
> > > > password="${ducc.broker.admin.password}"
> > > >                     groups="ducc-admin"/>
> > > >             </users>
> > > >         </simpleAuthenticationPlugin>
> > > >
> > > >
> > > > I'm registering services and running ducc as user ducc.
> > > >
> > > >
> > > >  -John
> > > >
> > > >
> > > > ________________________________
> > > > From: Lou DeGenaro <lou.degenaro@gmail.com>
> > > > Sent: Tuesday, July 18, 2017 12:19:21 PM
> > > > To: user@uima.apache.org
> > > > Subject: Re: DUCC Security Model for Service Deployment
> > > >
> > > > DUCC has its own password protected AMQ broker used for daemon
> > > > communications.  Are your services trying to use DUCC's broker?
> > > >
> > > > Lou.
> > > >
> > > > On Tue, Jul 18, 2017 at 1:02 PM, Osborne, John David (Campus) <
> > > > ozborn@uab.edu> wrote:
> > > >
> > > > > I am having deploying services with DUCC using a single system,
> > > > > single user setup. I can run test jobs, so I *think* my system
> > > > > is set up
> > > > correctly
> > > > > but I can't register and then start services due to
> > > > > "SecurityException on Connect".
> > > > >
> > > > >
> > > > > I don't understand how when registering services the
> > > > > credentionals are passed to DUCC, I know it uses activemq
> > > > > credentionals, but there is no parameter to pass them on the
> command line.
> > > > >
> > > > >
> > > > > I am registering the service like this:
> > > > >
> > > > > /web/servers/apache-uima-ducc-2.2.0/bin/ducc_services --register
> > > > > --process_descriptor_DD  ../resources/desc/deployment/
> > > > > ImportDocumentsDeploymentDescriptorSimple.xml --description
> > > > > "Pull Documents" --classpath $CLASSPATH --autostart true
> > > > >
> > > > >
> > > > > I have not messed with any of the configuration files
> > > > > (credentional.properties, etc...) under apache-activemq.
> > > > >
> > > > >
> > > > > My error (actually a warning) is below:
> > > > >
> > > > >
> > > > > Any help appreciated,
> > > > >
> > > > >
> > > > >  -John
> > > > >
> > > > >
> > > > > INFO: Controller: MedicsClobsConsumer Trying to Start Listener
> > > > > on
> > > > > Endpoint: queue://MRN_Document_Pull_Queue Selector: Command=2001
> > > Broker:
> > > > > tcp://localhost:61617
> > > > > >>> Service Container Deployed Successfully
> > > > > DuccAbstractProcessContainer.deploy() <<<<<<<<
User Container
> > > > > deployed .... Deployed Processing Container - Initialization
> > > > > Successful - Thread 1
> > > > > 18 Jul 2017 11:40:22,858  INFO AgentSession - T[1]
> > > > > notifyAgentWithStatus ... Job Process State Changed - PID:5803.
> > > > > Process State: Running. JMX
> > > > > Url:service:jmx:rmi:///jndi/rmi://uimaprapp1.hs.uab.edu:2105/jmx
> > > > > rm i Dispatched State Update Event to Agent with
> > > > > IP:10.23.142.165 Jul 18,
> > > > > 2017 11:40:23 AM org.apache.uima.adapter.jms.activemq.
> > > > > UimaDefaultMessageListenerContainer onException
> > > > > WARNING: Service: MedicsClobsConsumer Runtime Exception Jul 18,
> > > > > 2017
> > > > > 11:40:23 AM org.apache.uima.adapter.jms.activemq.
> > > > > UimaDefaultMessageListenerContainer onException
> > > > > WARNING: Jms Listener Failed. Endpoint: MRN_Document_Pull_Queue
> > > > > Managed
> > > > > By: tcp://localhost:61617 Reason: org.apache.activemq.
> > > > ConnectionFailedException:
> > > > > The JMS connection has failed: Force close due to
> > > > > SecurityException on connect Jul 18, 2017 11:40:23 AM
> > > > > org.apache.uima.adapter.jms.activemq.
> > > > > UimaDefaultMessageListenerContainer handleListenerSetupFailure
> > > > > WARNING: Uima AS Service:MedicsClobsConsumer Listener Unable To
> > > > > Connect
> > > > To
> > > > > Broker: tcp://localhost:61617 Retrying Until Successful ...
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message