manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Swapna Vuppala <swapna.kollip...@gmail.com>
Subject Re: Authority Connection works unpredictably
Date Wed, 23 Nov 2011 11:03:20 GMT
Hi Karl,

Am using trunk code, which I got couple of weeks back. I'll try to apply
the patch and see.

Thanks and Regards,
Swapna.

On Wed, Nov 23, 2011 at 4:25 PM, Karl Wright <daddywri@gmail.com> wrote:

> I've attached a patch to the ticket so even MCF 0.3 users should be
> able to apply it.
>
> Karl
>
> On Wed, Nov 23, 2011 at 5:40 AM, Karl Wright <daddywri@gmail.com> wrote:
> > To clarify, what I think may be happening is this.
> >
> > (1) The Java LDAP context is keeping a socket connection to the AD
> controller.
> > (2) The AD controller must be configured to close connections forcibly
> > after a certain period of time.
> > (3) The LDAP context's reconnect() operation doesn't recover from a
> > socket that was closed by the server.
> > (4) The authority code won't release the LDAP context until 5 idle
> > minutes go by.
> >
> > So basically, a connection winds up in a busted state and doesn't
> > recover, if the server closes the socket out from under the ldap
> > connection.
> >
> > It's easy to fix, so I've opened a ticket (CONNECTORS-291), and will
> > commit code changes to trunk shortly.  What version of MCF are you
> > using?
> >
> > Karl
> >
> > On Wed, Nov 23, 2011 at 5:23 AM, Karl Wright <daddywri@gmail.com> wrote:
> >> Hi Swapna,
> >>
> >> There should be manifoldcf log output that contains the actual stack
> >> trace of the exception.  That would be very helpful; I need the line
> >> numbers.
> >>
> >> The code is quite simple, and indicates that the LDAP server is
> >> refusing a connection:
> >>
> >>  protected void getSession()
> >>    throws ManifoldCFException
> >>  {
> >>    if (ctx == null)
> >>    {
> >>      // Calculate the ldap url first
> >>      String ldapURL = "ldap://" + domainControllerName + ":389";
> >>
> >>      Hashtable env = new Hashtable();
> >>
>  env.put(Context.INITIAL_CONTEXT_FACTORY,"com.sun.jndi.ldap.LdapCtxFactory");
> >>      env.put(Context.SECURITY_AUTHENTICATION,authentication);
> >>      env.put(Context.SECURITY_PRINCIPAL,userName);
> >>      env.put(Context.SECURITY_CREDENTIALS,password);
> >>
> >>      //connect to my domain controller
> >>      env.put(Context.PROVIDER_URL,ldapURL);
> >>
> >>      //specify attributes to be returned in binary format
> >>      env.put("java.naming.ldap.attributes.binary","tokenGroups
> objectSid");
> >>
> >>      // Now, try the connection...
> >>      try
> >>      {
> >>        ctx = new InitialLdapContext(env,null);
> >>      }
> >>      catch (AuthenticationException e)
> >>      {
> >>        // This means we couldn't authenticate!
> >>        throw new ManifoldCFException("Authentication problem
> >> authenticating admin user '"+userName+"': "+e.getMessage(),e);
> >>      }
> >>      catch (CommunicationException e)
> >>      {
> >>        // This means we couldn't connect, most likely
> >>        throw new ManifoldCFException("Couldn't communicate with domain
> >> controller '"+domainControllerName+"': "+e.getMessage(),e);
> >>      }
> >>      catch (NamingException e)
> >>      {
> >>        throw new ManifoldCFException(e.getMessage(),e);
> >>      }
> >>    }
> >>    else
> >>    {
> >>      // Attempt to reconnect.  I *hope* this is efficient and doesn't
> >> do unnecessary work.
> >>      try
> >>      {
> >>        ctx.reconnect(null);
> >>      }
> >>      catch (AuthenticationException e)
> >>      {
> >>        // This means we couldn't authenticate!
> >>        throw new ManifoldCFException("Authentication problem
> >> authenticating admin user '"+userName+"': "+e.getMessage(),e);
> >>      }
> >>      catch (CommunicationException e)
> >>      {
> >>        // This means we couldn't connect, most likely
> >>        throw new ManifoldCFException("Couldn't communicate with domain
> >> controller '"+domainControllerName+"': "+e.getMessage(),e);
> >>      }
> >>      catch (NamingException e)
> >>      {
> >>        throw new ManifoldCFException(e.getMessage(),e);
> >>      }
> >>    }
> >>
> >>    expiration = System.currentTimeMillis() + expirationInterval;
> >>
> >>    try
> >>    {
> >>      responseLifetime = Long.parseLong(this.cacheLifetime) * 60L *
> 1000L;
> >>      LRUsize = Integer.parseInt(this.cacheLRUsize);
> >>    }
> >>    catch (NumberFormatException e)
> >>    {
> >>      throw new ManifoldCFException("Cache lifetime or Cache LRU size
> >> must be an integer: "+e.getMessage(),e);
> >>    }
> >>
> >>  }
> >>
> >>
> >> Your problem description indicates that it is possible that the
> >> ctx.reconnect() call is failing to reconnect, but a new connection
> >> works OK on your setup.  A stack trace should tell me everything.
> >>
> >> Thanks,
> >> Karl
> >>
> >>
> >>
> >> On Wed, Nov 23, 2011 at 12:58 AM, Swapna Vuppala
> >> <swapna.kollipara@gmail.com> wrote:
> >>> Hi Karl,
> >>>
> >>> Even after reducing the max connections to 3, the connection fails
> abruptly
> >>> for me.
> >>>
> >>> Currently, the domain controller am using is mapped to only one IP
> address,
> >>> and that responds on ping, and the max connections are 3. It was
> working
> >>> yesterday and it fails suddenly throwing different exceptions like
> below:
> >>>
> >>> Threw exception: 'Couldn't communicate with domain controller
> 'globalad1':
> >>> null'
> >>> Threw exception: 'Couldn't communicate with domain controller
> >>> 'globalad1.global.arup.com': null'
> >>> Threw exception: 'globalad1.global.arup.com:389; socket closed'
> >>>
> >>> Sometimes, it works when I change the cache lifetime parameter. What
> others
> >>> factors do you think that can cause this to fail ?
> >>>
> >>> Thanks and Regards,
> >>> Swapna.
> >>>
> >>> On Tue, Nov 22, 2011 at 11:56 AM, Swapna Vuppala
> >>> <swapna.kollipara@gmail.com> wrote:
> >>>>
> >>>> OK.. Thanks for the information
> >>>>
> >>>> On Mon, Nov 21, 2011 at 6:31 PM, Karl Wright <daddywri@gmail.com>
> wrote:
> >>>>>
> >>>>> The sAMAccountName and UserPrincipalName LDAP fields were used by
> >>>>> different versions of Windows at different points in time.  Some
> >>>>> backwards compatibility was maintained, however Microsoft has
> >>>>> apparently decided to deprecate one of them (can't remember which),
> >>>>> and thus you need support for both.
> >>>>>
> >>>>> Karl
> >>>>>
> >>>>> On Mon, Nov 21, 2011 at 6:39 AM, Swapna Vuppala
> >>>>> <swapna.kollipara@gmail.com> wrote:
> >>>>> > Hi Karl,
> >>>>> >
> >>>>> > Yes, my Active Directory authority connection is configured
to
> talk to
> >>>>> > only
> >>>>> > one IP address and that particular one is responding to ping
> always.
> >>>>> >
> >>>>> > Earlier, the max connections parameter was set to 10, now I
> reduced it
> >>>>> > to 3.
> >>>>> > Its working as of now and I'll keep checking if its going to
throw
> an
> >>>>> > exception. Thanks a lot for the inputs.
> >>>>> >
> >>>>> > Also, I was wondering what the difference was between 2 options
for
> >>>>> > Login
> >>>>> > name AD attribute, sAMAccountName and UserPrincipalName ?
> >>>>> >
> >>>>> > Thanks and Regards,
> >>>>> > Swapna.
> >>>>> >
> >>>>> > On Mon, Nov 21, 2011 at 4:57 PM, Karl Wright <daddywri@gmail.com>
> >>>>> > wrote:
> >>>>> >>
> >>>>> >> So let me get this straight - your Active Directory authority
> >>>>> >> connection is configured to talk to only one IP address?
 and
> that IP
> >>>>> >> address responds to ping even when you are receiving an
error back
> >>>>> >> from the authority connection?
> >>>>> >>
> >>>>> >> Another possibility is that the DC can only accept a limited
> number of
> >>>>> >> connections at a time. What is the max connections parameter
for
> your
> >>>>> >> authority connection?  Try reducing it to no more than
3-4 and
> see if
> >>>>> >> that helps.
> >>>>> >>
> >>>>> >> Karl
> >>>>> >>
> >>>>> >>
> >>>>> >> On Mon, Nov 21, 2011 at 5:34 AM, Swapna Vuppala
> >>>>> >> <swapna.kollipara@gmail.com> wrote:
> >>>>> >> > Hi Karl,
> >>>>> >> >
> >>>>> >> > I think I see many domain controllers for the domain
am using.
> But I
> >>>>> >> > see
> >>>>> >> > only one IP address mapped to the domain controller
name that am
> >>>>> >> > using
> >>>>> >> > in
> >>>>> >> > the credentials form.
> >>>>> >> >
> >>>>> >> > As I told you, its working sometimes and throwing
exception
> >>>>> >> > sometimes.
> >>>>> >> > But
> >>>>> >> > ping works always fine on the  domain controller name
that am
> using,
> >>>>> >> > from
> >>>>> >> > which I assume that it is not unreachable.
> >>>>> >> >
> >>>>> >> > Can you tell me what else I should be checking or
what other
> factors
> >>>>> >> > could
> >>>>> >> > be causing this to fail ?
> >>>>> >> >
> >>>>> >> > Thanks and Regards,
> >>>>> >> > Swapna.
> >>>>> >> >
> >>>>> >> > On Thu, Nov 17, 2011 at 1:18 PM, Karl Wright <
> daddywri@gmail.com>
> >>>>> >> > wrote:
> >>>>> >> >>
> >>>>> >> >> Try doing nslookup on the domain controller. 
In some larger
> >>>>> >> >> companies
> >>>>> >> >> there are many domain controllers all with the
same name but
> >>>>> >> >> different
> >>>>> >> >> IP's.  These *should* all be in synch but it may
be the case
> that
> >>>>> >> >> they
> >>>>> >> >> are not - or some of them are unreachable or offline.
 This can
> >>>>> >> >> also
> >>>>> >> >> be the cause of intermittent authorization failures
during
> >>>>> >> >> crawling.
> >>>>> >> >>
> >>>>> >> >> If that is the case you have the option of setting
the local
> >>>>> >> >> machine's
> >>>>> >> >> /etc/hosts file to point to a couple of domain
controller
> instances
> >>>>> >> >> that are local and in good working order, rather
than rely on
> DNS
> >>>>> >> >> to
> >>>>> >> >> find one.
> >>>>> >> >>
> >>>>> >> >> Karl
> >>>>> >> >>
> >>>>> >> >> On Thu, Nov 17, 2011 at 1:32 AM, Swapna Vuppala
> >>>>> >> >> <swapna.kollipara@gmail.com> wrote:
> >>>>> >> >> > Hi,
> >>>>> >> >> >
> >>>>> >> >> > I seem to have some problem with Authority
Connection. When I
> >>>>> >> >> > define
> >>>>> >> >> > an
> >>>>> >> >> > Authority Connection specifying all the parameters
like
> Domain
> >>>>> >> >> > Controller,
> >>>>> >> >> > username, password etc, the connection status
shows
> "Connection
> >>>>> >> >> > Working"
> >>>>> >> >> > and
> >>>>> >> >> > everything works fine, crawling and sending
docs to solr,
> using
> >>>>> >> >> > mcf-authority-service to get only those docs
that a user has
> got
> >>>>> >> >> > permission
> >>>>> >> >> > to see etc.
> >>>>> >> >> >
> >>>>> >> >> > But suddenly, the connection status for the
Authority
> Connection
> >>>>> >> >> > throws
> >>>>> >> >> > an
> >>>>> >> >> > exception, and when I play around the credentials
form
> toggling
> >>>>> >> >> > Login
> >>>>> >> >> > name
> >>>>> >> >> > AD attribute, or changing domain controller
name, or
> >>>>> >> >> > authentication ,
> >>>>> >> >> > or
> >>>>> >> >> > sometimes even with the same settings that
threw an exception
> >>>>> >> >> > earlier,
> >>>>> >> >> > the
> >>>>> >> >> > status shows "Connection working" again.
I cannot define
> when it
> >>>>> >> >> > fails
> >>>>> >> >> > and
> >>>>> >> >> > when it works and for what settings it works.
> >>>>> >> >> >
> >>>>> >> >> > Can someone help me in understanding why
this is happening
> and
> >>>>> >> >> > what
> >>>>> >> >> > needs to
> >>>>> >> >> > be done to make it work always ?
> >>>>> >> >> >
> >>>>> >> >> > Thanks and Regards,
> >>>>> >> >> > Swapna.
> >>>>> >> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >
> >>>>> >
> >>>>
> >>>
> >>>
> >>
> >
>

Mime
View raw message