directory-api mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiran Ayyagari <kayyag...@apache.org>
Subject Re: Need Help with the LDAP API for large records
Date Wed, 16 Oct 2013 15:26:05 GMT
On Wed, Oct 16, 2013 at 8:48 PM, jitesh dundas <jbdundas@gmail.com> wrote:

> HI Kiran,
>
> Thank you for your reply. You are correct in the sense that we close the
> connection after every 1000 entries that are completed in the for loop and
> then open the connection again.
>
> *if (i%1000 == 0 || webphonePersons.size() == i){*
>             client.closeConnection(connection);
>             System.out.println("Completed => "+ i);
>             Thread.sleep(4000);
>             connection = client.getConnection();
> }//end if condition
>
> We are also putting Thread.sleep() to wait and give some time for the
> connection to start/stop. Also, we have added the same sleep condition of
> 400 milliseconds after every entry that is added( to prevent overlaps).
>
> However, based on your response, I am making the change to remove this
> logic and keep one connection only throughout the loop. Hopefully it will
> work.
>
> not that your method is wrong, but one connection is enough here
otoh, which version of ApacheDS you are using?

>
> Thanks & Regards,
> Jitesh Dundas
>
> Phone - 732-357-6292
>
>
>
>
>
> On Wed, Oct 16, 2013 at 10:56 AM, Kiran Ayyagari <kayyagari@apache.org
> >wrote:
>
> > On Wed, Oct 16, 2013 at 8:14 PM, jitesh dundas <jbdundas@gmail.com>
> wrote:
> >
> > > Hi Kiran,
> > >
> > > Thank you for your reply. My apologies for not being able to send the
> > full
> > > java file( As you can see that I am already doing that):-
> > >
> > >  public Main(String host, int port, String userName, String password) {
> > >     this.host = host;
> > >     this.port = port;
> > >     this.userName = userName;
> > >     this.password = password;
> > >   }
> > >
> > >   public LdapConnection getConnection() throws LdapException,
> > IOException {
> > >     LdapConnection connection = new LdapNetworkConnection(host, port);
> > >     connection.bind("uid=" + userName + ",ou=system", password);
> > >     return connection;
> > >   }
> > >
> > > Also, I see from the updated logs now that it traverses 3000 records
> and
> > > then it starts giving these error messages. Any ideas?
> > >
> > > Maybe it is the way in which I am making the entries.
> > >
> > > if entry creation is wrong you will get a different error (instead of
> > bind
> > error)
> > are you opening a new connection for adding each entry? it appears so to
> me
> > after seeing the method public static void
> addEntryIntoSonarTestLdap(Entry
> > labLdapEntry)
> >
> > > Thanks & Regards,
> > > Jitesh Dundas
> > >
> > > Phone - 732-357-6292
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Oct 16, 2013 at 10:26 AM, Kiran Ayyagari <kayyagari@apache.org
> > > >wrote:
> > >
> > > > On Wed, Oct 16, 2013 at 7:51 PM, jitesh dundas <jbdundas@gmail.com>
> > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > >  I am a new user who is implementing Apache LDAP on one of our
> > > company's
> > > > > servers. The records in the LDAP ( entries) come from a text file
(
> > > LDAP
> > > > > dump) having 600000 records in it.
> > > > >
> > > > > I have written a utility to insert it buyt it keeps failing with
> > these
> > > > > errors:-
> > > > >
> > > > > [10:16:25] ERROR
> > > > > [org.apache.directory.ldap.client.api.LdapNetworkConnection] - Bind
> > > > failed
> > > > > : timeout occurred
> > > > > [10:16:25] ERROR
> > > > > [org.apache.directory.ldap.client.api.LdapNetworkConnection] - The
> > > > response
> > > > > queue has been emptied, no response was found.
> > > > > org.apache.directory.api.ldap.model.exception.LdapException:
> TimeOut
> > > > > occurred
> > > > >     at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.directory.ldap.client.api.LdapNetworkConnection.bind(LdapNetworkConnection.java:1089)
> > > > >     at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.directory.ldap.client.api.AbstractLdapConnection.bind(AbstractLdapConnection.java:121)
> > > > >     at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.directory.ldap.client.api.AbstractLdapConnection.bind(AbstractLdapConnection.java:106)
> > > > >     at Main.getConnection(Main.java:38)
> > > > >     at Main.main(Main.java:130)
> > > > >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > >     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > > >     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > > > >     at java.lang.reflect.Method.invoke(Unknown Source)
> > > > >     at
> > > > >
> > > > >
> > > >
> > >
> >
> org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
> > > > >
> > > > >
> > > > > Here is the code to add it:-
> > > > >
> > > > >   public static void addEntryIntoSonarTestLdap(Entry labLdapEntry)
> > > throws
> > > > > LdapException, IOException {
> > > > >       //Sonar Ldap Connection Details
> > > > >       Main client = new Main("altv0018.aldc.att.com", 10389,
> > "admin",
> > > > > "secret");
> > > > >
> > > > replace "admin" with "uid=admin,ou=system" and try
> > > >
> > > > >
> > > > >       List<WebphonePerson> webphonePersons =
> > > > WebphoneFileReader.readFile();
> > > > >       for(int i=6000; i<webphonePersons.size() ; i++){
> > > > >           LdapConnection connection = client.getConnection();
> > > > >           connection.setTimeOut(1000000);
> > > > >           try {
> > > > >
> > > > >
> > > > >                   //create the new entries to add.
> > > > >                 Entry sonarLdapEntry = new
> > > > > DefaultEntry(labLdapEntry.getDn().toString().trim());//dn of the
> > entry
> > > > goes
> > > > > here.
> > > > >                 sonarLdapEntry.add("objectClass","top");
> > > > >                 sonarLdapEntry.add("objectClass","inetOrgPerson");
> > > > >                 sonarLdapEntry.add("objectClass","person");
> > > > >
> > > sonarLdapEntry.add("objectClass","organizationalPerson");
> > > > >                 sonarLdapEntry.add("cn",
> > > > webphonePersons.get(i).getUID());
> > > > >
> > > > > sonarLdapEntry.add("sn",webphonePersons.get(i).getFirstName() + "
> " +
> > > > > webphonePersons.get(i).getLastName());
> > > > >
> > > > sonarLdapEntry.add("uid",webphonePersons.get(i).getUID());
> > > > >
> > > > >
> > sonarLdapEntry.add("userPassword","{AES256}MT3QA5xD8c1ZNy9EJSgW/Q==");
> > > > >                 connection.add(sonarLdapEntry);
> > > > >                 LOGGER.info("Sonar Test LDAP@Added User
> > > > > sonarLdapEntry=>"+sonarLdapEntry.toString());
> > > > >             } catch (Exception e) {
> > > > >                 // TODO Auto-generated catch block
> > > > >                 //e.printStackTrace();
> > > > >                 System.out.println("UID FAILED => "+
> > > > > webphonePersons.get(i).getAttUID());
> > > > >             }
> > > > >                   //closing connections after every 1000 records
> > added
> > > > and
> > > > > then opened again.
> > > > >                   //to prevent memory issues.
> > > > >                   if (i%1000 == 0){
> > > > >                 client.closeConnection(connection);
> > > > >                 connection = client.getConnection();
> > > > >                   }
> > > > >             }
> > > > >
> > > > >     }
> > > > >
> > > > > Any ideas on how to resolve it? We have the Java memory options set
> > to
> > > > > -Xms1024m -Xmx1024m
> > > > >
> > > > > Thanks & Regards,
> > > > > Jitesh Dundas
> > > > >
> > > > > Phone - 732-357-6292
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Kiran Ayyagari
> > > > http://keydap.com
> > > >
> > >
> >
> >
> >
> > --
> > Kiran Ayyagari
> > http://keydap.com
> >
>



-- 
Kiran Ayyagari
http://keydap.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message