directory-api mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jitesh dundas <jbdun...@gmail.com>
Subject Re: Need Help with the LDAP API for large records
Date Wed, 16 Oct 2013 15:18:40 GMT
HI Kiran,

Thank you for your reply. You are correct in the sense that we close the
connection after every 1000 entries that are completed in the for loop and
then open the connection again.

*if (i%1000 == 0 || webphonePersons.size() == i){*
            client.closeConnection(connection);
            System.out.println("Completed => "+ i);
            Thread.sleep(4000);
            connection = client.getConnection();
}//end if condition

We are also putting Thread.sleep() to wait and give some time for the
connection to start/stop. Also, we have added the same sleep condition of
400 milliseconds after every entry that is added( to prevent overlaps).

However, based on your response, I am making the change to remove this
logic and keep one connection only throughout the loop. Hopefully it will
work.


Thanks & Regards,
Jitesh Dundas

Phone - 732-357-6292





On Wed, Oct 16, 2013 at 10:56 AM, Kiran Ayyagari <kayyagari@apache.org>wrote:

> On Wed, Oct 16, 2013 at 8:14 PM, jitesh dundas <jbdundas@gmail.com> wrote:
>
> > Hi Kiran,
> >
> > Thank you for your reply. My apologies for not being able to send the
> full
> > java file( As you can see that I am already doing that):-
> >
> >  public Main(String host, int port, String userName, String password) {
> >     this.host = host;
> >     this.port = port;
> >     this.userName = userName;
> >     this.password = password;
> >   }
> >
> >   public LdapConnection getConnection() throws LdapException,
> IOException {
> >     LdapConnection connection = new LdapNetworkConnection(host, port);
> >     connection.bind("uid=" + userName + ",ou=system", password);
> >     return connection;
> >   }
> >
> > Also, I see from the updated logs now that it traverses 3000 records and
> > then it starts giving these error messages. Any ideas?
> >
> > Maybe it is the way in which I am making the entries.
> >
> > if entry creation is wrong you will get a different error (instead of
> bind
> error)
> are you opening a new connection for adding each entry? it appears so to me
> after seeing the method public static void addEntryIntoSonarTestLdap(Entry
> labLdapEntry)
>
> > Thanks & Regards,
> > Jitesh Dundas
> >
> > Phone - 732-357-6292
> >
> >
> >
> >
> >
> > On Wed, Oct 16, 2013 at 10:26 AM, Kiran Ayyagari <kayyagari@apache.org
> > >wrote:
> >
> > > On Wed, Oct 16, 2013 at 7:51 PM, jitesh dundas <jbdundas@gmail.com>
> > wrote:
> > >
> > > > Hi All,
> > > >
> > > >  I am a new user who is implementing Apache LDAP on one of our
> > company's
> > > > servers. The records in the LDAP ( entries) come from a text file (
> > LDAP
> > > > dump) having 600000 records in it.
> > > >
> > > > I have written a utility to insert it buyt it keeps failing with
> these
> > > > errors:-
> > > >
> > > > [10:16:25] ERROR
> > > > [org.apache.directory.ldap.client.api.LdapNetworkConnection] - Bind
> > > failed
> > > > : timeout occurred
> > > > [10:16:25] ERROR
> > > > [org.apache.directory.ldap.client.api.LdapNetworkConnection] - The
> > > response
> > > > queue has been emptied, no response was found.
> > > > org.apache.directory.api.ldap.model.exception.LdapException: TimeOut
> > > > occurred
> > > >     at
> > > >
> > > >
> > >
> >
> org.apache.directory.ldap.client.api.LdapNetworkConnection.bind(LdapNetworkConnection.java:1089)
> > > >     at
> > > >
> > > >
> > >
> >
> org.apache.directory.ldap.client.api.AbstractLdapConnection.bind(AbstractLdapConnection.java:121)
> > > >     at
> > > >
> > > >
> > >
> >
> org.apache.directory.ldap.client.api.AbstractLdapConnection.bind(AbstractLdapConnection.java:106)
> > > >     at Main.getConnection(Main.java:38)
> > > >     at Main.main(Main.java:130)
> > > >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > >     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > >     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> > > >     at java.lang.reflect.Method.invoke(Unknown Source)
> > > >     at
> > > >
> > > >
> > >
> >
> org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
> > > >
> > > >
> > > > Here is the code to add it:-
> > > >
> > > >   public static void addEntryIntoSonarTestLdap(Entry labLdapEntry)
> > throws
> > > > LdapException, IOException {
> > > >       //Sonar Ldap Connection Details
> > > >       Main client = new Main("altv0018.aldc.att.com", 10389,
> "admin",
> > > > "secret");
> > > >
> > > replace "admin" with "uid=admin,ou=system" and try
> > >
> > > >
> > > >       List<WebphonePerson> webphonePersons =
> > > WebphoneFileReader.readFile();
> > > >       for(int i=6000; i<webphonePersons.size() ; i++){
> > > >           LdapConnection connection = client.getConnection();
> > > >           connection.setTimeOut(1000000);
> > > >           try {
> > > >
> > > >
> > > >                   //create the new entries to add.
> > > >                 Entry sonarLdapEntry = new
> > > > DefaultEntry(labLdapEntry.getDn().toString().trim());//dn of the
> entry
> > > goes
> > > > here.
> > > >                 sonarLdapEntry.add("objectClass","top");
> > > >                 sonarLdapEntry.add("objectClass","inetOrgPerson");
> > > >                 sonarLdapEntry.add("objectClass","person");
> > > >
> > sonarLdapEntry.add("objectClass","organizationalPerson");
> > > >                 sonarLdapEntry.add("cn",
> > > webphonePersons.get(i).getUID());
> > > >
> > > > sonarLdapEntry.add("sn",webphonePersons.get(i).getFirstName() + " " +
> > > > webphonePersons.get(i).getLastName());
> > > >
> > > sonarLdapEntry.add("uid",webphonePersons.get(i).getUID());
> > > >
> > > >
> sonarLdapEntry.add("userPassword","{AES256}MT3QA5xD8c1ZNy9EJSgW/Q==");
> > > >                 connection.add(sonarLdapEntry);
> > > >                 LOGGER.info("Sonar Test LDAP@Added User
> > > > sonarLdapEntry=>"+sonarLdapEntry.toString());
> > > >             } catch (Exception e) {
> > > >                 // TODO Auto-generated catch block
> > > >                 //e.printStackTrace();
> > > >                 System.out.println("UID FAILED => "+
> > > > webphonePersons.get(i).getAttUID());
> > > >             }
> > > >                   //closing connections after every 1000 records
> added
> > > and
> > > > then opened again.
> > > >                   //to prevent memory issues.
> > > >                   if (i%1000 == 0){
> > > >                 client.closeConnection(connection);
> > > >                 connection = client.getConnection();
> > > >                   }
> > > >             }
> > > >
> > > >     }
> > > >
> > > > Any ideas on how to resolve it? We have the Java memory options set
> to
> > > > -Xms1024m -Xmx1024m
> > > >
> > > > Thanks & Regards,
> > > > Jitesh Dundas
> > > >
> > > > Phone - 732-357-6292
> > > >
> > >
> > >
> > >
> > > --
> > > Kiran Ayyagari
> > > http://keydap.com
> > >
> >
>
>
>
> --
> Kiran Ayyagari
> http://keydap.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message