directory-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Carlo.Acco...@ibs-ag.com>
Subject RE: Plea for help with search performance
Date Wed, 11 Apr 2012 17:33:56 GMT
Emmanuel ok great.. I just responded to Kiran and realized the attachments don't come through.


I create everything with ldif files, restarting in between them. 
I start with a clean install, star the default instance once, the import the first file. 

## begin file 1##
#version: 1

dn: ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: top
objectClass: ads-base
objectclass: ads-partition
objectclass: ads-jdbmPartition
ads-indexes: apacheRdn
ads-indexes: apacheSubLevel
ads-indexes: apachePresence
ads-indexes: apacheOneLevel
ads-indexes: apacheOneAlias
ads-indexes: apacheSubAlias
ads-indexes: apacheAlias
ads-indexes: entryUUID
ads-indexes: entryCSN
ads-indexes: o
ads-indexes: krb5PrincipalName
ads-indexes: objectClass
ads-indexes: ou
ads-indexes: uid
ads-indexes: employeeNumber
ads-indexes: displayName
ads-indexes: cn
ads-indexes: mail
ads-partitionSuffix: o=cpro
ads-jdbmpartitionoptimizerenabled: TRUE
ads-partitioncachesize: 50000
ads-partitionsynconwrite: TRUE
ads-partitionid: cpro
ads-enabled: TRUE

dn: ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: organizationalUnit
objectclass: top
ou: indexes

dn: ads-indexAttributeId=apacheRdn,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apacheRdn
ads-enabled: TRUE
ads-indexcachesize: 5000

dn: ads-indexAttributeId=apacheSubLevel,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apacheSubLevel
ads-enabled: TRUE
ads-indexcachesize: 5000

dn: ads-indexAttributeId=apachePresence,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apachePresence
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=apacheOneLevel,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apacheOneLevel
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=apacheOneAlias,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apacheOneAlias
ads-enabled: TRUE
ads-indexcachesize: 5000

dn: ads-indexAttributeId=apacheSubAlias,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apacheSubAlias
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=apacheAlias,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: apacheAlias
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=o,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: o
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=krb5PrincipalName,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: krb5PrincipalName
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=objectClass,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: objectClass
ads-enabled: TRUE
ads-indexcachesize: 50000

dn: ads-indexAttributeId=entryUUID,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: entryUUID
ads-enabled: TRUE
ads-indexcachesize: 5000

dn: ads-indexAttributeId=entryCSN,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: entryCSN
ads-enabled: TRUE
ads-indexcachesize: 5000

dn: ads-indexAttributeId=ou,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: ou
ads-enabled: TRUE
ads-indexcachesize: 100

dn: ads-indexAttributeId=uid,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: uid
ads-enabled: TRUE
ads-indexcachesize: 50000

dn: ads-indexAttributeId=employeeNumber,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: employeeNumber
ads-enabled: TRUE
ads-indexcachesize: 50000

dn: ads-indexAttributeId=displayName,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-jdbmIndex
objectclass: ads-base
objectclass: top
ads-indexattributeid: displayName
ads-enabled: TRUE
ads-indexcachesize: 50000

dn: ads-indexAttributeId=cn,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-base
objectclass: ads-jdbmIndex
objectclass: top
ads-indexattributeid: cn
ads-enabled: TRUE
ads-indexcachesize: 50000

dn: ads-indexAttributeId=mail,ou=indexes,ads-partitionId=cpro,ou=partitions,ads-directoryServiceId=default,ou=config
objectclass: ads-index
objectclass: ads-base
objectclass: ads-jdbmIndex
objectclass: top
ads-indexattributeid: mail
ads-enabled: TRUE
ads-indexcachesize: 50000

#Change password policy to allow hashed passwords from ldif
dn: ads-pwdId=default,ou=passwordPolicies,ads-interceptorId=authenticationInterceptor,ou=interceptors,ads-directoryServiceId=default,ou=config
changetype: modify
replace: ads-pwdcheckquality
ads-pwdcheckquality: 0

## end file 1 ###

Then I restart the server and re connect

## begin file 2 ##
#version: 1

dn: o=cpro
objectClass: organization
objectClass: top
o: cpro

dn: ou=int,o=cpro
objectClass: organizationalUnit
objectClass: top
ou: int
description: Internal

dn: ou=users,ou=int,o=cpro
objectClass: organizationalUnit
objectClass: top
ou: users
description: Internal Users

dn: ou=groups,ou=int,o=cpro
objectClass: organizationalUnit
objectClass: top
ou: groups
description: Internal Groups

dn: ou=ext,o=cpro
objectClass: organizationalUnit
objectClass: top
ou: ext
description: External

dn: ou=users,ou=ext,o=cpro
objectClass: organizationalUnit
objectClass: top
ou: users
description: External Users

#External Password policy
dn: ads-pwdId=cproext,ou=passwordPolicies,ads-interceptorId=authenticationInterceptor,ou=interceptors,ads-directoryServiceId=default,ou=config
objectclass: top
objectclass: ads-base
objectclass: ads-passwordPolicy
ads-pwdattribute: userPassword
ads-pwdid: cproext
ads-enabled: TRUE
ads-pwdallowuserchange: TRUE
ads-pwdcheckquality: 0
ads-pwdexpirewarning: 600
ads-pwdfailurecountinterval: 30
ads-pwdgraceauthnlimit: 5
ads-pwdgraceexpire: 0
ads-pwdinhistory: 5
ads-pwdlockout: TRUE
ads-pwdlockoutduration: 0
ads-pwdmaxage: 0
ads-pwdmaxdelay: 0
ads-pwdmaxfailure: 5
ads-pwdmaxidle: 0
ads-pwdmaxlength: 0
ads-pwdminage: 0
ads-pwdmindelay: 0
ads-pwdminlength: 5
ads-pwdmustchange: FALSE
ads-pwdsafemodify: FALSE

dn: ads-pwdId=cproint,ou=passwordPolicies,ads-interceptorId=authenticationInterceptor,ou=interceptors,ads-directoryServiceId=default,ou=config
objectclass: ads-passwordPolicy
objectclass: ads-base
objectclass: top
ads-pwdattribute: userPassword
ads-pwdid: cproint
ads-enabled: TRUE
ads-pwdallowuserchange: TRUE
ads-pwdcheckquality: 0
ads-pwdexpirewarning: 10
ads-pwdfailurecountinterval: 30
ads-pwdgraceauthnlimit: 5
ads-pwdgraceexpire: 0
ads-pwdinhistory: 5
ads-pwdlockout: TRUE
ads-pwdlockoutduration: 0
ads-pwdmaxage: 60
ads-pwdmaxdelay: 0
ads-pwdmaxfailure: 5
ads-pwdmaxidle: 0
ads-pwdmaxlength: 0
ads-pwdminage: 0
ads-pwdmindelay: 0
ads-pwdminlength: 10
ads-pwdmustchange: FALSE
ads-pwdsafemodify: FALSE

## end file 2 ##

I then update the partition cache to 50000 and restart again , before importing any users.

Thanks. 

Regards,
Carlo Accorsi



-----Original Message-----
From: Emmanuel L├ęcharny [mailto:elecharny@gmail.com] 
Sent: Wednesday, April 11, 2012 1:17 PM
To: users@directory.apache.org
Subject: Re: Plea for help with search performance

Ok,

after some investigation, I'm pretty sure that the indexes are not created. When I create
the indexes, I get those results :

- 80000 entries injected into the server, with uid=<number>
- displayName is indexed, uid is not indexed

- first search :
connection.search("dc=example,dc=com", "(displayName=12345Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D)",
SearchScope.SUBTREE, "*" );

result :
Delta search : 16 ms
entry found :
     dn[n]: uid=12345,dc=example,dc=com
     objectclass: organizationalPerson
     objectclass: person
     objectclass: inetOrgPerson
     objectclass: top
     title: Snr Operations Technician (D)
     uid: 12345
     description: UI - S
     businesscategory: Ops MDS (Malaysia) Sdn Bhd
     departmentnumber: SMDS - UIA/G/MMO52D
     employeenumber: A-A-R.Awg-Rosli
     givenname: Awg-Abd-Rahim
     cn: Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D
     sn: Awg-Rosli
     telephonenumber: 555-1212
     mail: A-A-R.Awg-Rosli@acme.com
     displayname: 12345Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D

- Second search :
connection.search("dc=example,dc=com", "(displayName=34567*)", SearchScope.SUBTREE, "*" );

result :
Delta search substring : 9 ms
entry found :
     dn[n]: uid=34567,dc=example,dc=com
     objectclass: organizationalPerson
     objectclass: person
     objectclass: inetOrgPerson
     objectclass: top
     title: Snr Operations Technician (D)
     uid: 34567
     description: UI - S
     businesscategory: Ops MDS (Malaysia) Sdn Bhd
     departmentnumber: SMDS - UIA/G/MMO52D
     employeenumber: A-A-R.Awg-Rosli
     givenname: Awg-Abd-Rahim
     cn: Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D
     sn: Awg-Rosli
     telephonenumber: 555-1212
     mail: A-A-R.Awg-Rosli@acme.com
     displayname: 34567Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D

-third search :
connection.search("dc=example,dc=com", "(uid=67890)", SearchScope.SUBTREE, "*" );

result :
Delta search no index : 38985 ms

entry found :
     dn[n]: uid=67890,dc=example,dc=com
     objectclass: organizationalPerson
     objectclass: person
     objectclass: inetOrgPerson
     objectclass: top
     title: Snr Operations Technician (D)
     uid: 67890
     description: UI - S
     businesscategory: Ops MDS (Malaysia) Sdn Bhd
     departmentnumber: SMDS - UIA/G/MMO52D
     employeenumber: A-A-R.Awg-Rosli
     givenname: Awg-Abd-Rahim
     cn: Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D
     sn: Awg-Rosli
     telephonenumber: 555-1212
     mail: A-A-R.Awg-Rosli@acme.com
     displayname: 67890Awg-Rosli, Awg-Abd-Rahim SMDS-UIA/G/MMO52D


So if the index is properly set, the search takes milliseconds to complete (which is expected).
Otherwise, it can take tens of seconds...

Two possibilities :
- you don't initialize the index correctly. Please provide your configuration and if you are
using ADS embedded, please provide the code that you use to embed it
- we have an issue in the way we initialize the index, which is more likely to be the problem,
as I fixed one issue in this area 2 weeks ago (in M7-SNAPSHOT) and I just fixed another one
while doing the experiment...

I'd like to rule out the first hypothesis first, but in any case, I'll do some more check
tonite or tomorrow.


Many thanks for your patience.


-- 
Regards,
Cordialement,
Emmanuel L├ęcharny
www.iktek.com

Mime
View raw message