Mailing-List: contact api-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: api@directory.apache.org
Subject: Re: Fwd: Re: [LDAP API] Schema aware connection
To: api@directory.apache.org
References: <55F7FD75.9000503@gmail.com> <55F8116F.4000108@gmail.com>
 <55F81332.7000902@gmail.com>
From: Radovan Semancik <radovan.semancik@evolveum.com>
Message-ID: <55F92C5B.4060405@evolveum.com>
Date: Wed, 16 Sep 2015 10:46:19 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.2.0
MIME-Version: 1.0
In-Reply-To: <55F81332.7000902@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable

Hi,

I agree, of course.

Just a bit more information. Currently there is a way how to do this ...=20
kind of ...

			boolean schemaQuirksMode =3D ....
     			DefaultSchemaLoader schemaLoader =3D new DefaultSchemaLoader(conn=
ection, schemaQuirksMode);
     			DefaultSchemaManager schemaManager =3D new DefaultSchemaManager(s=
chemaLoader);
     			try {
     				if (schemaQuirksMode) {
         				schemaManager.setRelaxed();
         				schemaManager.loadAllEnabledRelaxed();
     				} else {
     					schemaManager.loadAllEnabled();
     				}
				} catch (Exception e) {
					throw new ConnectorIOException(e.getMessage(), e);
				}
     			if ( !schemaManager.getErrors().isEmpty() ) {
     				if (schemaQuirksMode) {
     					LOG.ok("There are {0} schema errors, but we are in quirks mode =
so we are ignoring them", defSchemaManager.getErrors().size());
     					for (Throwable error: schemaManager.getErrors()) {
     						LOG.ok("Schema error (ignored): {0}: {1}", error.getClass().ge=
tName(), error.getMessage());
     					}
     				} else {
     					throw new ConnectorIOException("Errors loading schema "+schemaM=
anager.getErrors());
     				}
     			}


This code works even with bad LDAP servers. But .... there are some=20
drawbacks:

1) The schemaManager cannot be attached to the connection. I.e. you must=20
not call connection.setSchemaManager(schemaManager). Otherwise the LDAP=20
operations will fail on many places as the loaded schema is=20
inconsistent. I guess that this is mostly OK. If the schema is not=20
consistent then it cannot be expected that it will work normally for all=20
the cases. But I believe it should work normally unless a the specific=20
inconsistency is encountered. I.e. if there are schema inconsistencies=20
in an attribute/syntax that is never used then th API should work=20
normally even with inconsistent schema.

2) Some errors do not appear in the error list=20
(schemaManager.getErrors()). E.g. the OID syntax error that I was fixing=20
recently. The error handling in schema processing is a bit inconsistent.=20
But I think this can be fixed in an evolutionary fashion.

3) This all not very user-friendly. The modifications that Emmanuel=20
suggested seem to be a good improvements.

4) We have quirks mode and relaxed mode. I've realized that quicks mode=20
is for parsing and relaxed mode is for schema processing and validation.=20
But, do we need to expose both to the API user? Maybe relaxed mode=20
should imply quirks mode.

To be more specific, I think we need this (ideally):

connection.loadSchema(mode)

... and internal implementation will set up quirks/relaxed mode as=20
appropriate. The current code is not very clean and I think that we will=20
need some "leeway" until we figure out what exactly needs to be done=20
here. Therefore we should provide a simple facade to API users that will=20
not change even if we change in internal interfaces (SchemaManager,=20
SchemaLoader).

To go one step further I guess we also need to clean up the=20
SchemaManager and SchemaLoader a bit. E.g. DefaultSchemaManager has=20
setRelaxed(), loadAllEnabled() and loadAllEnabledRelaxed(). One of these=20
methods seems to be redundant. I would expect either to use=20
setRelaxed(true) and then loadAllEnabled(), but as far as I can remember=20
that does not work. Speaking politically correctly this is a little bit=20
of a mess :-) I deserves to be cleaned up.

--=20
Radovan Semancik
Software Architect
evolveum.com


On 09/15/2015 02:46 PM, Emmanuel L=C3=A9charny wrote:
> Le 15/09/15 14:39, Emmanuel L=C3=A9charny a =C3=A9crit :
>> Forwarding this mail who deserves to be sent to API@directory.a.o...
>>
>>
>>
>> Le 15/09/15 12:03, Emmanuel L=C3=A9charny a =C3=A9crit :
>>> Hi,
>>>
>>> Radovan has added some way to support schemas that have some slight
>>> differences with the RFC, using the quirkmode flag. If you are using
>>> tehe NetworkLdapConnection, it works this way :
>>>
>>>                      SchemaLoader schemaLoader =3D new DefaultSchemaL=
oader(
>>> ldapConnection, true );
>>>                      ldapConnection.loadSchema( schemaLoader );
>>>
>>> Thats fine, except if you have declared your ldapConnection thsi way =
:
>>>
>>>                      LdapConnection ldapConnection =3D new
>>> LdapNetworkConnection( server.getServerName(), server.getPort() );
>>>
>>> because the LdapConnection interface does not expose the loadSchema(
>>> schemaLoader ) method.
>>>
>>> I suggest we expose it.
>>>
>>> I also think we should extend the ldapConnection.loadSchema() method,=
 in
>>> order to spare the users from declaring a SchemaLoader on their own.
>>> That would make the code looks like :
>>>
>>>
>>>                      ldapConnection.loadSchemaRelaxed();
>>>
>>> which would replace
>>>
>>>                      SchemaLoader schemaLoader =3D new DefaultSchemaL=
oader(
>>> ldapConnection, true );
>>>                      ldapConnection.loadSchema( schemaLoader );
>> It's a bit more complex that just using the quirkmode (which just rela=
x
>> the schema syntax). Sometime, we want to accept some inconsistencies i=
n
>> the schema itself. The relaxed mode is there to accept such schema, bu=
t
>> it's not exposed.
>>
>> Typically, the loadSchema() methods starts with :
>>
>>
>>      public void loadSchema( SchemaLoader loader ) throws LdapExceptio=
n
>>      {
>>          try
>>          {
>>              SchemaManager tmp =3D new DefaultSchemaManager( loader );
>>            =20
>>              tmp.loadAllEnabled();
>>
>>              if ( !tmp.getErrors().isEmpty() )
>>              {
>>                  String msg =3D "there are errors while loading the sc=
hema";
>>                  LOG.error( msg + " {}", tmp.getErrors() );
>>                  throw new LdapException( msg );
>>              }
>>              ...
>>
>> which creates a SchemaManager in strict mode. Obviously, the
>> tmp.loadAllEnabled() will fail. We should propagate the 'relaxed' flag
>> down to the SchemaManager.
>>
>>
>>
>>
> I will give a clear example :
>
> when you connect to an OpenLDAP server, and fetch the schema, you will
> get a failure because some MatchingRule are using a Syntax which is not
> loaded. Typically, a standard OpenLDAP server with core, cosine and
> inetorgperson schemas will have 248 ATs, 34 OCs, and 72 Syntaxes (from
> cn=3Dschema,ou=3Dconfig). The same server will expose only 32 Syntaxes =
(but
> 248 ATs and 63 OCs. Go fish...)
>
> The 40 missing Syntaxes are syntaxes which have no associated
> validators. That does not mean those syntax can't be used, though. The
> pb is that we don't see them in the SchemaLoader, and we default by
> rejecting the load.
>
> At this point, one solution would be to signal those errors, but don't
> stop and use default elements to replace the missing ones (ie, defining
> a Syntax with no specific check).
>
> I see no other ways to deal with that, becauee as Radovan says, it won'=
t
> be fixed any time soon...
>
> thoughts ?
>