Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of pajbam@gmail.com designates
 74.125.82.178 as permitted sender)
Sender: Pierre-Arnaud Marcelot <pajbam@gmail.com>
From: Pierre-Arnaud Marcelot <pa@marcelot.net>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_168098E1-3DA2-4131-AF57-703426A1D0DE"
Subject: [Studio][API] Binary attributes corruption issue
Date: Fri, 17 Feb 2012 15:50:16 +0100
Message-Id: <6F5D30FA-F438-4784-AB67-C8CF389EED65@marcelot.net>
To: Apache Directory Developers List <dev@directory.apache.org>
Mime-Version: 1.0 (Apple Message framework v1257)


--Apple-Mail=_168098E1-3DA2-4131-AF57-703426A1D0DE
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1

Hi guys,

Just a status report on the major issue we have in the API and Apache =
Directory Studio regarding binary attributes corruption.

For more background about this, read the following JIRA issues:
- [1] DIRAPI-78 (Binary attributes are not correctly recognized and are =
retrieved with data corruption)
- [2] DIRSTUDIO-772 (Entry UUID Editor can't display eDirectory GUID)
- [3] DIRSTUDIO-764 (Image Editor does not work correctly)

In a nutshell, due to the lack of schema manager, the LDAP API, which is =
used inside Studio, has no knowledge about binary attributes and handles =
all values as if they were string value.
Do so, for binary attributes, the received data is not correct and ends =
up being corrupted.

In order to fix this issue, Emmanuel and I worked on adding 2 ways for =
users to have binary attributes handled correctly.

1=B0/ The lightweight solution: the BinaryAttributeDetector

A DefaultConfigurableBinaryAttributeDetector class has been added to the =
configuration of an LDAP connection.
By default it includes a list of known binary attributes and methods to =
add and removed values to/from this list.
A user can also provide his own detector implementation via the =
BinaryAttributeDetector interface.

When the LDAP connection retrieves a new attribute value from a server, =
it will check via the BinaryAttributeDetector whether or not the =
attribute is binary and get the value according to this flag.

This first simple solution has been completely implemented and works =
fine for both the API and Studio.

2=B0/ The complete solution: loading the remote schema

The second solution is to make the connection schema-aware by allowing =
the load of the schema remotely.
It requires a lot more work as we're trying to load the whole schema of =
the server via LDAP through the SubSchemaSubEntry.

This solution is much more complex to implement than it seems and there =
are a lot of different corner cases:
- whether or not the SubSchemaSubEntry is exposed
- the kind of server (ApacheDS, OpenLDAP, OpenDJ, Active Directory, etc)
- is the schema complete and correct?

The more we implement this solution, the more we're hitting important =
issues with the internal design of the schema and how it is loaded =
(through the schema manager).

Implementing this second solution requires heavy refactoring of several =
crucial parts of the code (schema elements, schema loading objects, =
etc.) and might take longer than expected.

In the meantime, users of Studio and the LDAP API keep getting corrupted =
values for binary attributes. :(

I'd like to propose that we provide a new milestone for both projects =
(Studio 2.0 M3 and LDAP API 1.0 M11) that includes only the first =
solution (we also already have 15 resolved issues pending for Studio and =
5 for API).

The second solution could then take place in the next milestone and we =
could take the appropriate amount of time to implement it correctly.

WDYT?

Regards,
Pierre-Arnaud

[1] - https://issues.apache.org/jira/browse/DIRAPI-78
[2] - https://issues.apache.org/jira/browse/DIRSTUDIO-772
[3] - https://issues.apache.org/jira/browse/DIRSTUDIO-764=

--Apple-Mail=_168098E1-3DA2-4131-AF57-703426A1D0DE
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=iso-8859-1

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi =
guys,<div><br></div><div>Just a status report on the major issue we have =
in the API and Apache Directory Studio regarding binary attributes =
corruption.</div><div><br></div><div>For more background about this, =
read the following JIRA issues:</div><div>- [1] DIRAPI-78 (Binary =
attributes are not correctly recognized and are retrieved with data =
corruption)</div><div>- [2] DIRSTUDIO-772 (Entry UUID Editor can't =
display eDirectory GUID)</div><div>- [3]&nbsp;DIRSTUDIO-764 (Image =
Editor does not work correctly)</div><div><br></div><div>In a nutshell, =
due to the lack of schema manager, the LDAP API, which is used inside =
Studio, has no knowledge about binary attributes and handles all values =
as if they were string value.</div><div>Do so, for binary attributes, =
the received data is not correct and ends up being =
corrupted.</div><div><br></div><div>In order to fix this issue, Emmanuel =
and I worked on adding 2 ways for users to have binary attributes =
handled correctly.</div><div><br></div><div>1=B0/ The lightweight =
solution: the =
BinaryAttributeDetector</div><div><br></div><div>A&nbsp;DefaultConfigurabl=
eBinaryAttributeDetector class has been added to the configuration of an =
LDAP connection.</div><div>By default it includes a list of known binary =
attributes and methods to add and removed values to/from this =
list.</div><div>A user can also provide his own detector implementation =
via the&nbsp;BinaryAttributeDetector =
interface.</div><div><br></div><div>When the LDAP connection retrieves a =
new attribute value from a server, it will check via =
the&nbsp;BinaryAttributeDetector whether or not the attribute is binary =
and get the value according to this flag.</div><div><br></div><div>This =
first simple solution has been completely implemented and works fine for =
both the API and Studio.</div><div><br></div><div>2=B0/ The complete =
solution: loading the remote schema</div><div><br></div><div>The second =
solution is to make the connection schema-aware by allowing the load of =
the schema remotely.</div><div>It requires a lot more work as we're =
trying to load the whole schema of the server via LDAP through the =
SubSchemaSubEntry.</div><div><br></div><div>This solution is much more =
complex to implement than it seems and there are a lot of different =
corner cases:</div><div>- whether or not the&nbsp;SubSchemaSubEntry is =
exposed</div><div>- the kind of server (ApacheDS, OpenLDAP, OpenDJ, =
Active Directory, etc)</div><div>- is the schema complete and =
correct?</div><div><br></div><div>The more we implement this solution, =
the more we're hitting important issues with the internal design of the =
schema and how it is loaded (through the schema =
manager).</div><div><br></div><div>Implementing this second solution =
requires heavy refactoring of several crucial parts of the code (schema =
elements, schema loading objects, etc.) and might take longer than =
expected.</div><div><br></div><div>In the meantime, users of Studio and =
the LDAP API keep getting corrupted values for binary attributes. =
:(</div><div><br></div><div>I'd like to propose that we provide a new =
milestone for both projects (Studio 2.0 M3 and LDAP API 1.0 M11) that =
includes only the first solution (we also already have 15 resolved =
issues pending for Studio and 5 for API).</div><div><br></div><div>The =
second solution could then take place in the next milestone and we could =
take the appropriate amount of time to implement it =
correctly.</div><div><br></div><div>WDYT?</div><div><br></div><div>Regards=
,</div><div>Pierre-Arnaud</div><div><br></div><div>[1] -&nbsp;<a =
href=3D"https://issues.apache.org/jira/browse/DIRAPI-78">https://issues.ap=
ache.org/jira/browse/DIRAPI-78</a></div>[2] -&nbsp;<a =
href=3D"https://issues.apache.org/jira/browse/DIRSTUDIO-772">https://issue=
s.apache.org/jira/browse/DIRSTUDIO-772</a><div>[3] -&nbsp;<a =
href=3D"https://issues.apache.org/jira/browse/DIRSTUDIO-764">https://issue=
s.apache.org/jira/browse/DIRSTUDIO-764</a></div></body></html>=

--Apple-Mail=_168098E1-3DA2-4131-AF57-703426A1D0DE--