directory-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [CONF] Apache Directory ASN.1 Documentation > ASN.1
Date Fri, 07 May 2010 17:19:00 GMT
    <base href="">
            <link rel="stylesheet" href="/confluence/s/1810/9/1/_/styles/combined.css?spaceKey=DIRxASN1&amp;forWysiwyg=true"
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="">ASN.1</a></h2>
    <h4>Page <b>edited</b> by             <a href="">Emmanuel
                         <h4>Changes (2)</h4>
<div id="page-diffs">
            <table class="diff" cellpadding="0" cellspacing="0">
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" > <br>h2. Implementations  <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">Two</span>
<span class="diff-added-words"style="background-color: #dfd;">Three</span> codecs
are currently available : <br></td></tr>
            <tr><td class="diff-unchanged" > * [LdapCodec|LdapCodec] which encodes
and decodes LDAP messages <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">
* [KerberosCodec|Kerberos] which encodes an decodes KERBEROS messages  <br></td></tr>
            <tr><td class="diff-unchanged" > * [SpnegoCodec] which encodes an
decodes SPNEGO messages <br></td></tr>
</div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h1><a name="ASN.1-ASN.1encoder%2Fdecoder"></a>ASN.1 encoder/decoder
    <li><a href='#ASN.1-ASN.1encoder%2Fdecoder'>ASN.1 encoder/decoder</a></li>
    <li><a href='#ASN.1-Components'>Components</a></li>
    <li><a href='#ASN.1-POJOs'>POJOs</a></li>
    <li><a href='#ASN.1-PDUs%2FTLVs'>PDUs/TLVs</a></li>
    <li><a href='#ASN.1-ByteBuffer'>ByteBuffer</a></li>
    <li><a href='#ASN.1-Compiler'>Compiler</a></li>
    <li><a href='#ASN.1-Processing'>Processing</a></li>
    <li><a href='#ASN.1-Encoder'>Encoder</a></li>
    <li><a href='#ASN.1-Decoder'>Decoder</a></li>
    <li><a href='#ASN.1-Performance'>Performance</a></li>
    <li><a href='#ASN.1-Implementations'>Implementations</a></li>

<h2><a name="ASN.1-Components"></a>Components </h2>

<p>Communications between clients and server can be seen as a two ways / multi <em>layers</em>
system. The client submits a request to the server, which replies.</p>

<p><em>Layers</em> are just used to facilitate the implementation of this
communication. From the developer point of view, working on a specific level, he just has
to know the two layers above and under, but can be seen as a communication at she same level
between the client and the server. Here is a view of these layers :</p>

<p><span class="image-wrap" style=""><img src="/confluence/download/attachments/44491/cs-stack.png?version=1&amp;modificationDate=1171390134000"
style="border: 0px solid black" /></span></p>

<p>We have depicted three layers:</p>
	<li>Request/Response: This is the more abstract layer. Exchanged messages are 'human
readable'. Each message is a Java Bean, containing all the information about a Request or
a Response.</li>
	<li>PDU: As communication petween the Client and the Server is done through a network,
we need to transform the beans to something more 'network friendly'. The data are stored in
PDU, or <b>P</b>rotocol <b>D</b>ata <b>U</b>nit. Those
PDU contain an encoded form of messages, specified in <a href=""
class="external-link" rel="nofollow">RFC 2251</a> and <a href=""
class="external-link" rel="nofollow">ASN.1</a></li>
	<li><a href="" class="external-link"
rel="nofollow">ByteBuffers</a>: To transfer PDU from/to Client to/from Server, we
need to store bytes in a structure that will permit to deal with network latency. Thus we
are using byte buffers, which allow us to send pieces of PDU until the whole PDU has been
transmitted. (Note : <a href=""
class="external-link" rel="nofollow">ByteBuffer</a> is also a Java NIO class, but
can be seen just as a byte container. It could have been something totally different from
the NIO class).</li>

<p>This layering allows many different implementations. </p>

<p>One can also imagine inter-layers used to trace debug informations.</p>

<p>Inter layer communication rely on a pipe-line: each layer push some piece of information
to the next layer (up or down), and so on.</p>

<p>Each layer may implement its own strategy to fulfill the reception and transmission
of the data it is responsible of :</p>
		<li>asynchronous push</li>
		<li>synchronous push</li>
		<li>established and dedicated channel</li>
		<li>multiplexed channel</li>
		<li>established and dedicated channel</li>
		<li>multiplexed channel</li>

<h3><a name="ASN.1-POJOs"></a>POJOs </h3>
<p><b>POJOs</b> are Java classes that contain high level informations.</p>

<p>A client create an <em>instance</em> of a class to communicate with the
server, which create an other one to reply. They implement a kind of <em>application
layer</em> between clients and server.</p>

<p>Ideally, they are generated by an <b>ASN.1</b> compiler, but can be hand

<h3><a name="ASN.1-PDUs%2FTLVs"></a>PDUs/TLVs </h3>

<p>PDU stands for <b>P</b>rotocol <b>D</b>ata <b>U</b>nit.
An ASN.1 encoded element is stored in a PDU. This is what is transfered between a client and
a server.</p>

<p>TLV stands for <b>Type/Length/Value</b>. A PDU is made of <b>TLV</b>
s. Each <b>TLV</b> represent either a primitive element, and it has a <b>V</b>alue,
or a constructed element, and the <b>V</b>alue is itself one ore more <b>TLV</b>
(The <b>V</b> can contain more than one <b>TLV</b>). The <b>PDU</b>
structure is like a tree, where the <b>PDU</b> is the whole tree, and where <b>TLV</b>
are leaves (primitives) and branches (constructed)</p>

<p>Further information about <b>TLV</b>s can be found here :</p>
	<li><a href="/confluence/display/DIRxASN1/TLV+Page+Info" title="TLV Page Info">TLV
Page Info</a>: Informations about <b>Tlv</b>s</li>

<h3><a name="ASN.1-ByteBuffer"></a>ByteBuffer </h3>
<p>Buffering the incoming request or the ourgoing response is essential. As a request
or a response can be huge (for example, if we want to store images), it is necessary to store
bytes in buffers in order to be able to pipeline the processing. Flushing informations byte
by byte is totally insane, from the network point of view.</p>

<p>We are using the <b>NIO</b> <a href=""
class="external-link" rel="nofollow">ByteBuffer</a> structure to store chunks of
information, before pushing them on the network, and reversly, store incoming bytes into buffers
before processing the request.</p>

<h2><a name="ASN.1-Compiler"></a>Compiler </h2>

<p>TO BE DONE ...</p>

<h2><a name="ASN.1-Processing"></a>Processing </h2>

<p>There are two kind of processing: <b>encoding</b> and <b>decoding</b>.
Encoding is quite easy, decoding is much more complicated.</p>

<div class='panelMacro'><table class='infoMacro'><colgroup><col width='24'><col></colgroup><tr><td
valign='top'><img src="/confluence/images/icons/emoticons/information.gif" width="16"
height="16" align="absmiddle" alt="" border="0"></td><td><b>Important</b><br
/>Important : decoding an ASN.1 PDU is generally not possible if you have no knowledge
of the grammar being decoded. To limit the size of PDUs, the encoding schemes used (PER, DER,
BER, ...) permits the elimination of some TL if the constructed TLV that encapsulate the previous
one is unambiguiously known. One who want to decode a PDU <b>MUST</b> know which
grammar has been used.</td></tr></table></div>

<h3><a name="ASN.1-Encoder"></a>Encoder</h3>

<p>The encoding process is quite easy. As we know what has to be encoded, the structure
of the PDU is somehow dependent on the structure of the POJO which contains the data. The
only tricky things is the <b>Length</b> part, which has to be computed. As a <b>TLV</b>
may have a <b>V</b> part which is itself one or more <b>TLV</b> s,
its <b>L</b> part will be the sum of each included <b>TLV</b> s length.
This is typically a recursive processing, but we can also process the POJO in two passes :</p>
	<li>the first pass compute each length</li>
	<li>the second pass generate the <b>PDU</b></li>

<p>The <a href="/confluence/pages/createpage.action?spaceKey=DIRxSBOX&amp;title=Encoding+Asn.1"
class="createlink">DIRxSBOX:Encoding Asn.1</a> page gives informations about the
encoding process.</p>

<h3><a name="ASN.1-Decoder"></a>Decoder </h3>
<p>The decoding process is a loop which reads PDUs and constructs objects on the fly.
It can stop and restart without loosing information, as PDU may be very long (it also means
that we must store a current state for each decoding).</p>

<p>The <a href="/confluence/pages/createpage.action?spaceKey=DIRxSBOX&amp;title=Decoding+Asn.1"
class="createlink">DIRxSBOX:Decoding Asn.1</a> page gives informations about the
encoding process.</p>

<h3><a name="ASN.1-Performance"></a>Performance</h3>
<p>TODO : performance against memory/scalability/failover<br/>
TODO : which kind of performance should we deliver? Maximum throughput = bandwith/average
PDU size. For instance, with a 1Gb network connection, assuming that we have an average PDU
size of 100 bytes, the system must deliver 1 M Pdu/s to saturate the network.</p>

<p>Actually, the new decoder eats 110 000 <a href="/confluence/display/DIRxASN1/BindRequest"
title="BindRequest">BindRequest</a> PDU or 37 000 SearchResultEntry PDU per second
on my 2.8Ghz computer, but we have to take into account the works that must be done aside.</p>

<h2><a name="ASN.1-Implementations"></a>Implementations </h2>
<p>Three codecs are currently available :</p>
	<li><a href="/confluence/display/DIRxASN1/LdapCodec" title="LdapCodec">LdapCodec</a>
which encodes and decodes LDAP messages</li>
	<li><a href="/confluence/display/DIRxASN1/Kerberos" title="Kerberos">KerberosCodec</a>
which encodes an decodes KERBEROS messages</li>
	<li><a href="/confluence/display/DIRxASN1/SpnegoCodec" title="SpnegoCodec">SpnegoCodec</a>
which encodes an decodes SPNEGO messages</li>

        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href=""
class="grey">Change Notification Preferences</a>
        <a href="">View Online</a>
        <a href="">View

View raw message