directory-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Directory Wiki] Update of "Asn1Home" by EmmanuelLecharny
Date Wed, 30 Mar 2005 22:25:19 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Directory Wiki" for change notification.

The following page has been changed by EmmanuelLecharny:
http://wiki.apache.org/directory/Asn1Home

------------------------------------------------------------------------------
  
  
  
- === Stubs ===
+ === Stubs/POJOs ===
- '''Stubs''' are Java classes that contain high level informations.
+ '''Stubs''' or '''POJOs''' are Java classes that contain high level informations.
  
  A client create an ''instance'' of a stub to communicate with the server, which create an
other one to reply. They implement a kind of ''application layer'' between clients and server.
  
  
  Ideally, they are generated by an '''ASN.1''' compiler, but can be hand crafted.
  
- === Tuples ===
- 
  === TLVs ===
   * ["TLVPageInfo"]: Informations about '''Tlv'''s
- === Buffers ===
  
  === IO ===
  
@@ -80, +77 @@

  === Encoder ===
  
  === Decoder ===
+ The decoding process is an infinite loop that can read a PDU. A PDU is composed of TLVs
which could have sub-TLVs and so on.
  
- == Bottlenecks ==
  
  === Performance ===
  TODO : performance against memory/scalability/failover
- TODO : which kind of performance should we deliver? Maximum throughput = bandwith/average
PDU size. For instance, with a 1Gb network connection, assuming that we have an average PDU
size of 100 bytes, the system should deliver 1 M Pdu/s, not to mention TCP/IP overloading
(typically, a TCP/IP packet size is 1500 bytes, so the previous number is to be divided by
15 : 66 000 PDU/s)
+ TODO : which kind of performance should we deliver? Maximum throughput = bandwith/average
PDU size. For instance, with a 1Gb network connection, assuming that we have an average PDU
size of 100 bytes, the system should deliver 1 M Pdu/s, not to mention TCP/IP overloading
(typically, a TCP/IP packet size is 1500 bytes, so the previous number is to be divided by
15 : 66 000 PDU/s).
+ 
+ Actually, the new decoder eats 130 000 BindRequest PDU per second, but we have to take into
account the works that must be done aside.
  
  So, ideally, treating a PDU in 15 ns might be enough.
  
@@ -101, +100 @@

  
  A global pool of instance could be allocated on initialization, depending on the global
memory. each thread will ask for an instance and will keep it in its own pool.
  
+ The new implementation create a pool per object's type for each decoding thread, and a global
synchronized pool. When a thread need a new object, either it find it in its local pool, or
it asks for a new one from the global pool.
- {{{
- instances
-  ^X
-  |
-  | X
-  |
-  |  X
-  |
-  |   X                                  2
-  |                       2
-  |     X       2                      1
-  |         2       1
-  |      2 X1
-  |    12      X
-  |  12               X
-  | 12                               X
-  |12
-  +----------------------------------------> time
  
+ To limit the memory consumption, we must also implement a requisition mechanism where the
exhausted global pool asks local pools some free objects. When no more objects are available
on local pools, a choice must be made to kill or serialize a decoder, freing the associated
objects. This mechanism is still to be implemented.
-  X : instance in the global pool
-  1 : instance in the thread-1 pool
-  2 : instance in the thread-2 pool
-  ...
- }}}
  
- === Network consumption ===
+ Another problem is the String allocations. Strings are immutable, so they can't be reused.
It's not possible to create a String pool, so memory is not manageable with Strings. We have
two ways here :
+  - create a new MutableString class, which is quite a piece of work, because we need to
handle UTF-8
+  - pre-allocate as much as String than we can, up to the StringPool limit, and each time
we need to create a new String, we will just have to nullify the pre-allocated one. As String
could have different sizes, we have to pre-allocate Strings with different sizes, like 16-chars
length Strings, 32-chars Strings, and so on. If we have to allocate, say, a 44 char long String,
we will free a 48-chars String, so the memory will nevcer be exhausted. If there is no more
48-chars String available, then we could free either a 64-chars String (or a bigger one),
or free one 32-chars String and a 16-chars String. Strings above 1024 bytes will be streamed.
+ 
+ This may seems very complicated, but we have to deal with this kind of constraints, as the
PDU that we can receive may be really strange, or built to break the server. 
+ 
  
  
  == LDAP ASN.1 State Automaton ==
+ 
+ LDAP Grammar can be seen as a State Automaton. 
+ 
+ attachment:LdapMessage.png
+ 
+ === BindRequest ===
+ 
+ attachement:BindRequest.png
  
  Here is the LDAP state automaton :
  
Mime
View raw message