directory-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Directory Wiki] Update of "MinaIrcMeeting/20060509" by CKoppelt
Date Thu, 15 Feb 2007 23:56:04 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Directory Wiki" for change notification.

The following page has been changed by CKoppelt:
http://wiki.apache.org/directory/MinaIrcMeeting/20060509

------------------------------------------------------------------------------
- = Summary =
+ deleted
  
-  * All reported bugs have to get resolved in the next release.
-  * Discussed issues:
-    * [http://issues.apache.org/jira/browse/DIRMINA-206 Prevent OutOfMemoryError when a server
or a client is overloaded]
-      * Will discuss further in the mailing list.
-    * [http://issues.apache.org/jira/browse/DIRMINA-194 Extract Reactor from Acceptor and
Connector like ACE in C++.]
-      * No interest yet; won't fix for now
-    * [http://issues.apache.org/jira/browse/DIRMINA-166 Common internal messages that can
be shared by filters and transport implementations]
-      * Just provide a callback interface instead of internal messages - this will affect
ApacheDS, too, so we need to synchronize this change in the next release.
-    * [http://issues.apache.org/jira/browse/DIRMINA-208 Allow datagrams to be directed to
a specific destination on a per-datagram instead of a per-session basis.]
-      * Provide BroadcastSession which extends DatagramSession, so users can downcast
-        * setRemoteAddress()
-        * write(SocketAddress, Object)
-    * [http://issues.apache.org/jira/browse/DIRMINA-29 JMX integration]
-      * Trustin will review Juilien's patch and apply it.
-  * It would be nice if we do IRC meetings periodically.
- 
- 
- = Full Log =
- 
- {{{
- 22:10:25 <trustin>      proyal, are you back?
- 22:12:28              * doniv (n=Vinod@59.182.0.189) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 22:12:46 <trustin>      Hi doniv
- 22:12:53 <doniv>        hi guys.. sorry i'm a bit late
- 22:13:03 <trustin>      no worries.  we didn't start yet
- 22:13:03              * doniv is Vinod Panicker
- 22:13:12 <doniv>        oh cool
- 22:13:27 <trustin>      Other committers are not responding so we're waiting for them
for a while.
- 22:13:37 <doniv>        alright
- 22:13:47 <trustin>      Let's wait for another 7 minutes and start.
- 22:14:07 <vrm>          Hi
- 22:15:18 <doniv>        hi vrm
- 22:15:21 <doniv>        vrm = ?
- 22:15:28 <vrm>          Julien Vermillard
- 22:15:33 <vrm>          try /whois :)
- 22:15:44 <proyal>       trustin: i'm back :)
- 22:15:48 <proyal>       was i off by an hour?
- 22:15:50 <trustin>      Great.
- 22:15:55 <proyal>       (got here an hour ago thinking it was then)
- 22:16:02 <trustin>      Dunno :)
- 22:16:10 <proyal>       heh, good then :)
- 22:16:14 <trustin>      It seems like Dave has a firewall problem.
- 22:16:33 <trustin>      I pinged him several minutes ago but it seems like he's away
from his PC now.
- 22:16:54 <trustin>      It's too bad that niklas is not here.
- 22:17:14 <trustin>      Anyway... shall we start?
- 22:17:34 <proyal>       yup!
- 22:17:42 <trustin>      OK..
- 22:18:33 <trustin>      As all you guys know we've been releasing 5 releases since
0.8.2 is released.
- 22:19:01 <trustin>      But we still have a lot of issues to resolve:
- 22:19:16 <trustin>      http://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&pid=10670&status=1
- 22:19:42              * luke_red5 (n=luke@203.209.43.193) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 22:19:51 <trustin>      And our feature addition is slowing down these days unfortunately
for some reason.
- 22:20:16 <trustin>      I got a new job and it's taking up quite a lot of time and
I guess it is same for other contributors.
- 22:20:25 <luke_red5>    ello all, I will probably just hang out, and listen in. 
- 22:20:31 <trustin>      Hi luke_red5 
- 22:20:36 <trustin>      Good to see you!
- 22:21:09 <luke_red5>    :) thanks for working on mina guys. 
- 22:21:10 <trustin>      In this meeting, I'd like to know which issues are on demand
and therefore required to be fixed sooner.
- 22:21:39 <trustin>      .. so we can set our road map to 1.0 more effectively.
- 22:22:18 <proyal>       one thing that would help, is if people voted on issues in
jira that they care about
- 22:22:30 <doniv>        proyal: thats a good idea
- 22:22:34 <trustin>      Sounds great.
- 22:22:45 <doniv>        how many votes are available?
- 22:22:59 <proyal>       i believe that every user can vote for an issue once
- 22:23:19 <proyal>       yes, you get one vote per issue
- 22:23:32 <doniv>        cumulative - no limit?
- 22:23:35 <proyal>       correct, no limit
- 22:23:48 <trustin>      JIRA is very slow
- 22:23:55 <doniv>        lol, yea
- 22:24:08 <doniv>        is it congenital or is it coz of the hardware?
- 22:24:18 <trustin>      How can I list issues which are voted mostly?
- 22:24:24 <proyal>       partially congenital, since iirc they don't ship with any
DB indexes
- 22:24:28 <trustin>      It has been always slow hehe
- 22:24:43 <vrm>          I heard some Apache Innfra guys saying it's a Java software
:)
- 22:25:11 <doniv>        possible
- 22:25:39 <vrm>          was more a troll than a serious statement
- 22:25:42 <trustin>      OK guys we have 33 open issues so let's just open them randomly
and find out which issues got votes.
- 22:25:44 <proyal>       trustin: i think you have to add the column to the issue navigator,
i'm trying that now.. but.. it sslow :)
- 22:25:53 <doniv>        "Service Temporarily Unavailable"
- 22:26:14 <vrm>          was working 3 minutes before :/
- 22:26:19 <doniv>        mina effect
- 22:26:33 <trustin>      Actually the issue with most vote got one vote AFAIK hehe
- 22:26:52 <proyal>       heh, since no-one is using it yet :)
- 22:27:06 <trustin>      :)
- 22:27:10 <trustin>      JIRA is down :(
- 22:27:35 <luke_red5>    :) lol, i was just about to add that resuse address possible
bug. 
- 22:27:50 <trustin>      While JIRA is up again, let's talk about important issues
that everyone here want to address.
- 22:27:55 <doniv>        luke_red5: isnt reuse address an issue with linux jre
- 22:28:02 <luke_red5>    yes
- 22:28:15 <luke_red5>    so its not mina just jre on linux?
- 22:28:31 <luke_red5>    only happens on linux and osx 
- 22:28:32 <doniv>        thats what i've seen
- 22:28:40 <luke_red5>    and only some times
- 22:28:42 <trustin>      I think people here are the biggest users who got much more
concern than others.  So I'd like to respect all of your opinions.
- 22:29:16 <trustin>      luke_red5, it is because the file descriptor is disposed when
GC occurrs.
- 22:29:32 <trustin>      luke_red5, it's a known issue.  there was a JIRA issue hehe
- 22:29:42 <luke_red5>    ok, cool
- 22:29:59 <doniv>        DIRMINA-206 is really critical
- 22:30:06 <doniv>        the outofmemory issue
- 22:30:34 <trustin>      I didn't look into it yet.
- 22:31:02 <luke_red5>    thanks trustin, just wanted to make sure it was reported
- 22:31:06 <trustin>      proyal, do you have any idea on DIRMINA-206?
- 22:31:48 <trustin>      doniv, could you explain the issue a little bit/
- 22:31:49 <trustin>      ?
- 22:32:04 <vrm>          could help,I can't open the jira page :(
- 22:32:20 <doniv>        2 mins
- 22:32:21 <trustin>      Me either
- 22:32:31 <doniv>        will paste the link
- 22:32:32 <luke_red5>    i found it in an email
- 22:32:33 <doniv>        to the archive
- 22:32:36 <luke_red5>    MINA doesn't throttle read or write traffic even if a client
or a server suffers from bottleneck.  It will eventually cause the event buffer become full
can throw an OutOfMemoryError.  We need to prevent it.
- 22:32:44 <doniv>        cool
- 22:32:45 <doniv>        thats it
- 22:33:18 <trustin>      Ah I got it. It was scheduled for 0.9.5 right?
- 22:34:06 <lneves>       yes.. it was... I was looking at it a while ago
- 22:34:16 <trustin>      Anyone got a list of all open issues?  If so, it would be
nice he can paste the list to rafb.net/paste
- 22:34:44 <doniv>        i've got the first page
- 22:34:48 <trustin>      I closed all tabs unfortunately.
- 22:34:58 <trustin>      Great.  Let's start from the first page :)
- 22:36:15 <trustin>      doniv, could you paste the list?
- 22:36:35 <doniv>        http://rafb.net/paste/results/sqKtsB18.html
- 22:37:19 <trustin>      Great.  Everyone browsing the URL?
- 22:37:28 <proyal>       yup
- 22:37:30 <luke_red5>    yup
- 22:37:36 <trustin>      GReat.
- 22:38:25 <proyal>       -144 and -196 might be duplicates? or at least candidates
for merging
- 22:38:25 <trustin>      Any issues you want to address first?
- 22:38:56 <proyal>       sure! we were discussing -184 briefly before you arrived..
- 22:38:57 <trustin>      Hmm makes sense.
- 22:39:02 <proyal>       that's the pluggable ThreadPoolFilter
- 22:39:06 <trustin>      Ah I see.
- 22:39:16 <proyal>       was thinking that perhaps we can create a new java5 module
to put that in?
- 22:39:39 <proyal>       thus, java5 users could then use the alternate TPF that could
use the ThreadPoolExec
- 22:39:43 <lneves>       jira seems to be up again... albeit slow
- 22:39:54 <trustin>      What do you think about choosing issues required for 1.0?
- 22:40:08 <proyal>       a good approach :)
- 22:40:09 <trustin>      so we can move schedule some issues to 1.1 branch?
- 22:40:21 <doniv>        http://rafb.net/paste/results/wegTlG47.html
- 22:40:25 <doniv>        page 2
- 22:40:27 <doniv>        jira is back up
- 22:40:35 <trustin>      Great.  Now we got the whole pages.
- 22:41:07 <trustin>      Everyone, could you list the issues which has to be included
in 1.0?
- 22:41:21 <doniv>        i wanted to add another issue
- 22:41:26 <doniv>        regarding bytebuffer allocation
- 22:41:28 <doniv>        reallocation actually
- 22:41:49 <trustin>      Isn't there 'expand' method already?
- 22:41:51 <doniv>        a 2x increase that is being done right now can be optimized
- 22:42:13 <proyal>       optimized in what manner?
- 22:42:19 <doniv>        instead of 2x
- 22:42:25 <doniv>        reallocate on need basis
- 22:42:33 <doniv>        my server also needs to handle file transfers
- 22:42:47 <proyal>       i think that's partially an artifact of the pooling
- 22:42:51 <doniv>        so memory allocation will become an issue
- 22:43:01 <proyal>       since it makes the pool easy since it just stores sets by
^2
- 22:43:08 <proyal>       if not pooling, no need to do the increases like that
- 22:43:38 <trustin>      right.
- 22:44:14 <doniv>        my requirement is only that memory should be allocated carefully
- 22:44:21 <doniv>        only on a need basis
- 22:44:27 <doniv>        if that is getting addressed, no issues
- 22:44:40 <proyal>       doniv: have you had problems with the existing allocation
in your tests?
- 22:44:53 <doniv>        havent gotten around to a load test yet
- 22:44:58 <trustin>      hmm..
- 22:44:59 <doniv>        should happen in the next few weeks
- 22:45:01 <proyal>       ah, k
- 22:45:17 <proyal>       since the pool now expires unused buffers, you might be okay
with what we have now
- 22:45:26 <trustin>      I feel so many issues should be fixed before 1.0 is released.
- 22:45:43 <proyal>       i think we should at least fix all bugs
- 22:45:55 <trustin>      Yes.  All bugs should be fixed in the next released.
- 22:45:59 <proyal>       some things, like filters, as long as the API is stable, they
can be released somewhat independently
- 22:46:01 <trustin>      released -> release.
- 22:46:08 <trustin>      Exactly.
- 22:46:43 <trustin>      Then what we have to resolve is what affects the API.
- 22:46:56 <proyal>       correct
- 22:47:06 <proyal>       a decision on ByteBuffer allocation will be good
- 22:47:10 <trustin>      k then let's list them... let me see
- 22:47:19 <trustin>      yeah great idea.
- 22:47:52 <doniv>        i'd also like to see more support for stream based i/o
- 22:48:02 <doniv>        thats currently kinda on the back burner
- 22:48:06 <luke_red5>    i have a q about byte buffer methods, but i will wait till
you guys have chatted over the open issues
- 22:48:23 <proyal>       ByteBuffer allocation is https://issues.apache.org:443/jira/browse/DIRMINA-210
everyone
- 22:49:11 <trustin>      DIRMINA-210 	Investigate removal of static methods in ByteBuffer
- 22:49:11 <trustin>      DIRMINA-206 	Prevent OutOfMemoryError when a server or a client
is overloaded
- 22:49:11 <trustin>      DIRMINA-205 	Create a utility to list ports and sockets being
used
- 22:49:11 <trustin>      DIRMINA-194 	Extract Reactor from Acceptor and Connector like
ACE in C++.
- 22:49:11 <trustin>      DIRMINA-184 	Allow ThreadPool used by ThreadPoolFilter to
be pluggable
- 22:49:11 <trustin>      DIRMINA-166 	Common internal messages that can be shared by
filters and transport implementations
- 22:49:11 <trustin>      DIRMINA-128  	 IoSession.shutdown(TrafficMask)
- 22:49:11 <trustin>      DIRMINA-68 	Automatic reconnect configuration for client channels.
- 22:49:39 <trustin>      These are the issues which will affect the API.  Some will
just add a few methods.
- 22:49:43 <proyal>       trustin, i like your idea about ByteBuffer.allocate, but i'd
want to keep the existing way of doing it via statics, rather than having it check to the
current thread.. ByteBuffer.allocate() would basically be a deprecated way.. IoSession.getAllocator().allocate()
would be preferred (when the user wants pooling)
- 22:50:37 <luke_red5>    205 isnt critical for us, but I guess it would effect the
spring api part
- 22:50:47 <trustin>      proyal, yes it's simpler way for us.  But IoSession.getAllocator().allocate()
takes more time to type hmm..
- 22:51:04 <trustin>      luke_red5, yes it's an API addition.
- 22:51:29 <luke_red5>    i guess integration isnt that core to mina, at least i dont
mind updating as we go
- 22:51:59 <vrm>          I don't care much about memory pooling optimisations, I would
hate to need to type IoSession.getAllocator().allocate() :)
- 22:52:01 <proyal>       trustin: right, that's the downside, more typing..
- 22:52:03 <trustin>      You're right.  It's a good-to-have feature.
- 22:52:30 <proyal>       i'm also 100% fine with leaving ByteBuffer allocation as-is
- 22:52:30 <trustin>      Perhaps we could provide a method in IoHandlerAdapter.
- 22:52:47 <trustin>      IoHandlerAdapter.allocate(session, size)?
- 22:53:00 <trustin>      as a static method?
- 22:53:23 <proyal>       trustin, heck, ByteBuffer.allocate( session, size) then..
- 22:53:34 <trustin>      ahhh you're right, proyal 
- 22:53:44 <trustin>      that would be very nice.
- 22:53:47 <proyal>       dunno if that's mixing the two interfaces together unnecessarily
then..
- 22:53:55 <luke_red5>    will it still be possible to allocate without a session
- 22:53:57 <proyal>       but the general idea is fine
- 22:54:01 <doniv>        that will create an unnecessary dep
- 22:54:09 <trustin>      hmm right.
- 22:54:17 <trustin>      though they are in the same package.
- 22:54:40 <doniv>        starts getting ugly when u test bytebuffer
- 22:54:57 <trustin>      It might be OK because these static methods are just a utility.
hmm...
- 22:55:14 <trustin>      yeah we'll need a mock object.
- 22:56:21 <proyal>       luke_red5: yes, would be able to allocate w/o a session, just
that there would be an Allocator per-IoHandler or such
- 22:56:26 <luke_red5>    can someone tell me why the session is needed to be passed
in. Will you set the allocator for the session?
- 22:56:50 <luke_red5>    ok :) you answered before i asked :)
- 22:57:03 <proyal>       so you could always instantiate your own allocator and allocate
from that
- 22:57:46 <trustin>      Anyway, we can talk about this issue more deeply.  I think
what we have to focus now is to decide what issue to consider sooner. :)
- 22:57:55 <proyal>       yes, lets continue on
- 22:57:59 <proyal>       i'll add some notes to the jira for that
- 22:58:11 <trustin>      Great.
- 22:58:22 <vrm>          for sure ByteBuffer will have a great impact on API so it's
before 1.0 ?
- 22:58:27 <proyal>       vrm: correct
- 22:58:56 <trustin>      DIRMINA-210, DIRMINA-206, DIRMINA-194, DIRMINA-166 can affect
the API seriously.  WDYT?
- 22:59:23 <trustin>      DIRMINA-210 	Investigate removal of static methods in ByteBuffer
- 22:59:23 <trustin>      DIRMINA-206 	Prevent OutOfMemoryError when a server or a client
is overloaded
- 22:59:23 <trustin>      DIRMINA-194 	Extract Reactor from Acceptor and Connector like
ACE in C++.
- 22:59:23 <trustin>      DIRMINA-166 	Common internal messages that can be shared by
filters and transport implementations
- 22:59:56 <trustin>      DIRMINA-206 will also impact the API because we have to decide
where to throttle the I/O.
- 23:00:08 <vrm>          206 can't a be Filter limiting I/O  if the memory usage is
growing too much ?
- 23:00:18 <proyal>       trustin, that list lokos good
- 23:00:33 <trustin>      :)
- 23:00:53 <trustin>      The OOM error is thrown when ThreadPoolFilter is engaged.
- 23:01:26              * niklas (n=niklas@d183.gvc.gu.se) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 23:01:31 <trustin>      So I thought ThreadPoolFilter is the right place to throttle
the traffic, but it might not work as we expected.
- 23:01:37 <proyal>       the TPF should limit the size of the per-session queue, and
disable reads when it reaches a limit
- 23:01:37 <trustin>      Hi niklas!
- 23:01:47 <niklas>       Hi! Sorry I'm late! 
- 23:01:52 <trustin>      no problem:)
- 23:02:02 <vrm>          I don't see 20- impact on API
- 23:02:03 <vrm>          206
- 23:02:12 <trustin>      niklas, we chose a list of issues which will impact the API:
DIRMINA-210 	Investigate removal of static methods in ByteBuffer
- 23:02:12 <trustin>      DIRMINA-206 	Prevent OutOfMemoryError when a server or a client
is overloaded
- 23:02:12 <trustin>      DIRMINA-194 	Extract Reactor from Acceptor and Connector like
ACE in C++.
- 23:02:12 <trustin>      DIRMINA-166 	Common internal messages that can be shared by
filters and transport implementations
- 23:02:15 <proyal>       vrm: right, it could be internal to the TPF
- 23:02:45 <trustin>      vrm, proyal: right it will just impact the internal implementation.
- 23:02:59 <trustin>      As you know a user can change the traffic mask.
- 23:03:12 <trustin>      and now do we need another mask for overload prevention?
- 23:03:20 <vrm>          at least it's some limit parameters to give to the TPF 
- 23:03:41 <proyal>       if the user re-enabled reads after the TPF disabled them for
overload prevention, i think they get what they deserve :)
- 23:03:56 <proyal>       ie, at some point, we need to just say "if you do this, you'll
hurt yourself"
- 23:04:10 <trustin>      hmm.. what if a user uses traffic throttling filter with TPF?
- 23:04:32 <proyal>       the throttling filter should only re-enable reads when its
okay
- 23:04:44 <proyal>       it could keep track of whether or not it was the one to toggle
the state
- 23:04:53 <trustin>      how?
- 23:05:27 <proyal>       the filter could keep track of when it toggled reads off and
only turn them back on if it turned them off
- 23:05:54 <luke_red5>    guys, dinners arrived ;) will check back later see if you
are still chatting
- 23:06:07 <proyal>       i think the problem case is if the TPF had disabled, and the
throttle wanted to re-enable?
- 23:06:12 <proyal>       luke_red5: enjoy!
- 23:06:17 <trustin>      For example both TTF and TPF could disable read at the same
time and then TTF might enable the read in spite that TPF doesn't want it.
- 23:06:38 <proyal>       they can't disable at the same time can they? since they'd
be executed in serial not parallel?
- 23:06:46 <trustin>      Hmm.. right.
- 23:06:57 <proyal>       if TTF enabled when TPF didn't want it, TPF would have to
accept it, and could then re-disable
- 23:07:21 <proyal>       overall, i think its solvable w/o a special "overload" traffic
mask
- 23:07:28 <trustin>      We need some synchronization in IoSession.setTrafficMask then,
right?
- 23:07:47 <proyal>       yes, it should be synchronized i think.
- 23:08:08 <proyal>       since you're right, you could get parallel execution of filters
that are after the TPF
- 23:08:30 <trustin>      proyal, you're right. hmm this won't impact the API.
- 23:08:56 <trustin>      But this issue has to be resolved soon because it affects
the stability of the server.
- 23:09:53 <doniv>        are u guys saying that its possible to do overload prevention
with a filter?
- 23:10:29 <doniv>        solution for dirmina-206?
- 23:11:08 <trustin>      yes.
- 23:11:15 <proyal>       doniv: we'd give the ThreadPoolFilter some smarts to prevent
it
- 23:11:44 <trustin>      But the problem becomes complex if a user adds more than one
TPFs.
- 23:12:00 <proyal>       only the first one needs to do it
- 23:12:11 <doniv>        maybe i'm missing something... but hasnt the memory already
been allocated if its reaching a filter?
- 23:12:24 <proyal>       well, no, since the backup can occur at a later filter..
- 23:12:51 <doniv>        so mina is not read() ing from the network into a buffer?
- 23:12:56 <trustin>      no.
- 23:13:28 <trustin>      It reads but filter will disable read operation from the next
time.
- 23:14:02 <trustin>      So we can limit the memory consumption at certain level.
- 23:14:32 <doniv>        so ur proposing a new filter that will toggle read() s?
- 23:14:45 <trustin>      No.  ThreadPoolFilter will do the job.
- 23:15:02 <proyal>       k guys, i've gotta run for a bit, will be back in an hour
or so, been very good so far, perhaps we should do this on a regular basis?
- 23:15:13 <trustin>      proyal, of course! :)
- 23:15:18 <doniv>        bye
- 23:15:26 <vrm>          bye
- 23:15:27 <trustin>      If there's no ThreadPoolFilter in the chain, OutOfMemoryError
won't be thrown.
- 23:15:29 <niklas>       bye
- 23:15:37 <trustin>      proyal, I'll schedule the next meeting soon.
- 23:15:42 <proyal>       going to leave my client connected, so i can read the buffer
when i get back
- 23:15:48 <proyal>       trustin: sounds good
- 23:15:49 <proyal>       bye all, thanks for coming!
- 23:15:54 <trustin>      thanks, too! :)
- 23:15:56 <trustin>      bye
- 23:16:16 <niklas>       An idea: how about wrapping the TPF instead of adding this
logic to it? Or maybe you've been over that already?
- 23:16:36 <trustin>      hmm.. that's also a good idea.
- 23:16:41 <trustin>      wdyt doniv ?
- 23:16:46 <niklas>       You could measure the incoming messages versus the outgoing
and detect if it's become overlodaded that way.
- 23:16:59 <doniv>        i was thinking of a read buffer size per session configuration
- 23:17:25 <niklas>       If the ratio is too high this wrapping filter will suspend
reads for some sessions somehow.
- 23:17:34 <trustin>      I thought about that option before but it doesn't count for
the case when there's no TPF.
- 23:18:04 <trustin>      BTW if we can control the read totally, then we could just
create a sole filter which will be located just before TPF.
- 23:18:23 <trustin>      totally -> totally using a traffic mask
- 23:19:08 <elecharny_>   Hi guys !
- 23:19:12 elecharny elecharny_ 
- 23:19:20 <trustin>      elecharny_: hi!
- 23:19:26 <doniv>        hi
- 23:19:28 <vrm>          hi
- 23:19:30 <niklas>       hi!
- 23:19:38 <vrm>          BTW anybody is working on the performance monitoring filter
?
- 23:19:41 <elecharny_>   I don't know if the meeting is over ?
- 23:19:45 <trustin>      No, nobody yet
- 23:19:50 <trustin>      not over.
- 23:19:52 <trustin>      we're in the middle
- 23:19:57 <elecharny_>   ok.
- 23:20:17 <trustin>      We're talking about an issue: DIRMINA-206 	Prevent OutOfMemoryError
when a server or a client is overloaded
- 23:20:18 <elecharny_>   Don't want to disturb you a lot. I just have a suggestion
for the next version...
- 23:20:26 <trustin>      No worries.
- 23:20:48 <trustin>      Another problem we have to deal with regarding OOM error is
throttling writes.
- 23:21:02 <lneves>       FWIW i had the OOM problem and I just followed the sugestion
of Trustin to use suspendRead() and resumeRead() ... why can't this be pushed to user?
- 23:21:33 <doniv>        lneves: it would be an unnecessary burden, since tcp already
handles these semantics
- 23:21:44 <trustin>      lneves, we can, but eventually we want to provide it out of
the box to lessen the burden of the user.
- 23:22:12 <trustin>      Because MINA should be easy and simple yet powerful. :)
- 23:22:35 <trustin>      and we'll get less message about OOM hehe
- 23:23:25 <trustin>      For reads, we can use a filter to throttle the traffic, but
write is another matter.
- 23:24:19 <trustin>      Let's name the OOM preventer filter as BOP (Buffer Overrun
Protector).
- 23:24:28 <vrm>          when the write load is too heavy just drop writed buffers
;)
- 23:25:10 <trustin>      like returning a future which immediately returns false when
a user call 'isWritten'?
- 23:25:22 <trustin>      The chain will look like this: BOP -> TPF -> Handler
- 23:25:35 <trustin>      Here, BOP can protect reads but not writes.
- 23:25:36 <lneves>       ok... but at some point the user must to some work :-)  ...
I can't think of good heuristics to throttle reads/writes.
- 23:25:45 <niklas>       will writes really be an issue?
- 23:25:51 <doniv>        niklas: yea
- 23:26:03 <niklas>       i mean you get messageSent when your message has been written
- 23:26:03 <trustin>      If the client is too slow then we can get OOM there too.
- 23:26:07 <niklas>       then you send the next
- 23:26:19 <niklas>       or am i just naive? :)
- 23:26:34 <doniv>        yea, but the jvm wont give u the write readiness notification
- 23:26:43 <niklas>       precisely
- 23:26:53 <trustin>      niklas, it works.  but it won't be optimal in the aspect of
performance.
- 23:26:59 <doniv>        memory someplace will get filled up
- 23:27:00 <niklas>       you won't get messageSent until messages has been delivered
to the kernel buffer right?
- 23:27:10 <doniv>        right
- 23:27:13 <trustin>      niklas, on some platforms, yes.
- 23:28:03 <trustin>      BTW would it be OK to block IoSession.write() method when
buffer is full?
- 23:28:22 <trustin>      Otherwise, would fail-fast be the better way?
- 23:28:28 <doniv>        ideally i'd prefer a future
- 23:29:08 <trustin>      How can we use a future in this case?
- 23:29:50 <trustin>      I can't think of a future other than a future which returns
false immediately.
- 23:29:58 <trustin>      ... when buffer is full.
- 23:30:02 <doniv>        yea
- 23:30:28 <doniv>        it has the info of whether the write is pending, failed or
completed
- 23:30:38 <trustin>      sounds like a good idea.
- 23:30:53 <trustin>      Then how can a user get notified when a buffer has a room
for a new write?
- 23:31:10 <vrm>          when he retry ?
- 23:31:12 <doniv>        they can wrap the future
- 23:31:27 <trustin>      actually they could use messageSent event?
- 23:31:27 <doniv>        implement their own, that has the retry logic
- 23:31:47 <trustin>      if messageSent is fired, this means at least the memory is
freed up for the previous message.
- 23:32:40 <doniv>        but that would cause a control jump
- 23:32:47 <trustin>      hmm right.
- 23:32:50 <doniv>        basically, a change of flow
- 23:33:21 <doniv>        it would be good if we could offer a WriteRetryFuture
- 23:33:25 <doniv>        or something like that
- 23:33:44 <trustin>      hmm.. Sounds interesting
- 23:34:01 <trustin>      Then we have maintain a list of WriteRetryFutures for notification.
- 23:34:03              * 접속 종료됨 (접속이 초기화됨).
- **** 기록 종료 : Tue May 09 23:34:03 2006
- 
- **** 기록 시작 : Tue May 09 23:35:58 2006
- 
- 23:35:58              * 당신은 대화방 #directory-dev 에 참여합니다.
- 23:36:08 <trustin_>     Sorry I lost my connection
- 23:36:47 <trustin_>     My last messagE: Then we have maintain a list of WriteRetryFutures
for notification.
- 23:36:52 <trustin_>     have -> have to 
- 23:37:00 <doniv>        niklas: dunno... trustin, niklas was asking of the OOM will
occur in user code or MINA code
- 23:37:25 <trustin_>     user code probably.  But it can also happen in MINA code even
if MINA is not the main cause of the problem.
- 23:37:35 <niklas>       I mean it seems to me like the user will do the allocation
of bytebuffer when writing so he will cause the OOME right?
- 23:37:52 <niklas>       True
- 23:37:54 <trustin_>     yes, but it's just a matter of probability.
- 23:38:19 <trustin_>     anyway...
- 23:38:36 <trustin_>     i guess this issue can be solved without changing the API
a lot.  It will be just an addition.
- 23:38:56 <trustin_>     Let's talk about this issue later.  For now let's discuss
the next issue. :)
- 23:39:08 <doniv>        trustin_: good idea
- 23:39:10 <doniv>        :)
- 23:39:11 <trustin_>     :)
- 23:39:14 <doniv>        this needs deliberation
- 23:39:29 <trustin_>     What we needed was just a concensus that this issue won't
affect the API design at all.
- 23:39:42 <doniv>        are we sure on that?
- 23:40:14 <trustin_>     Not 100% sure.  I guess we need to try to implement it.
- 23:40:28 <trustin_>     At least it will add some properties to our configuration
classes.
- 23:40:30              * luke_red6 (n=luke@203.209.43.192) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 23:40:35 luke_red5 luke_red6 
- 23:40:39 <trustin_>     hi again luke_red6
- 23:40:48 <doniv>        hmm
- 23:41:06 <trustin_>     or to TPF.
- 23:41:27 <trustin_>     Next issue: DIRMINA-194 	Extract Reactor from Acceptor and
Connector like ACE in C++.
- 23:41:52 <trustin_>     I don't know whether we need to resolve this issue.
- 23:42:00 <trustin_>     What do you think?
- 23:43:16              * doniv has no idea abt this
- 23:43:18 <niklas>       I haven't looked at ACE
- 23:43:34 <doniv>        i'd seen some of it 5 yrs ago :)
- 23:43:45 <trustin_>     :)
- 23:43:48 <trustin_>     Me either.
- 23:44:24 <niklas>       on to the next issue then? ;)
- 23:44:36 <trustin_>     OK.  Then let's skip this issue untli we have an extra transport
implementation such as AIO.
- 23:45:00 <trustin_>     The last issue: DIRMINA-166 	Common internal messages that
can be shared by filters and transport implementations
- 23:45:37 <trustin_>     The idea of this issue is basically to provide some common
set of message objects.
- 23:45:54 <trustin_>     For example, we can get notified when a TLS session is secured
or unsecured.
- 23:46:09 <trustin_>     We're using messages which are specific to SSLFilter now.
- 23:46:25 <trustin_>     What if we use a crypto filter which is not an SSLfilter?
- 23:47:19 <trustin_>     More explanation is here: http://issues.apache.org/jira/browse/DIRMINA-166
- 23:47:43 <trustin_>     I'm not sure if this is a good idea though.
- 23:47:48 <doniv>        secured / unsecured in the sense?
- 23:48:36 <doniv>        wont sslfilter throw an exception if the ssl handshake failed?
- 23:48:42 <doniv>        afaik, it does
- 23:48:44 <trustin_>     It might be better to create a subclass of IoHandler which
has an extra event handler methods.
- 23:49:10 <trustin_>     Secured is emitted when a SSL handshake is finished successfully.
- 23:49:39 <trustin_>     Unsecured is emitted when TLS session is closed and the session
is going to transfer data in clear text.
- 23:50:17 <doniv>        but wont that mean removal of the filter?
- 23:50:20 <trustin_>     Subclassing has its own problem that it bypasses the filter
chain.
- 23:50:23 <niklas>       Won't this break transperancy? I mean I don't want my IoHandler
to know whether an SSLFilter is used or not.
- 23:50:27 <trustin_>     It can, but it sometimes not.
- 23:50:40 <doniv>        any specific case u can recollect?
- 23:50:41              * luke_red5 has quit (Read error: 110 (Connection timed out))
- 23:50:48 <trustin_>     StartTLS is the case.
- 23:51:05 <doniv>        alright.. taking that example...
- 23:51:17 <doniv>        if i want to enable tls over an existing connection
- 23:51:19 <trustin_>     You can start TLS session in the middle of communication and
then finish the session before the connection is closed.  and then it can start again.
- 23:51:25 <doniv>        i apply the filter
- 23:51:38 <doniv>        and set disable_encryption_once
- 23:51:42 <doniv>        and then write() data
- 23:51:51 <doniv>        signifying starttls enabled
- 23:52:04 <trustin_>     yep.
- 23:52:12 <doniv>        when the user wants to remove tls, he calls remove on the
filter
- 23:52:22 <doniv>        which causes the tls connection to be torn down as well
- 23:52:41 <trustin_>     hmm yes.
- 23:52:46 <doniv>        anything i'm missing here?
- 23:52:56 <trustin_>     Client can close the TLS session.
- 23:53:14 <doniv>        yep
- 23:53:16 <doniv>        true...
- 23:53:20 <doniv>        filter cant remove itself
- 23:53:35 <trustin_>     hmm it might be able to .. let me think
- 23:54:00              * luke_red5 (n=luke@203.209.43.155) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 23:54:20 <doniv>        if it can, then notifications can be added for filterAdded()
filterRemoved()
- 23:54:29 <doniv>        dont remember if they are already there
- 23:54:58 <trustin_>     hmm..
- 23:55:26 <trustin_>     I used these messages in LDAP StartTLS support.
- 23:55:35 <trustin_>     Darn, I forgot the SVN URL.
- 23:56:14              * trustin has quit (Read error: 110 (Connection timed out))
- 23:56:32 <trustin_>     found it hehe
- 23:58:10 <trustin_>     When we implement StartTLS and StopTLS(?) in a complex protocol.
- 23:58:55 <trustin_>     we have to exchange some messages which conforms to that protocol.
- 23:59:06 <doniv>        yes
- 23:59:48 <trustin_>     So the Secured and Unsecured event can be part of the protocol.
 For example, you might want to degrade the authentication level when a session is unsecured.
- 00:00:25 <trustin_>     So a user need to get notified when a session is secured or
unsecured.
- 00:00:54 <doniv>        thats not necessarily a protocol notification
- 00:01:11 <doniv>        i mean, the protocol wont have a command to say that the connection
is now secure
- 00:01:31 <doniv>        atleast the one i work with doesnt have
- 00:01:33 <trustin_>     right.  It's just an internal message in TLS level.
- 00:02:01 <trustin_>     We just catch the state of the session and perform an appropriate
action.
- 00:03:02 <doniv>        a proposition..
- 00:03:02 <trustin_>     Oops it's midnight here.
- 00:03:16 <doniv>        currently filters have methods that can be invoked
- 00:03:21 <doniv>        by external classes
- 00:03:24 <trustin_>     yes.
- 00:03:39 <doniv>        so can we have a method that performs a callback
- 00:03:45 <doniv>        for such notifications, if needed
- 00:04:03 <trustin_>     We could even define a callback interface.
- 00:04:05 <doniv>        like for eg, sslfilter can have connectionStateNotification()
- 00:04:29 <trustin_>     sounds good.
- 00:04:29 <doniv>        so in this case, IoHandlers dont need to be changed
- 00:04:42 <trustin_>     Hmm you're right.
- 00:05:00 <trustin_>     nice idea indeed.
- 00:05:08 <niklas>       great!
- 00:05:08 <trustin_>     so we don't need to resolve this issue? heh
- 00:05:09 <doniv>        thanks :)
- 00:05:24 <doniv>        dont make me blush now :)
- 00:05:27 <trustin_>     :)
- 00:05:35 <trustin_>     Another issue is here:
- 00:05:43 <trustin_>     http://issues.apache.org/jira/browse/DIRMINA-208
- 00:05:57              * luke_red6 has quit (Read error: 110 (Connection timed out))
- 00:06:21 <doniv>        Forever "waiting for issues.apache.org"
- 00:06:30 <niklas>       me too :(
- 00:06:47 <trustin_>     Allow datagrams to be directed to a specific destination on
a per-datagram instead of a per-session basis.
- 00:07:01 <trustin_>     It's an issue for broadcast.
- 00:07:19 <doniv>        hmm
- 00:07:26 <trustin_>     Broadcast packet basically doesn't have a source address.
- 00:07:34 <trustin_>     so it cannot write back the response.
- 00:07:37 <doniv>        dunno abt the datagram support in mina
- 00:07:56 <trustin_>     :)
- 00:08:13 <niklas>       broadcast is more or less a black hole for me. sorry.
- 00:08:27 <trustin_>     We cannot know the address of the remote peer when we receive
a broadcast message.
- 00:08:43 <vrm>          why not a special session for handling broadcasts ?
- 00:09:03 <trustin_>     Actually there is a reply-to field in DHCP message.
- 00:09:16 <trustin_>     Hmm.  that makes sense.
- 00:09:27 <trustin_>     so the reply should go to that reply-to address.
- 00:09:28 <doniv>        i agree
- 00:09:31              * luke_red6 (n=luke@203.209.43.172) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 00:10:06 <trustin_>     We have a DatagramSession.  We could subclass this interface
and define a BroadcastSession.
- 00:10:32 <trustin_>     BroadcastSession.write() has an extra SocketAddress parameter.
- 00:10:39 <trustin_>     What do you think?
- 00:10:59 <trustin_>     and setRemoteAddress perhaps.
- 00:11:04 <doniv>        from whatever i understand, it should be good
- 00:11:16 <doniv>        am not sure abt the subclassing though
- 00:11:30 <doniv>        but might be wrong since i havent seen datagramsession
- 00:11:45 <trustin_>     OK.  I'll a comment about this issue after I summarize this
meeting.
- 00:12:02 <trustin_>     No problem.  This is just a brainstorming :)
- 00:12:07 <trustin_>     niklas, what do you think?
- 00:12:36 <trustin_>     it seems like he's away hehe
- 00:12:43 <trustin_>     OK.. it's late here.
- 00:12:49 <niklas>       Subclassing sounds like a nice idea
- 00:12:54 <doniv>        maybe it would be better to have BroadcastSession and DatagramSession
at the same hierarchical level
- 00:12:59 <niklas>       Sorry, got distracted elsewhere
- 00:13:04 <doniv>        with a common super class
- 00:13:24 <trustin_>     We had two hours conversation.  It was great indeed.  It would
be nice if we held this kind of meeting periodically.
- 00:13:47 <niklas>       Yes, I think it would be great
- 00:13:52 <trustin_>     doniv, but broadcast is really just a subset of datagram.
- 00:13:54 <doniv>        turned out to be more of a brainstorming meeting rather than
a roadmap meeting, eh
- 00:14:00 <trustin_>     hehe yeah
- 00:14:23 <trustin_>     At least we chose some high priority issues.
- 00:14:36 <doniv>        yea, that was good
- 00:14:42 <niklas>       Will a log of this meeting be put somewhere?
- 00:14:51 <niklas>       Could be good for the future
- 00:14:54 <trustin_>     yes. I'll e-mail it.
- 00:15:09 <trustin_>     Next release will have to resolve most issues we addressed
here and other bugs.
- 00:15:20 <niklas>       Ok, great. Maybe we could put the logs on the wiki?
- 00:15:26 <trustin_>     Sure
- 00:15:54 <trustin_>     BTW It would be nice if we can shorten the release cycle.
- 00:16:06 <doniv>        i agree too..
- 00:16:17 <trustin_>     two to three weeks will be optimal.
- 00:16:22 <doniv>        but possibly, theres another way to look at it
- 00:16:30 <trustin_>     yes.
- 00:16:34 <doniv>        maybe the core features have been completed
- 00:16:41 <doniv>        thats why the slowdown?
- 00:16:52 <trustin_>     perhaps.  we're all satisfied haha
- 00:17:09 <doniv>        exactly :)
- 00:17:14 <niklas>       well actually mina is working great!
- 00:17:20 <trustin_>     :)
- 00:17:26 <doniv>        according to that logic, 1.0 should be out yesterday :)
- 00:17:29 <niklas>       :)
- 00:17:46 <trustin_>     :)
- 00:17:59 <trustin_>     At least OOM and ByteBuffer static method issues have to be
resolved before 1.0 IMHO.
- 00:18:01 <doniv>        on a more serious note, 1.0 readiness can be judged by the
number of current users as well
- 00:18:14 <doniv>        definitely
- 00:18:25 <trustin_>     Ah and Broadcast hehe
- 00:18:38 <doniv>        if there are sufficient number of users running it successfully
in production systems, we could be 1.0 ready
- 00:18:41 <trustin_>     and Datagram session management scheme I fotgot to mention.
- 00:18:45 <doniv>        after these bug fixes, of course
- 00:18:54 <trustin_>     yeah correct.
- 00:19:04 <doniv>        "sufficient" needs to be defined
- 00:19:14 <trustin_>     I feel that we need to take care of datagram side, too hehe
- 00:19:42 <doniv>        but then there would be todo's piling up all the time
- 00:19:59 <doniv>        trick is to draw the line somewhere
- 00:20:12 <doniv>        its a huge psychological advantage if we have a 1.0
- 00:20:32 <trustin_>     You're right.  My wish for 1.0 is to make the APi design fit
to both datagram and socket nicely so we don't need to revamp the API later.
- 00:20:38 <vrm>          yes but you need to freeze the API
- 00:20:46 <doniv>        agreed
- 00:20:48 <vrm>          heh
- 00:20:55 <trustin_>     yeah releasing 1.0 will attract much more people.
- 00:21:08 <trustin_>     and they will start to submit more patches :D
- 00:21:15              * luke_red5 has quit (Read error: 110 (Connection timed out))
- 00:21:28 <doniv>        true
- 00:22:00 <elecharny_>   trustin : almost done with the session ?
- 00:22:02 <trustin_>     The changes from 0.8 to 0.9 was huge, but I'd like to minimize
the effect when we move from 1.0 to 1.1.
- 00:22:04 elecharny elecharny_ 
- 00:22:07 <trustin_>     elecharny_: yes.
- 00:22:13 <elecharny_>   ok. 
- 00:22:32 <doniv>        right...
- 00:22:39 <elecharny_>   We had a convo with alex lately, about the dependence 
- 00:22:45 <elecharny_>   between MINA and shared-asn1
- 00:22:46 <trustin_>     OK.. it seems like we discussed most important issues and
basic idea on what to include in 1.0.
- 00:22:49 elecharny elecharny_ 
- 00:22:53 <trustin_>     elecharny_: Oh I see.
- 00:23:01 <elecharny_>   is it possible to ease this constraint ?
- 00:23:14 <trustin_>     ease?
- 00:23:33 <trustin_>     My laptop is running out of battery.
- 00:23:39 elecharny elecharny_ 
- 00:23:40 <elecharny_>   relax, get rid of
- 00:23:54 <trustin_>     hmm.. so MINA doesn't depend on shared-asn1?
- 00:23:55 <vrm>          BTW can my Session MBean can be integrated ?
- 00:24:25 <elecharny_>   what about moving this statefull-decoder stuff out of shared-asn1
?
- 00:24:36 <trustin_>     vrm: JMX integration is a new feature related to integration.
 We discussed about the issues that can affect the core API.
- 00:24:38 <elecharny_>   yep.
- 00:24:47 elecharny elecharny_ 
- 00:24:48              * lneves has quit (Connection reset by peer)
- 00:25:03              * luke_red5 (n=luke@203.209.43.204) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 00:25:05 <elecharny_>   because if I change something in shared-asn1, then that means
a new version of MINA
- 00:25:06 <vrm>          JMX need some API change
- 00:25:08              * lneves (n=cbd00ab@194.65.5.235) 님께서 대화방 #directory-dev에
참여하셨습니다.
- 00:25:15 elecharny elecharny_ 
- 00:25:17 <vrm>          like for displaying ByteBuffer poolsize
- 00:25:36 <vrm>          but the Session MBean need no API changes
- 00:25:40 <trustin_>     vrm: it's basically an addition of the API, not the impacting
change.
- 00:26:13 <trustin_>     elecharny_: ah I see. we could have some middle area between
shared-asn1 and mina.
- 00:26:14 <vrm>          so post 1.0?
- 00:26:22 <elecharny_>   yeah.
- 00:26:39 <trustin_>     vrm: not sure.  it depends on how fast we can add features.
:D
- 00:26:40 <elecharny_>   It could go to commons-coec, but Alex syas that may take months
;)
- 00:26:50 elecharny elecharny_ 
- 00:27:09 <trustin_>     elecharny_: Can we talk about this on the maliling list. 
my battery is almost 0. :(
- 00:27:29 <trustin_>     vrm: My hope is to resolve all issues we got.
- 00:28:04 <elecharny_>   ok
- 00:28:23 <doniv>        alrighty then, gnite trustin
- 00:28:31 <vrm>          g'night
- 00:28:35 <trustin_>     vrm: Please keep posting your patches.  I'll make sure it
is reviewed and merged.
- 00:28:37 <doniv>        gnite everyone
- 00:28:41 <trustin_>     good night!
- 00:28:49 <doniv>        or good day :)
- 00:28:51 <trustin_>     :D
- 00:28:54 <vrm>          ok :)
- 00:28:54 <trustin_>     TTYL
- }}}
- 

Mime
View raw message