felix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre De Rop <pierre.de...@gmail.com>
Subject Re: Bundle cache changes
Date Sat, 02 Oct 2010 16:09:51 GMT
Hi Richard,

Ok, when I clean the disk buffer cache (using echo 1 >
/proc/sys/vm/drop_caches trick),
it seems that when felix starts with a non-empty cache, it takes some
substantial delay
to load its existing cache before firing a "Framework Started" event.

(Our app server is installed by a specific bundle installer, which waits for
the "Framework Started" event,
before installing other bundles).

I never noticed this delay because, generally, when I start my fwk in dev
environment, the bundles are already
available from linux buffer cache, and felix starts quickly.
But if I do the "echo > ..." trick in order to clear disk cache, then the
delay is important
I mean: the framework takes time to load its existing cache.

To be more specific, I describe exactly what I tested:

- I install/start only one bundle (our bundle installer)
- When started, the bundle installer listens to the "Framework Started"
event,
- So, when the start event is caught, the bundle installer starts
installing/starting
   other bundles (190), which might be already present in the cache.

Here are the logs of a cold start (disk buffers cleared, and felix cache
empty):

   2010-10-02 17:18:14,219 - Starting Felix OSGi Framework ...
   2010-10-02 17:18:15,536 - BundleInstaller starting
   2010-10-02 17:18:15,538 - Framework started: Installing 190 bundles ...
   2010-10-02 17:18:20,562 - Done.

And here are the logs of a cold start (disk buffers cleared, but existing
felix cache with 190 bundles in it):

   2010-10-02 17:18:58,167 - Starting Felix OSGi Framework ...
   2010-10-02 17:19:07,705 - BundleInstaller starting
   2010-10-02 17:19:07,707 - Framework started: Installing bundles ...
   2010-10-02 17:19:07,709 - Done.

-> Here, we see that when the cache is non empty, felix takes 9.6 seconds
before starting the bundle installer:
   2010-10-02 17:18:58,167 - Starting Felix OSGi Framework ...
   2010-10-02 17:19:07,705 - BundleInstaller starting (at this point, the
"Framework started" event is caught)


What do you think ?

/pierre

On Fri, Oct 1, 2010 at 11:30 PM, <heavy@ungoverned.org> wrote:

> On Fri, 1 Oct 2010 23:25:35 +0200, Pierre De Rop <pierre.derop@gmail.com>
> wrote:
> > On Fri, Oct 1, 2010 at 10:57 PM, Richard S. Hall
> > <heavy@ungoverned.org>wrote:
> >
> >>  One thing, was your performance comparison based on using the new
> cache
> >>  or
> >> old?
> >>
> >> I assume you are using a newly created cache for the framework snapshot
> >> and
> >> [obviously] an old cache for the 3.0.2, correct?
> >>
> > yes:
> > old cache with 3.0.2, and new cache with trunk.
> > but I also checked that new fwk/trunk is properly working when being
> > started
> > with the old cache (from 3.0.2)
> >
> >>
> >> You should also be testing an already created cache (i.e., a framework
> >> restart), not an empty cache.
> >>
> > In the previous test, I compared with already created cache, not with
> empty
> > caches.
> > With empty caches, here are the results:
> >
> >
> >    - old fwk 3.0.2 / fresh empty cache -> 15.8 sec. (this is surprising:
> I
> >    would expect to get a longer delay, since the cache is empty when I
> >    start
> >    the fwk)
> >
> > 2010-10-01 23:10:17,403 Starting Felix 3.0.2
> > 2010-10-01 23:10:23,885 Starting cluster node "test"
> > 2010-10-01 23:10:33,208 Cluster node "test" ready
> >
> >
> >    - new fwk / fresh empty cache ->15.2 sec
> >
> > 2010-10-01 23:15:46,2 Starting Felix 3.1.0.SNAPSHOT
> > 2010-10-01 23:15:52,417 Starting cluster node "test"
> > 2010-10-01 23:16:01,225 Cluster node "test" ready
> >
> > hmmm, I don't understand why the fwk startup time is better when the
> cache
> > is empty (15.2 sec) than when the cache is non-empty (21.2 sec from
> > previous
> > mail)
> > Ok, I will do more tests ... may be I did something wrong ? ... to be
> > continued ...
>
> Hmm. Yeah, that seems a little odd. In that case, you should just delete
> your cache each time to get better startup performance! ;-)
>
> Definitely let me know what you find out. I too would expect framework
> restarts to be faster since we don't have to copy the JAR files first.
>
> -> richard
>
> >
> > /pierre
> >
> >
> >
> >
> >>
> >> Is that what you are doing? Just want to make sure I understand what is
> >> being measured. :-)
> >>
> >>
> >> -> richard
> >>
> >> On 10/1/10 16:43, Pierre De Rop wrote:
> >>
> >>> Hi Richard,
> >>>
> >>> I tried our app server with the trunk (old cache with the new
> fwk/trunk)
> >>> ->
> >>> everything works ok
> >>> (I install 195 bundles, and start 41 bundles)
> >>>
> >>> With new cache/fwk trunk ->  it's also ok.
> >>>
> >>> I also compared the performance of the startup time between felix
> 2.0.2
> >>> and
> >>> felix trunk.
> >>> To do this, I managed to ensure that my linux IO buffers cache are
> empty
> >>> before doing each test, by doing this:
> >>>
> >>> sync
> >>> echo 1>  /proc/sys/vm/drop_caches
> >>> echo 2>  /proc/sys/vm/drop_caches
> >>> echo 3>  /proc/sys/vm/drop_caches
> >>>
> >>> This ensures that my linux has released all buffer cache disk, like it
> >>> is
> >>> the case when I boot my host (cold start).
> >>> Now, for felix 3.0.2, I load my server in around *21,1 *seconds (it's
> >>> slow
> >>> because all disk buffers are empty):
> >>>
> >>>   2010-10-01 22:15:07,880 Starting Felix 3.0.2
> >>>   2010-10-01 22:15:18,121 Starting cluster node "test"
> >>>   2010-10-01 22:15:28,966 Cluster node "test" ready
> >>>
> >>> and with felix/trunk, indeed, the server starts a little bit quicker,
> >>> but
> >>> not so much: around *in 20,3* seconds (I also flushed disk buffers
> >>> before
> >>> starting the test):
> >>>
> >>>   2010-10-01 22:17:47,505 Starting Felix 3.1.0.SNAPSHOT
> >>>   2010-10-01 22:17:56,815 Starting cluster node "test"
> >>>   2010-10-01 22:18:07,798 Cluster node "test" ready
> >>>
> >>>
> >>> /pierre
> >>>
> >>>
> >>> On Fri, Oct 1, 2010 at 5:39 PM, Richard S. Hall<heavy@ungoverned.org
> >>> >wrote:
> >>>
> >>>   p.s. I deploy snapshots of the everything, including the convenience
> >>>> framework distribution or you can build from trunk.
> >>>>
> >>>>
> >>>> On 10/1/10 11:37, Richard S. Hall wrote:
> >>>>
> >>>>   Hey everyone,
> >>>>>
> >>>>> I've made some changes to the bundle cache layout and handling to
> >>>>> improve
> >>>>> performance and clean up code. I've tried to maintain backward
> >>>>> compatibility
> >>>>> with old caches, but I'd appreciate is someone could try it out
on
> >>>>> their
> >>>>> caches to see if it works.
> >>>>>
> >>>>> ** Back up your caches just in case! **
> >>>>>
> >>>>> Note that the new cache format will not work with older frameworks.
> So
> >>>>> you
> >>>>> cannot use a cache created with this new framework with previous
> >>>>> frameworks;
> >>>>> however, previously created caches should mostly continue to work
> >>>>> between
> >>>>> the two although there will be a loss of fidelity since changes
to
> >>>>> state
> >>>>> are
> >>>>> only saved to the new way or the old way depending on if you are
> >>>>> running
> >>>>> on
> >>>>> the new framework or the old one.
> >>>>>
> >>>>> For the curious, I've combined five bundle cache files (id,
> location,
> >>>>> state, start level, last modified, and refresh count) into a single
> >>>>> file
> >>>>> and
> >>>>> try to avoid excessive file accesses. This appears to improve
> startup
> >>>>> time
> >>>>> when you have a cache with lots of bundles, but your mileage will
> vary
> >>>>> based
> >>>>> on platform and other factors. Although, you won't necessarily see
> any
> >>>>> improvements if you are using an old cache, since it will revert
to
> >>>>> the
> >>>>> old
> >>>>> way.
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> ->  richard
> >>>>>
> >>>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message