httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter J. Cranstone" <cransto...@msn.com>
Subject RE: the wheel of httpd-dev life is surely slowing down, solutions please
Date Tue, 11 Nov 2003 13:02:36 GMT
Something else to think about...

What's the differentiator "in the market place" between 1.x and 2.x?

(hint: it's not a feature list)

If I was to go out and buy Apache (Covalent) apart from some management
tools (features) what's the biggest differentiator between the old version
(public domain 1.x) and the new version (2.x.) 

Invariably three things come to mind - performance, stability,
compatibility? 

So is Apache 2.x faster or slower than 1.x? Has anyone shown definitively
that 2.x can serve pages faster than 1.x? and if so by what percentage or
factor?

Looks like 2.1 is going to be more stable than the old 2.x versions but is
it more stable than 1.x?

Finally compatibility? Can I run my 1.x modules unmodified in a 2.x
environment? 

HTTPD dev life is slowing down - simply fixing bugs and adding new features
is not the way to differentiate, there's probably 50,000 users of 2.x while
there's millions and millions of 1.x users.

What's it going to take to shift them to 2.x?

Not features (only the minority of users now care) - not compatibility
(although it helps) - my bet is raw performance.

So, anyone got any hard data that shows Apache 2.x serving pages "factors"
faster than 1.x?

Regards,


Peter


-----Original Message-----
From: William A. Rowe, Jr. [mailto:wrowe@apache.org] 
Sent: Tuesday, November 11, 2003 2:00 AM
To: dev@httpd.apache.org
Cc: dev@httpd.apache.org
Subject: Re: the wheel of httpd-dev life is surely slowing down, solutions
please

At 07:14 PM 11/10/2003, Stas Bekman wrote:
>I have several reasons to believe that the wheel of httpd-dev life is
>slowing down and something has to be done to get this wheel up to the
>speed like in the good old days. The following observation are listed
>in no particular order. I've also tried to offer suggestions how to
>resolve problems.

Always good to discuss, but httpd is about folks contributing and the team
reviewing patches.  When there are lots of activity, things move quickly.
Summer is a slow period for alot of projects.  Stability slows things down
too (and yes, 2.0.48 is far more stable than anything released before on
the 2.0 branch.)

>1) Bugs
>
>searching for NEW and REOPENED bugs in httpd-2.0 returns: 420 entries
>http://nagoya.apache.org/bugzilla/buglist.cgi?bug_status=NEW&bug_status=REO
PENED&product=Apache+httpd-2.0

Yes - these are important, I'm the first to admit I have a number I track
that
I haven't resolved.  And more eyes are needed :)  However...

>Suggestion: make bugzilla CC bug-reports to the dev list. Most
>developers won't just go and check the bugs database to see if there
>is something to fix, both because they don't have time and because
>there are too many things to fix. Posting the reports to the list
>raises a chance for the developer to have a free minute and may be to
>resolve the bug immediately or close it.

Nope, it assures a good number of interested contributors SHUT DOWN
their subscriptions to httpd-dev.  Trust me, I subscribe to bug traffic as
well.

Guess which of the two I read daily?  If I didn't split them, I would never
be caught up on the developer discussions.  (Even those I don't understand
completely I do read.)

>2) Lack of Design
>
>In my personal opinion, the move to CTR from RTC had very bad
>implications on the design of httpd 2.1.

You seem to be misinformed here...  Apache 2.0 was ALWAYS CTR
until we decided to put a roadblock in the way of breaking the stable
version, and encourage that experimentation to move to an unstable
branch.  RTC was only introduced after the last ApacheCon.  And very
very resisted, at that.

>2a). Design of new on-going features (and changes/fixes) of the
>existing features is not discussed before it's put to the code. When
>it's committed it's usually too late to have any incentive to give a
>design feedback, after all it's already working and we are too busy
>with other things, so whatever.

However, "Show me the code" remains the mantra of httpd.  So continuing
to use CTR, the code is in place.  With httpd-test/perl-framework the coder
can ever verify that their new change doesn't break any tests that are
already defined.

>The worst part is that it's now easy to sneak in code which otherwise
>would never be accepted (and backport it to 2.0). I don't have any
>examples, but I think the danger is there.

That's always been true in httpd.  We didn't lock 1.3 until 2.0 was nearing
completion.  We didn't lock 2.0 until 2.1 was available for folks to hack
on.

It is VERY hard to get code into 2.0.  That code is RTC.  Gratuitous changes
and new features aren't encouraged.  Time to move on to 2.1 for all of the
great and wonderful things our web server aught to do.  In fact, it's why
I've
suggested point blank that the 'which pass at the startup file' patch you
offered
will never belong in 2.0 - anyone designing for an arbitrary 2.0 can't even
count
on such a feature.  But anyone using the future 2.2 release should trust
that
it is present.

>2b). As a side-effect when someone asks a design question (e.g. me)
>it's not being answered because people tell: we are in the CRT mode,
>go ahead code it and commit it. But if the poster doesn't know what's
>the right design, if she did she won't have asked in first place.

Nothing stops anyone from posting up code first.  Many of us post up
code when we just aren't sure if it's the one best answer - and although
it takes folks a while sometimes to review, the dialog is useful.  But that
discussion shouldn't be an obstacle to writing code.

>2c). Future design. There is no clear roadmap of where do we go with
>Apache 2.1. People scratch their itches with new ideas which is cool,
>but if there was a plan to work on things, this may have invited new
>developers to scratch their itches.

The roadmap is this.  When httpd-2.1 solves enough issues that 2.0 cannot
(such as a major auth reorganization, perhaps auth credential chains, and
hopefully some improvements to the filtering schema and file system
provider)
AND it can pass the regressions as gracefully as 2.0 - we are ready to start
with some alphas.  All someone has to do is speak up and say it's time.

No fixed lists of features set in stone, no.  If that *must have* feature
isn't
done, then it moves on to 2.3 to await inclusion in 2.4.  It all depends on
what folks can dedicate their time to.

>2d). CRT seemed to come as a replacement for design discussions. It's
>very easy to observe from the traffic numbers:
>[snip]
>Quiz: based on this input, tell which date CRT was introduced at.

CRT was always there.

The discrepancy was that 2.0 was a RADICAL overhaul :)

So early in development, folks raised many, many invaluable questions
about a whole new design.  httpd-2.0 (and 2.1) are not being changed
nearly as radically.

Remember httpd is also the sum of httpd-dev and apr-dev.  And that the
presence (at last) of a users list has pulled some burden off of httpd-dev.

>3). Contributions
>
>I don't have numbers to support my clause, but I have a strong feeling
>that nowadays we see a much smaller number of posts with contributions
>from non-developers, since most contributions are simply ignored and
>it's a known thing among Apache community that posting fixes and
>suggestions to httpd-dev list is usually a waste of time. (This is
>based on my personal experience and discussions with other developers
>who aren't httpd-core coders). I realize that I'm not entirely correct
>saying that, since some things are picked up, so I apologize to those
>folks who do try hard to pick those things.

I've heard this complaint too, and felt guilty about all of the interesting
and
worthwhile patches that I personally did not have time to address.  Many
still sit in my dev@httpd box awaiting free cycles (marked priority-urgent.)

>The obvious problem is that most of those things are unsexy, usually
>don't fit someones itch and they sometimes require a lot of
>communication overhead with the reporter/contributor just to
>understand what exactly is going on.

A problem in every open source project, and certainly one that we don't have
any obvious answer to.

>Solutions:
>
>a. certain (most?) chunks of httpd lack owners. If httpd was to have
>clear owners of certain code bases then it'll be easy to take care of
>bug reports and contributions/patches. Even though httpd is
>collectively owned, it doesn't preclude champions in certain areas,
>who can see that good things happen to the areas they overlook.

We do have champions, but many are preoccupied (I have mod_isapi
patches just sitting there waiting to be reviewed and finally collected into
a proof set that the replacement patches work for the existing cases plus
all the edge cases folks have discovered.)

>b. similar to the root rotation we have on the infrastructure, I
>suggest to have a voluntary weekly rotation of httpd-dev list champion
>whose responsibility is to make sure that most (all?) bug-reports and
>contributions are dealts with: assigned to those who those champions
>who know how to fix/review/adopt things (See (a) above), asking for
>more input if needed, etc. You don't have to know httpd as your five
>fingers to be able to do a great rewarding job.

We do, folks pick up the ball and bust through those reports for a while,
then lose time/interest/energy to keep up.  It's up to others to step up,
or have patience.

>c. turn the dealing with contributions and bugs into a sexy
>thing. Everybody wants to feed their ego, there is no better way to
>make your ego happy then getting a positive feedback from a person you
>have helped to resolve a bug or handhold to provide a good patch for
>the new feature, they spent so much time working on.

Of course, I recognize this, I hope others do too.  I'm proud to have
shepherded
many new contributor's patches into the code.

>3a) I can hardly see new developers joining the team, should we blame
>the economy or the lack of encouragement for people to contribute. I
>think we now have a higher rate of developers who leave the project
>(even though that technically they are still committers and all) the
>those who join the project. Which obviously has clear effects on the
>productivity of the dev list.

We have a high barrier to entry, because we have so many users who
count on the project.  We are always open to new suggestions, and these
belong on the pmc list.

>4). Feedback Preference and List Karma
>
>List Karma is a known issue in all Open Source projects. It's obvious
>that known and respected members of the httpd-dev list come to a
>higher preference when it comes to posting feedback and any answers at
>all. It's very easy to see that questions from most known httpd
>developers almost always have a long thread with resolution. Other
>posters are usually lucky to get any responses at all, and if they do
>usually the thread would die out without any resolutions.

That's unfair :-)  My comments are just as ignored as yours, or Joe
Newauthor
who dropped into the list once, I believe.  The relationship between posts
and
answers has more to do with the number of literate developers who understand
the area being discussed, and understand the contributor's comments.

>I'm not questioning the validity of the list karma, this is something
>that I happen to stick to myself on the lists where I can give useful
>feedback: I usually answer first questions from people whom I know for
>good and bad. What I want to talk about is about the karma of the
>posters who don't ask to solve their "personal" problems, but act on
>behalf sub-projects built upon httpd. These posters try to solve a
>problem for hundreds, thousands and sometimes millions of people, who
>won't use Apache at all if they all they needed is to serve static
>pages -- there are several other web servers which outperform Apache
>on this front. They use Apache because of the 3rd party modules that
>build upon Apache. It's those 3rd party modules that Apache should be
>thankful to to its current world domination in the web server sector.

Undoubtedly.

>Therefore I think httpd developers should stop thinking, "this
>poster's question is not httpd's problem, I have better things to
>do". I'd like to suggest that it's a very own httpd problem, because
>usually you get questions from 3rd party developers when something in
>httpd doesn't cooperate with those 3rd party modules and needs to be
>fixed. I think it's much less important to work on Apache 2.1 and much
>more important to polish 2.0 and make 3rd party developers
>happy. (wearing the 3rd party module developer hat) we don't need no
>Apache 2.1 in the sky, we need Apache 2.0 on steady on the ground, so
>we can run our modules on.

We have 2.0 - we bolted it to the wall so that mod_newfeature.so would
load on 2.0.43, or 2.0.99.  We are diverting the creative (read - not yet
fully
though out) energies to an unstable branch that assures 2.0 is almost always
ready for release when someone has the impetus to cut another release.

>and if I may add that giving those 3rd party developers a commit
>access is not a solution to the problem, because they hardly survive
>coping with their own projects' bugs, support issues and feature
>demands. So telling them: "hey, we gave you a commit access, just go
>and code/fix it". is not always working.

Commit access isn't lightly granted either, so this certainly isn't an
option 
that is likely to be entertained.

Let me reiterate about httpd-2.1.  It will never be released, it is the
sandbox
that leads us to 2.2 - when it's close.  That means we must beg our third
party module authors to hack in the workaround for 2.0 - and look forward
and propose those changes they want that will make 2.2 the answer to all
of their problems.

In reality, we will see a handful of patches to 2.0 that won't ever be
applied
to that tree, but should be considered for immediate inclusion in 2.1 so
that
we can hand it back and ask, "Here - a new version - does your module work
better with our next release?"  We hope, of course the answer is yes.

And when we make changes to 2.1 - we need to be the community to try
the code, and offer up "No!!! That messes up my beautiful design for
mod_foo!"
Tell us early so we offer the best platform for supporting third party
modules.

Stas, thanks for raising these issues, and I hope your comments do generate
some additional activity, around making 2.0.49 even more stable, and around
improving the software so that the 2.2.0 release solves hosts of issues.

Bill


Mime
View raw message