spamassassin-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Plesset <>
Subject Re: Email / Inbox Speed Problems
Date Sat, 24 Oct 2009 18:35:53 GMT

Ted Mittelstaedt wrote:
> Jay Plesset wrote:
>> Ted Mittelstaedt wrote:
>>> What is the point of a quota system that does not limit the
>>> received mail?  And if it does limit it then we get irate calls from
>>> people complaining that sally sue sent them a message and got it
>>> returned.  Of course, sally sue never reads the error message
>>> and tells our user that their e-mail box is too large - or if
>>> she did, then irate user thinks it's our problem.
>> Um, well, that's not exactly how it works.
>> System messages and "guranteed delivery" messages always get through.
>> Messages that will take a user over quota are held for a configurable 
>> "grace" period, and the user is warned that they are over quota at a 
>> configurable repeat rate.  Messages are returned to the sender after 
>> a configurable hold period.  there are plenty of knobs for you to 
>> turn. . .
> I can understand that, and in a corporate environment where you
> have more control over the userbase (and the users are much more
> inclined to listen to you, after all it's not their money on the line)
> I am sure it would work well.  Of course, if I was using a
> -standards- based method of handling mail in such an environment
> (ie: NOT MS Exchange) then I wouldn't be using POP3 in the first
> place, I'd be using IMAP and I'd also setup a set of shared
> e-mail folders accessible from the IMAP client.  I'd also probably
> run some scripts that warned me when people were letting their
> inbox get too large, so I could go train them in how to drag the
> mail messages they want to save into private or shared folders
> on the server.  But, that's my style - other admins might go out
> and buy software to do this.  Ultimately it works the same way.
> This discussion really illustrates the disconnect between people who
> write e-mail systems for a living and what ISP's need.  While I've
> not looked at the Sun comm suite your talking about, I'm sure it's
> not that much different from many other commercial e-mail systems
> I've been pitched over the years from people wanting to make my
> life easier as an ISP admin (in exchange for some money, of course)
Just to be clear, the software I'm offering is designed not to "replace 
Exchange", but for ISP's or large corp accounts.  One of the customers 
I'm assigned to support has 100 "store" systems, each with 500,000 
mailboxes and typically sees 30,000 simultaneous imap connections.

We often see systems with a million mailboxes.

You like webmail?  Our webmail interface also talks to our Calendar 
Server, our IM server, and should shortly include gateways into other IM 
systems.  It's all pretty open, based on standard protocols, and no, 
there isn't a gui admin interface.  Maybe later.  The MTA has been 
around for 25 years, previously called, "PMDF". 

Yes, we'd like you to license it, and pay for support.  You can download 
and use at no cost. . .

> The problem though is when I've drilled into them, I've always found
> issues like this.  Those systems are written first as competitors to
> Exchange, and make a boatload of assumptions about the users, and
> the admin's skill level.  Usually they assume the users are smarter
> and the admins are dumber.  That's about right for the corporate
> networks I've admined.  But ISPs don't survive unless the admin is
> a lot smarter - because the users in general are a lot dumber.
> Oh, there's exceptions - but most of the time it's customers who
> work in office environments and come home and want the same level of
> support they get at the office.  Those people are in a minority.
> The majority of customers quite obviously don't understand very much,
> and with a surprising number of them they don't even understand the
> accepted nomenclature.
> If I had a nickel for every time I've told a user "OK now open your
> web browser" and gotten back "what's a web browser" I'd be a rich
> man.  I've learned to refer to web browsers with phrases like
> "go to google" or "click on the Internet".  This is the level of
> skill we deal with regularly.  After all, it's not the new-technology
> embracers who are calling in for ISP support.  It's the people
> who were left behind years ago, who are only on the Internet because
> the rest of their family won't spend the time to communicate with
> them unless they are on facebook or e-mail.  At least once a week
> I and the other admins get someone who we just shake our head over
> and wonder why in the world this person is even wasting their money
> and time with a computer at all - they are like the old grandmother
> who never drives on the highway and never drives faster than 45Mph
> who owns a Lamborghini.  It's really a sad thing, to be honest.
>>> Not to mention the user thinks their inbox is -on their mac-
>>> not on our mailserver, since of course they are entirely
>>> unaware that their applemail has the setting flipped that
>>> leaves a copy of the message on the mailserver.
>>> Sending them notifications is worthless since they don't know what
>>> they are, they don't know how to shut them off, and 3/4 of
>>> the time they think they are spam anyway.
>>> The whole point of this is customer management.  Your average mac
>>> user is as dumb as a stump.  As long as things work they assume
>>> everything is hunky-dory.  If things stop working they NEVER
>>> assume it's their Mac that's the problem because Steve Jobs
>>> told them Mac's are infallible and they worship the ground he
>>> walks on.  And of course, if they stop working they stop working
>>> at the worst time for them, (late at night on Friday) because of
>>> the laws of Mr. Murphy.  So by the time they get ahold of us
>>> they are hopping mad, they assume it's our problem, and the Pope
>>> himself could tell them that it's their own stupid fault and
>>> they wouldn't believe him.
>> Well, it's true that most users don't know much, but it's my 
>> experience that many admins don't know much more. . .
> That is true and I've seen it myself, but mostly in the corporate
> arena.  I'd say easily 3/4 of the admins in corporate America would
> fail miserably if they went to work for an ISP and half of those
> would go back to corporate work after a few months and never look
> at working an ISP again, even if they were so poor they were living
> out of paper bags.
>>> Naturally, if things start getting slow they ALSO
>>> automatically assume it's our problem - but they generally
>>> are not emotional to the point that they won't listen.  So
>>> they call in, expecting to "inform" us about something we
>>> are doing wrong - whereupon we have to tell them that their
>>> Mac that they believe is infallible is really fallible because
>>> Apple's programmers are idiots and select retarded defaults.
>>> That's a terrible blow to their world view, and it's often about
>>> the most that they can digest.
>> It's very similar to what I tell admins when they get "winmail.dat" 
>> attachments they can't read.  Yep, Exchange isn't very compliant.
> That's the understatement of the year.
>>> But the key here is that when they get off the call they are
>>> fixed (because their Applemail is now correctly deleting the
>>> mail that it downloads) and that they DON'T believe that it
>>> was "our problem", and they have actually learned something
>>> about how e-mail works.
>>> I can also see your next argument - if we inform them in advance
>>> that their mail client isn't deleting the mail it downloads
>>> that we might avoid this.  The problem is that first, we don't
>>> know in advance if they are running a large mailbox because they
>>> are dumb-as-post mac users, oblivious to the world, or if they
>>> are running a large mailbox because they are running IMAP or
>>> some such that doesn't have that problem with the mailserver.
>>> If they do know what they are doing, and we call them, we look
>>> like idiots, and it's annoying to them, or worse they get the
>>> impression we want them to go away.  Second, if they are 
>>> dumb-as-post users, they automatically assume that if we tell them 
>>> to change a setting in their Applemail, that it's because our 
>>> mailserver is screwed up - because, after all Macs are infallible, 
>>> and everything that Apple does must be the One True Way to setup a 
>>> computer.
>>> It's really better in the long run to make them come to us, not
>>> for us to go to them.  If they come to us at least they are
>>> acknowledging that there's a problem.  Remember, problems with
>>> computers are very frightening to people who are ignorant about
>>> computers.  Think about it, you don't know squat about your car's
>>> transmission - so if a mechanic tells you your transmission has
>>> a problem, your going to be scared to death it's going to cost you
>>> thousands.  Your average Mac user will go into denial when they
>>> have a problem with their Mac - they will refuse to believe for
>>> the longest time that there's a problem even when it's obvious
>>> there's a problem to a blind monkey.  They have to believe there's
>>> a problem before they are even willing to be educated in how
>>> to fix the problem.
>>> As I said, this is customer management.  Just keep in mind that
>>> when your dealing with the general public, the more ignorant the
>>> person you work with, the more likely they are to assume they are
>>> right, and you are wrong.  For us to win at the game we must
>>> educate the users, and the most ignorant of the users will only
>>> open their minds for knowledge for a very short time, before it
>>> snaps closed like a steel trap, and they will never believe
>>> there's a problem unless they see it for themselves.
>>> After all, just think of your average conservative Republican's
>>> reaction to Global Warming.  It's not something they can see and
>>> their brains are (apparently) incapable of imagination so they cannot
>>> imagine that Global Warming is real, that's why they make silly
>>> arguments like "global warming must not be happening because
>>> we are having a pretty cold winter"  It's the same principle in 
>>> operation here.
>> Well, it's the devil you know vs the one you don't.  I was offereing 
>> a solution that doesn't slow down.  If you don't think it would help 
>> you, then you don't have to look at it.
> And what I was saying is that your initial post is only looking at
> one small thing of an entire system.
> Have you ever seen a manual typewriter?  Did you know that the
> qwerty keyboard was deliberately designed to be ridiculously
> inefficient?  Better keyboard layouts exist that allow people to
> type at 100's of wpm without much practice or training, because
> they move the most commonly used keys to the strongest fingers.
> qwerty was setup the way it was because the designers of the
> typewriter first tried doing it correctly - and then discovered
> typists were jamming the mechanicals of the typewriters because
> they were typing so fast.
> This is an example of design that improved one small thing and
> caused a lot of problem side effects.
> The same issues exist in e-mail systems.  Users by their nature do
> a lot of very bad things to mail systems - the most common one out
> there is treating the e-mail server like a Lotus Notes document
> management system.  Microsoft discovered this with Exchange - they
> had a lot of complaints from users that searching for and managing
> e-mail attachments was difficult under Outlook.  The proper thing
> to do would have been to tell those customers that they needed to
> buy a document management system.  Instead, MS improved the document
> management capabilities in Outlook.  Then, a few years later customers
> found that once they exceeded 4GB of space in their Outlook inbox,
> every mail message and document attachment in the inbox would
> become corrupted.  MS's answer was to fix Exchange to allow 64GB
> inboxes - but of course, that only works if the inbox on Outlook
> is on the Exchange server.  If they are using Outlook with an ISPs
> POP3 server, they get screwed due to the side-effect of this limit -
> and lots of them run into this because Outlook's document management
> is so easy to use.
> I realize this is a complex answer and probably doesn't neatly
> fit in a sales literature sound bite.  In summary, sure, changing
> software would speed one thing up, would "fix" one thing.  I don't 
> argue this.  But it is going to have undesirable side effects that 
> represent
> breakages of other things.  And those breakages then have other fixes,
> which have yet more side effects, and so on and so on.  Software 
> development houses love this stuff - because they can just write
> code additions forever that fix more and more things, and they are 
> always in business - but at the same time, the software gets more and
> more bloated with features until it becomes so complex to use that
> nobody can use it anymore.
> As was stated by Scotty in Star Trek III, The Search for Spock:
> "The more they overthink the plumbing, the easier it is to stop up
> the drain"
> That quote was made over 25 years ago.  It's amazing how many people
> still don't get it.
> Ted
>> jay
>>> Ted
>>> Jay Plesset wrote:
>>>> Many of my users use the various quota settings in Messaging 
>>>> Server.  You can set quotas on message number and/or mailbox size.  
>>>> Notifications are sent to the user, even if they're over quota. . .
>>>> You can set quota individually, by "class of service", or globally.
>>>> Yes, it'll run on the same hardware you're running now.  On Redhat 
>>>> 4 or 5, or Solaris.
>>>> jay
>>>> Ted Mittelstaedt wrote:
>>>>> Jay Plesset wrote:
>>>>>> Geez, unless your users are into the millions of messages, maybe

>>>>>> you need a more scalable mail server.   My day job is support of

>>>>>> the Sun comms suite.  I only get these when there are litterally

>>>>>> tens of millions of messages in an inbox.
>>>>> Where we generally get these problems is when users are running 
>>>>> MacOS X
>>>>> and using the included free Apple Mail as a POP3 client, because one
>>>>> of the DEFAULTS of that client is to leave a copy of the mail message
>>>>> on the server.  The typical scenario is that we get one of these 
>>>>> users
>>>>> who runs it this way for a couple months, then one day their relative
>>>>> starts e-mailing them 50MB pictures of their latest vacation, and 
>>>>> once
>>>>> their e-mail box exceeds 800MB in size, popper (qpopper) starts 
>>>>> getting
>>>>> really slow in downloading the message ID list and their client 
>>>>> starts
>>>>> running like a dog.
>>>>> There's probably many ways I could fix it, from replacing qpopper to
>>>>> going to faster disks or more powerful hardware, or running a nightly
>>>>> script that squawks about the bad citizens, but I frankly don't
>>>>> feel compelled to allocate all of our POP3 users a gigabyte of 
>>>>> disk space for their mailbox, and if did fix it then I'd have to 
>>>>> setup
>>>>> quotas on /var/mail
>>>>> Doing it this way penalizes only the users who engage in the 
>>>>> objectionable behavior, and it penalizes them in such a way that 
>>>>> it doesn't cause them to lose mail, or cause the server to reject 
>>>>> incoming mail messages to them, or causes mail they have to be 
>>>>> truncated.  And
>>>>> it also doesn't do it in a way that is sudden - the user just starts
>>>>> noticing things getting slower and slower and slower over time - so
>>>>> they have plenty of time to contact us at their leisure.
>>>>> I suppose that one of these days the author of qpopper will rewrite
>>>>> the search logic in the qpopper program to fix this and then I'll 
>>>>> have
>>>>> to find some other way to gently enforce this.
>>>>> Ted
>>>>>> jay
>>>>>> Ted Mittelstaedt wrote:
>>>>>>> Sean Leinart wrote:
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Sean Leinart [] Sent:

>>>>>>>>> Friday, October 23, 2009 2:04 PM
>>>>>>>>> To: TJ Russ
>>>>>>>>> Cc:; Spamassassin Mailing
>>>>>>>>> Subject: Email / Inbox Speed Problems
>>>>>>>>> Hi TJ,
>>>>>>>>> Looking over your Inbox situation, you suffer from the
>>>>>>>>> problem as most here do. You have too much email stored
on the 
>>>>>>>>> server. Can you give me a rundown of the folders that
can be 
>>>>>>>>> eliminated in your Inbox, we can archive them off then
>>>>>>>>> them from your folders that are online, this will help
a great 
>>>>>>>>> deal.
>>>>>>>>> Thank you,
>>>>>>>>> Sean Leinart
>>>>>>>>> Network Systems Engineer
>>>>>>>>> First Service Carolina Inc.
>>>>>>>>> Raleigh, North Carolina
>>>>>>>>> United States
>>>>>>>>> 919-832-5553
>>>>>>>> DOH!!
>>>>>>>> List, please disregard the erroneous CC: post to the list.
>>>>>>> I had to look twice since it was the identical problem to what
>>>>>>> we deal with every week around here.
>>>>>>> Ted

View raw message