incubator-ooo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Ramm <m...@geek.net>
Subject Re: Sourceforge and AOO 3.4 distribution
Date Thu, 22 Mar 2012 16:58:48 GMT
*We have finally assessed the capacity and capabilities needed to serve the
surge of Apache OpenOffice 3.4 release-time traffic.  Before we could
commit to delivering the full download volume, we wanted to produce a
vetted plan, including a clear timeline and backing technical
implementation plans.

First let me quickly recap my understanding of the problems we are trying
to solve for:

   - Apache OpenOffice 3.4 will be released in mid April and we want to
   assure capacity to handle that traffic both in terms of bandwidth and
   simultaneous connections.
   - The Apache OpenOffice project would benefit to be able to promote the
   release heavily without worrying about capacity.


Given those needs and the fact the Apache Infrastructure team said they’d
welcome our assistance, we at SourceForge think we can help and that there
would be mutual benefit.
What we are proposing is an elaboration of Joe’s ‘hybrid’ approach:

   - Both AOO and SF.net mirror networks would be used to provide download
   capacity for the 3.4 release.
   - SourceForge.net would be the “recommended default download” on the
   website.
   - Apache Mirror network would be an alternate download option.
   - Apache OpenOffice team and Infrastructure team will maintain control
   of the the auto-update URL’s and possibly follow Rob’s suggestion to
   stagger automatic updates.


SourceForge.net will manage the full burst capacity for web-based downloads
through our global network of OSS mirrors, global CDN network(s) and cloud
file server providers.   Using these resources, we anticipate our capacity
is well above the expected delivery requirements for the upcoming release.

In addition to basic download capacity, SourceForge will provide detailed
download statistics, which will support future product, infrastructure and
marketing plans.  We will commit to make stats available on the
SourceForge.net website and provide stats delivery APIs.  We are able to
capture initiated downloads, not just page views, and will provide them
split by geography and operating system.  We’re also willing to consider
additional stats needs.

Proposed Timeline:

   - Immediately: SourceForge sets up Apache Infra team with credentials on
   an AOO mirror project in sf.net
   - First week:  SourceForge updates contracts with CDN and other
   providers to handle full AOO peak release traffic
   - Second Week: AOO Infra team works with sf.net operations team to ramp
   traffic to sf.net in a controlled way in order to gather statistical
   data, verify assumptions, and give the Apache infrastrucure team time to
   verify our capacity.
   - 1-2 days post test:  SF.net analyzes traffic data, assures that our
   assumptions about geographic mix, and interactive vs automated download
   mix, are valid and we can do this in a fiscally responsible way.
   - 1-2 days post test: AOO infrastructure team analyses traffic data,
   lets sf.net team know any additonal data needs, and validates that the
   system will work for them


Once everything is tested and vetted on both sides, we will need to make a
CDN bandwidth commit, and would like the AOO team to commit to notifying us
30 days prior to shutting down the flow of traffic, so that we can update
our contracts and avoid penalties.

We believe that the combination of SF.net mirrors, and CDN based burst
capacity will provide a fast and stable download experience for AOO users,
and **will allow the AOO team to publicize the release in an agressive
manner.*

On Wed, Mar 21, 2012 at 10:55 AM, Mark Ramm <mark@geek.net> wrote:

> >And finally: would you have any objection to us using a mix of fixed
>> >mirrors, elastic file delivery services (like s3), and commercial CDN
>> >service to handle spikes in download gracefully and assure that global
>> >users get good download performance when local mirrors are overloaded
>> >or not available?
>>
>>
>> No, we may even be willing to budget some amount for this purpose.
>> Cost estimates would be appreciated as our budget numbers for FY2012
>> need to be finalized next week.
>
>
> Sorry that it's taken a bit to get back to you.   We are working on
> getting pricing from a variety of providers, and my personal goal is to
> find a way for us to fund the CDN and S3 costs, and to provide this to the
> community as a free (as in beer) service.
>
> Thanks everybody who provided anecdotal information on historical traffic
> peaks, and particularly for the steady state run rate information.   That
> has been invaluable as we talk with vendors about the suplemental capacity
> we need to acquire to handle peak loads.
>
> There's one key input to figuring out if I can pay for all of this out of
> ad revenue, which is what percentage of the daily downloads are expected to
> come from auto-updater software or other non-browser scripts?   Would that
> traffic still be pointed primarily at AOO owned domains and mirrors, or
> would we be handling some of that from the sf.net service?
>
> And finally, I'd also be interested in finding out if you know percentage
> of traffic is from North America vs the rest of the world because some
> providers give very different rates for different locations, for example
> Cloudfront publishes $0.02/gb US and $0.12/gb in South America.
>
> Thanks again for to everybody who helped with data so far!
>
> --Mark Ramm
>



-- 
*Mark Ramm*
Director of Engineering,
SourceForge Developer Experience
phone: 734-707-7266
email: mark@geek.net
skype: geekmark

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message