trafficserver-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Kenny <>
Subject Re: Proposal for how to update source code layout.
Date Tue, 26 Jan 2016 20:23:13 GMT

So the alternative build system is a low priority item as if we clean up the automake files
they will be easier to maintain and understand. For me it more about being able to easily
use other systems to build ATS. Personally I would like to make a Scons/Parts build setup
for ATS. I am a little bias on this honestly (as i wrote Parts and work with SCons), but I
have had a lot of luck with these systems, and honestly for larger systems that I know use
this combo, people are generally happy about what it can do. The stuff is not perfect ( what
is) but it is being improved. It still a big improvement to raw make and autotools in my view
( and from others that have used it). I would like to provide such a system to the team so
people could try it out and praise or curse it as they like. 

While I can go on about the many advantages for SCosn and Parts over autotools, the main difference
between the two would be that autotools is a build file generator which makes make files based
on some configuration. SCons and Parts are a build system that can build differently based
on a configuration information passed in one of many different ways. This means if you are
building ats on a linux box you can say:

scons --cfg=debug all 

and get a debug build, then

scons --cfg=release

and get a release build, then realize you need to support building with Intel compiler v15
for 32-bit machines compatible with gcc 4.8 so you do a 

scons --tc=icc_15,gcc_4.8 --target=x86

without wasting a minutes configuring new files rebuilding stuff, losing stuff if you did
not run configure in a different directory, etc.
The main value of make in the unix/linux world is that it exists as part of the system, SCons
and Parts are not, and requires python run. 

I am happy to talk more about it. However for me the issue is that having options to only
makes everything better and stronger. At the moment I think the current setup is not doing


From: Phil Sorber <>
To:; Jason Kenny <> 
Sent: Tuesday, January 26, 2016 1:34 PM
Subject: Re: Proposal for how to update source code layout.

Can you talk more about the alternative build system? This is one that I don't think we need
to change really, but I am open to the idea and want to understand it better.


On Tue, Jan 26, 2016 at 12:11 PM Jason Kenny <> wrote:

Thank you for your input. 
>Items like P_ I_ and other general code clean up I was hoping to do in a different phase.
many of us could easy make a massive diff with all most of the issues corrected. However this
diff would make most of our heads explode. I think that it is very important to make each
phase small and directed to help allow movement in a good direction. I know personally there
are a number of item in the code itself I would like to address as well as fixing up the testing
system. I believe that getting the relayout down first will help all other efforts as this
will allow for clearer and more understandable diffs when we address issues such as fixing
up the P_ I_ stuff or adding new tests, etc.
>Personally I would like to address:
>1) provide an alternative build system to the current auto-tools2) clean up the testing.
This is not small amount of work, and I would hope that I would be able to provide a better
subsystem that we all can use. One that is more accessible and useful than what we have at
the moment.3) some stuff in the code like the memory allocator and threaded event system,
which i have a background in doing
>But before this can be done correctly I really believe we have to clean up the source
layout. I want to get this done first and as quick as possible to reduce the pain caused by
the source moves that would upset existing ( and future) pull requests.
>      From: Phil Sorber <>
> To:; Jason Kenny <>
> Sent: Tuesday, January 26, 2016 12:32 PM
> Subject: Re: Proposal for how to update source code layout.
>This seems generally good to me. I welcome a more modern structure. I also prefer the
first layout you mentioned, but it may be necessary to do the second layout and then move
to the first. That might make others more comfortable. I personally see a lot of value in
what you describe, but I know others may still need more convincing and specific details.
>Also, you didn't mention some things specifically, but I assume we are doing away with
things like the P_ and I_ in the header files? We should also make it so that the docs don't
get needlessly built. I'm sure there are a bunch of other things, but those are the ones that
come to mind.
>Another thing I think we need to be careful with is people who have long lived feature
branches. They really need to be rebasing against master a lot or perhaps just pushing their
changes upstream quicker.
>Be prepared for a lot of bike shedding. This is likely to be a long slow process as we
address the merits of each change and how it fits into the bigger picture and what are the
pros and cons of each.
>On Mon, Jan 25, 2016 at 3:43 PM Jason Kenny <> wrote:
>Hi everyone,
>Second try as the first attempt seems to be a blob of unformated text.
> I want to see about moving forward on the source clean up I believe we all agreed to
the last ATS summate. What I would like to get general agreement on what I plan to do to the
source layout so it is clear why and where everything will go before I start making lots of
small pull requests pulling moving source and cleaning up the build.
>First what are the goals?
>Cleaning up the source to will proved:
>1)            Better clarity on where to put new code.
>    a.            Making it easier to refactor code
>    b.            Making it easier for others to change code
>2)            simplify the build code
>    a.            making it easier to use other build system if desired
>    b.            making it easier for others to add or remove code
>3)            untangle the dependency mess of everything needing everything else
>4)            untangle what modules are being built
>5)            Make it easier for everyone to understand the Architecture and design
>6)            Make it easier to improve/cleanup  the testing systems
>7)            Allow a better setup to hopefully avoid any major source layout refactors
for a long time
>I want to clarify that this was generally agreed on by everyone at the ATS summate. We
want to avoid this unless needed. I believe people feel overall this would be a good thing,
but I want to make sure that I define the envisioned layout we will move to so we all understand
the where we are going before doing this. Ideally we will want to do the major part of this
work as fast as we can to minimize the pain as much as possible.
>Before I go into the layout, I want to talk about process. I plan to break this up into
>Phase 1 – move source to new location and any header cleanup that might be needed. Once
this phase is done we will still have dependency issues to deal with, however it should be
clearer on what is wrong, and some of the dependency issues should just go away
>Phase 2 – refactor source to deal with dependency issues, to clean up mega source files
and start some cleanup of the tests. This will make it easier to find objects and make changes
in a way to that should not make compiling more complex. It should also make it easier to
understand and learn the code.
>The hope I have is that will help minimize the size of any given diff. Will have more
diff overall, but these will all be small and to the point.
>Now let talk about layout (finally :-) )
>The current layout has something like this at the top layer
>Source needed to build Traffics Server exits in /proxy, /mgmt, /lib, /cmd and /iocore.
These directories also have tests in them as well. In some case the tests and the source is
mixed. I would like to change the top level to look like this instead:
>We move the code in /examples, /proxy, /mgmt, /lib, /cmd and /iocore. The addition of
a src and test directory will help clarify what is the source to than traffic server application
vs that of being a test or something else, such as an example or an optional plug-in. The
code for /examples moves under /plugins, everything else under /src
>In the /src directory we would have a layout like this:
>  api/
>    c/
>    cpp/
>  config/
>  cmd/
>    …
>  core/
>    …
>  Infra/
>    …
>api – contains all the API code for a given language. Any new language API would be
added here under a directory correctly names for the language.
>config- Contains the default config file to run traffic server.
>cmd – Contains the code for each program in a name directory.
>core – Contains the main modules to build different programs under the cmd/ directory.
Modules under here depend on modules in Infra, or other modules in Core itself. Code here
should be more Traffic server specific modules.
>infra – Contains the support modules. Modules in here are leaf modules, may be reusable.
If the module is not a leaf module it can only depend on code in Infra and should be generally
>Modules should not depend on other modules that depend on them. The only expectations
would be header only dependencies that don’t require link time circular dependencies.
>Modules in CMD can depend on modules in API, Core, Infra
>Modules in API can depend on modules in Core, Infra
>Modules in Core can depend on modules in Core, Infra
>Modules in Infra can depend on modules in Infra
>Header Patterns:
>The general include pattern will be changed to #include <module/header.h> for modules
depending in on other headers. Files depending on headers in the same module will be in the
form of #include “header.h”
>An alternative to this layout would be
>  api/
>    c/
>    cpp/
>  config/
>  mgmt/
>  proxy/
>    …
>  iocore/
>    …
>  lib/
>    …
>The main difference is here is that it look like it did before. However much of the code
in proxy or mgmt would be moved to lib as it is a leaf component, or the code would be moved
in to a directory based on  module it would be used to generate, vs having lots of modules
in a given directory. Because of this I felt it might be easier to use the first layout as
it contains a simpler set of logic on where code goes.
>Given that we agree with the layout. The process will be to move components that are on
the leaf first and slowly move up the chain. Each commit will be on component at a time, with
changes to the source files #include and any make file changes to allow the code to build
as expected. After a component is moved that changes that break up source can happen, where
the commit will include the makefile changes and the split source any header file changes
need to support the split.
>I think that is it. I will deal with “tests” after main source changes.
>Any question or concerns?

View raw message