trafficserver-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Kenny <jke...@yahoo-inc.com.INVALID>
Subject Re: Proposal for how to update source code layout.
Date Tue, 26 Jan 2016 19:57:55 GMT
I recall your concerns at the last summate.

So I believe this proposal is to avoid big bang by making a set of small and lots of small
pull requests. I do have a OneNote page with general depends in it I can copy paste my notes
here if people want to see it. ( it will be a little ugly FYI as there are notes not a presentation)

The layout I propose here is based on my talk with you BTW as you had the very useful feedback
on what should be done. I will try to clarify a few more items.

the layout here is to simplify the layout and to remove a number of existing macro dependency
loops we have at the moment and make it a little harder to add them back in. By Macro I mean
stuff like modules lib depends on modules in iocore which depends on modules in lib. By fixing
this we will have a clear macro depends between iocore and libs. Of course the layout I suggest
does not have a lib or iocore, but a simple directory for API's( ie api), programs (ie cmds),
ATS specific code (core), general reusable code (infra). I cannot come up with a reason to
have logic like proxy or mgmt given what i see in the depend tree at this time, and with any
explanation of functionality. I think this layout would make it easier for us to see duplicate
logic and help us better refactor code in a more reliable form.

Given the step is done there will still exists micro depend issues between modules and files,
but that will be easier to address independently and after we relayout the source as and breaking
up of source will have a clear place for it to go.

Based on what you said to me with wanting to clean up xyz as we moved, I found that it will
be easier for use to move the source as a phase 1 then as a phase 2 or 3 fix up a module more
directly. The reason for this is to reduce overall pain and to keep the commits more understandable
and directed to a certain goal. 

I think in the end all the items you talk about needing to fix are items we all agree should
be addressed in some form. Much of this will be technically easier and less painful to address
after we relayout the source.

Again small directed fixes. This will not be a big bang, but a set of lot of small pull requests
as modules by module is moved over. If you want, for example tcl to disappear during the move.
I can make that happen (It is easy to have code fall out). Again to stress the pull request
will be like "move tsutil to src/infra/tsutil and updated makefiles". It will not be move
everything in a massive diff.
 
I hope this helps address your concerns. If you still have questions or concerns or suggestion
about something please let me know. I would like to make sure we have a clear understand of
why this is a needed pre-step to a number of other wanted and needed improvements to ATS.

Jason



________________________________
From: James Peach <jpeach@apache.org>
To: dev@trafficserver.apache.org; Jason Kenny <jkenny@yahoo-inc.com> 
Sent: Tuesday, January 26, 2016 12:47 PM
Subject: Re: Proposal for how to update source code layout.


I really think that developer time would be better spent on projects like removing the Tcl
dependency, figuring out whether proxy/StatSystem.cc is really needed any more, cleaning up
the XML dependencies, removing STL containers where it makes sense, improving test coverage
and usability, making it easy to run TSQA on releases, decoupling specific subsystems, moving
traffic_manager to iocore, etc.

Neither of the proposed layouts seems to capture the dependency graph that well, but there's
not a lot of detail here. For example, what are the modules in core? Why is core not just
traffic_server?

Overall, I don't think a big-bang approach makes sense; we should make incremental improvements,
being careful to avoid regressions and build problems.


> On Jan 25, 2016, at 2:42 PM, Jason Kenny <jkenny@yahoo-inc.com.INVALID> wrote:
> 
> 
> 
> 
> Hi everyone,
> 
> Second try as the first attempt seems to be a blob of unformated text.
> I want to see about moving forward on the source clean up I believe we all agreed to
the last ATS summate. What I would like to get general agreement on what I plan to do to the
source layout so it is clear why and where everything will go before I start making lots of
small pull requests pulling moving source and cleaning up the build.
> First what are the goals?
> Cleaning up the source to will proved:
> 1)            Better clarity on where to put new code.
>    a.            Making it easier to refactor code
>    b.            Making it easier for others to change code
> 2)            simplify the build code
>    a.            making it easier to use other build system if desired
>    b.            making it easier for others to add or remove code
> 3)            untangle the dependency mess of everything needing everything else
> 4)            untangle what modules are being built
> 5)            Make it easier for everyone to understand the Architecture and design
> 6)            Make it easier to improve/cleanup  the testing systems
> 7)            Allow a better setup to hopefully avoid any major source layout refactors
for a long time
> 
> I want to clarify that this was generally agreed on by everyone at the ATS summate. We
want to avoid this unless needed. I believe people feel overall this would be a good thing,
but I want to make sure that I define the envisioned layout we will move to so we all understand
the where we are going before doing this. Ideally we will want to do the major part of this
work as fast as we can to minimize the pain as much as possible.
> 
> Before I go into the layout, I want to talk about process. I plan to break this up into
phases.
> Phase 1 – move source to new location and any header cleanup that might be needed.
Once this phase is done we will still have dependency issues to deal with, however it should
be clearer on what is wrong, and some of the dependency issues should just go away
> Phase 2 – refactor source to deal with dependency issues, to clean up mega source files
and start some cleanup of the tests. This will make it easier to find objects and make changes
in a way to that should not make compiling more complex. It should also make it easier to
understand and learn the code.
> 
> The hope I have is that will help minimize the size of any given diff. Will have more
diff overall, but these will all be small and to the point.
> 
> Now let talk about layout (finally :-) )
> 
> The current layout has something like this at the top layer
> /build
> /ci
> /cmd
> /contrib
> /doc
> /example
> /iocore
> /lib
> /mgmt
> /plugins
> /proxy
> /rc
> /tools
> 
> Source needed to build Traffics Server exits in /proxy, /mgmt, /lib, /cmd and /iocore.
These directories also have tests in them as well. In some case the tests and the source is
mixed. I would like to change the top level to look like this instead:
> /build
> /ci
> /contrib
> /doc
> /plugins
> /rc
> /src
> /tests
> /tools
> 
> We move the code in /examples, /proxy, /mgmt, /lib, /cmd and /iocore. The addition of
a src and test directory will help clarify what is the source to than traffic server application
vs that of being a test or something else, such as an example or an optional plug-in. The
code for /examples moves under /plugins, everything else under /src
> In the /src directory we would have a layout like this:
> src/
>  api/ 
>    c/
>    cpp/
>  config/
>  cmd/ 
>    …
>  core/ 
>    …
>  Infra/ 
>    …
> 
> Details:
> api – contains all the API code for a given language. Any new language API would be
added here under a directory correctly names for the language.
> config- Contains the default config file to run traffic server.
> cmd – Contains the code for each program in a name directory.
> core – Contains the main modules to build different programs under the cmd/ directory.
Modules under here depend on modules in Infra, or other modules in Core itself. Code here
should be more Traffic server specific modules.
> infra – Contains the support modules. Modules in here are leaf modules, may be reusable.
If the module is not a leaf module it can only depend on code in Infra and should be generally
reusable.
> 
> Dependencies:
> Modules should not depend on other modules that depend on them. The only expectations
would be header only dependencies that don’t require link time circular dependencies.
> Modules in CMD can depend on modules in API, Core, Infra
> Modules in API can depend on modules in Core, Infra
> Modules in Core can depend on modules in Core, Infra
> Modules in Infra can depend on modules in Infra
> 
> Header Patterns:
> The general include pattern will be changed to #include <module/header.h> for modules
depending in on other headers. Files depending on headers in the same module will be in the
form of #include “header.h”
> 
> An alternative to this layout would be
> Src/
>  api/
>    c/
>    cpp/
>  config/
>  mgmt/
>  proxy/ 
>    …
>  iocore/
>    …
>  lib/ 
>    …
> 
> The main difference is here is that it look like it did before. However much of the code
in proxy or mgmt would be moved to lib as it is a leaf component, or the code would be moved
in to a directory based on  module it would be used to generate, vs having lots of modules
in a given directory. Because of this I felt it might be easier to use the first layout as
it contains a simpler set of logic on where code goes.
> Given that we agree with the layout. The process will be to move components that are
on the leaf first and slowly move up the chain. Each commit will be on component at a time,
with changes to the source files #include and any make file changes to allow the code to build
as expected. After a component is moved that changes that break up source can happen, where
the commit will include the makefile changes and the split source any header file changes
need to support the split.
> 
> I think that is it. I will deal with “tests” after main source changes.
> 
> Any question or concerns?
> 
> Jason

Mime
View raw message