mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhitao Li <zhitaoli...@gmail.com>
Subject Re: CMake build refactoring
Date Fri, 25 Aug 2017 21:40:51 GMT
Hi Andrew,

Thank you for the great work. Really looking forward to see a more
structured dependency graph.

A couple of questions:

1) is there a target version of this work? Will the chain land on 1.5?

2) For people who maintains Mesos modules, is there a example build of how
people can arrange their build?

3) Is there plan to make CMake the only supported build system? If yes,
what's the graduation standard?

Thanks!

On Fri, Aug 25, 2017 at 1:00 PM, Vinod Kone <vinodkone@apache.org> wrote:

> +1
>
> On Fri, Aug 25, 2017 at 12:08 PM, <andrew@schwartzmeyer.com> wrote:
>
> > I have the Java libraries building now on both Linux and Windows (though
> > the initial setup on Windows is a bit annoying to get all the tools in
> the
> > right place, I'd like to follow up this chain with a `docs/cmake.md`
> with
> > all the odd little bits of information).
> >
> > I also enabled the ZooKeeper unit tests (which I needed in order to
> > integration test the patches that went upstream to ZooKeeper to give
> their
> > C Client library a CMake build, too).
> >
> > Furthermore, with the Java library, I've successfully run Marathon
> against
> > CMake built Mesos libraries on Linux.
> >
> > I do not have the Python libraries building yet, and there is a "Java
> > examples" library that is not yet built. I'll make sure to file issues to
> > update these.
> >
> > Joe is working to update the Mesosphere CI, I will check with him about
> > updating `docker-build.sh`.
> >
> > Thanks Vinod!
> >
> >
> > On 08/25/2017 11:59 am, Vinod Kone wrote:
> >
> >> This is great to hear. Thanks for your perseverance on making this
> happen
> >> Andy!
> >>
> >> Btw, do we build Java and Python libraries too now?
> >>
> >> Once we land the CMake changes, lets make sure to update
> >> `support/docker-build.sh` to make sure it is continuously being tested
> on
> >> our CI.
> >>
> >> On Fri, Aug 25, 2017 at 11:47 AM, <andrew@schwartzmeyer.com> wrote:
> >>
> >> Hello Mesos developers,
> >>>
> >>> I wanted to take the time to announce a (long) patch chain that fixes
> up
> >>> the CMake build system. There were a lot of open bugs due to the
> initial
> >>> design (and constraints of CMake 2), and by moving to CMake 3, I was
> able
> >>> to rewrite the CMake build in a target-based dependency graph manner
> (the
> >>> way CMake is meant to be used).
> >>>
> >>> The very end of the chain is here: https://reviews.apache.org/r/61753/
> >>>
> >>> Joseph Wu is currently shepherding it. That final patch includes the
> what
> >>> and the why explanation of this patch series.
> >>>
> >>> My main guideline in this effort was MESOS-3576, which pointed out the
> >>> incorrectness of the link flags generated by CMake. This has been
> >>> resolved,
> >>> and moreover, a significant amount of superfluous CMake code was
> deleted
> >>> (that final patch was about a 4 to 1 ratio of deletions to adds).
> >>>
> >>> I took a two-phase approach to this reworking. The first phase imported
> >>> all of our third party dependencies correctly. Instead of linking to
> >>> these
> >>> dependencies by manually including their various include/library
> >>> directories and compilation/link flags, each dependency was added to
> the
> >>> CMake graph as an imported target. All necessary information for
> linking
> >>> to
> >>> the dependency is recorded in its target, and used is transitively
> passed
> >>> to targets which take it as dependency by CMake. Adding a new
> dependency
> >>> is
> >>> now relatively simple, with many easy-to-replicate patterns available
> in
> >>> 3rdparty/CMakeLists.txt.
> >>>
> >>> An important note is that this was also done for header-only libraries,
> >>> including stout. CMake 3 has a notion of "interface libraries" where a
> >>> header-only library can be represented as a CMake target like any other
> >>> library. With this done correctly, linking stout to boost is as simple
> as
> >>> `target_link_libraries(stout boost)`, and linking libprocess to stout
> is
> >>> `target_link_libraries(process stout)`, and the boost dependency is
> >>> understood transitively.
> >>>
> >>> The second phase was refactoring the Mesos build itself (that is, not
> the
> >>> third party dependencies). With all our dependencies imported
> properly, I
> >>> was able to delete the files of extraneous information such as
> >>> `MasterConfigure.cmake`, `AgentConfigure.cmake`, etc. This information
> is
> >>> instead correctly stored in the aforementioned CMake dependency graph.
> >>>
> >>> With this all put together, the entirety of the required CMake code to
> >>> build the agent executable, on all platforms, is the following _two
> >>> lines_:
> >>>
> >>>     add_executable(mesos-agent main.cpp)
> >>>     target_link_libraries(mesos-agent PRIVATE mesos)
> >>>
> >>> All necessary link and compilation flags are parsed by CMake through
> the
> >>> dependency graph as it visits libmesos as the mesos-agent dependency.
> >>>
> >>> I keep an updated tree on GitHub here for easy testing:
> >>> https://github.com/andschwa/mesos/blob/cmake-refactor/src/sl
> >>> ave/CMakeLists.txt
> >>>
> >>> As we move closer to deprecating Autotools, I wanted to ensure that the
> >>> replacement build system was as correct and easy-to-use as possible. As
> >>> with any build system (and any software engineering effort), there is
> >>> always more to clean up and improve. However, I am satisfied with the
> >>> result of my efforts, and I hope this build system makes your work as
> >>> developers easier. If you have the time, please take a look at the the
> >>> patches and give it a test. Let me know know I can make it work better
> >>> for
> >>> you.
> >>>
> >>> Thank you,
> >>>
> >>> Andrew Schwartzmeyer
> >>>
> >>>
>



-- 
Cheers,

Zhitao Li

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message