mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <vinodk...@apache.org>
Subject Re: CMake build refactoring
Date Fri, 25 Aug 2017 20:00:45 GMT
+1

On Fri, Aug 25, 2017 at 12:08 PM, <andrew@schwartzmeyer.com> wrote:

> I have the Java libraries building now on both Linux and Windows (though
> the initial setup on Windows is a bit annoying to get all the tools in the
> right place, I'd like to follow up this chain with a `docs/cmake.md` with
> all the odd little bits of information).
>
> I also enabled the ZooKeeper unit tests (which I needed in order to
> integration test the patches that went upstream to ZooKeeper to give their
> C Client library a CMake build, too).
>
> Furthermore, with the Java library, I've successfully run Marathon against
> CMake built Mesos libraries on Linux.
>
> I do not have the Python libraries building yet, and there is a "Java
> examples" library that is not yet built. I'll make sure to file issues to
> update these.
>
> Joe is working to update the Mesosphere CI, I will check with him about
> updating `docker-build.sh`.
>
> Thanks Vinod!
>
>
> On 08/25/2017 11:59 am, Vinod Kone wrote:
>
>> This is great to hear. Thanks for your perseverance on making this happen
>> Andy!
>>
>> Btw, do we build Java and Python libraries too now?
>>
>> Once we land the CMake changes, lets make sure to update
>> `support/docker-build.sh` to make sure it is continuously being tested on
>> our CI.
>>
>> On Fri, Aug 25, 2017 at 11:47 AM, <andrew@schwartzmeyer.com> wrote:
>>
>> Hello Mesos developers,
>>>
>>> I wanted to take the time to announce a (long) patch chain that fixes up
>>> the CMake build system. There were a lot of open bugs due to the initial
>>> design (and constraints of CMake 2), and by moving to CMake 3, I was able
>>> to rewrite the CMake build in a target-based dependency graph manner (the
>>> way CMake is meant to be used).
>>>
>>> The very end of the chain is here: https://reviews.apache.org/r/61753/
>>>
>>> Joseph Wu is currently shepherding it. That final patch includes the what
>>> and the why explanation of this patch series.
>>>
>>> My main guideline in this effort was MESOS-3576, which pointed out the
>>> incorrectness of the link flags generated by CMake. This has been
>>> resolved,
>>> and moreover, a significant amount of superfluous CMake code was deleted
>>> (that final patch was about a 4 to 1 ratio of deletions to adds).
>>>
>>> I took a two-phase approach to this reworking. The first phase imported
>>> all of our third party dependencies correctly. Instead of linking to
>>> these
>>> dependencies by manually including their various include/library
>>> directories and compilation/link flags, each dependency was added to the
>>> CMake graph as an imported target. All necessary information for linking
>>> to
>>> the dependency is recorded in its target, and used is transitively passed
>>> to targets which take it as dependency by CMake. Adding a new dependency
>>> is
>>> now relatively simple, with many easy-to-replicate patterns available in
>>> 3rdparty/CMakeLists.txt.
>>>
>>> An important note is that this was also done for header-only libraries,
>>> including stout. CMake 3 has a notion of "interface libraries" where a
>>> header-only library can be represented as a CMake target like any other
>>> library. With this done correctly, linking stout to boost is as simple as
>>> `target_link_libraries(stout boost)`, and linking libprocess to stout is
>>> `target_link_libraries(process stout)`, and the boost dependency is
>>> understood transitively.
>>>
>>> The second phase was refactoring the Mesos build itself (that is, not the
>>> third party dependencies). With all our dependencies imported properly, I
>>> was able to delete the files of extraneous information such as
>>> `MasterConfigure.cmake`, `AgentConfigure.cmake`, etc. This information is
>>> instead correctly stored in the aforementioned CMake dependency graph.
>>>
>>> With this all put together, the entirety of the required CMake code to
>>> build the agent executable, on all platforms, is the following _two
>>> lines_:
>>>
>>>     add_executable(mesos-agent main.cpp)
>>>     target_link_libraries(mesos-agent PRIVATE mesos)
>>>
>>> All necessary link and compilation flags are parsed by CMake through the
>>> dependency graph as it visits libmesos as the mesos-agent dependency.
>>>
>>> I keep an updated tree on GitHub here for easy testing:
>>> https://github.com/andschwa/mesos/blob/cmake-refactor/src/sl
>>> ave/CMakeLists.txt
>>>
>>> As we move closer to deprecating Autotools, I wanted to ensure that the
>>> replacement build system was as correct and easy-to-use as possible. As
>>> with any build system (and any software engineering effort), there is
>>> always more to clean up and improve. However, I am satisfied with the
>>> result of my efforts, and I hope this build system makes your work as
>>> developers easier. If you have the time, please take a look at the the
>>> patches and give it a test. Let me know know I can make it work better
>>> for
>>> you.
>>>
>>> Thank you,
>>>
>>> Andrew Schwartzmeyer
>>>
>>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message