Mailing-List: contact dev-help@mesos.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mesos.apache.org
MIME-Version: 1.0
From: Kapil Arya <kapil@mesosphere.io>
Date: Tue, 9 Feb 2016 19:14:02 -0500
Message-ID: 
 <CA+fJHLjV=-W5OJnxfYU3sVhkqSv4MsMjZEvSrAtNEMZ+9xmbzg@mail.gmail.com>
Subject: Reorganize 3rdparty directory
To: dev <dev@mesos.apache.org>
Content-Type: multipart/alternative; boundary=047d7b5d2fd2d5a34a052b5f505c

--047d7b5d2fd2d5a34a052b5f505c
Content-Type: text/plain; charset=UTF-8

Hi All,

TLDR: Move everything from 3rdparty/libprocess/3rdparty/* into 3rdparty/.
(Optionally) Move libprocess/stout to the top-level directory.

I wanted to start some discussion around reorganizing stuff inside
"3rdparty". I apologize for the length of the email, please bear with me.

Traditionally, 3rdparty has been used to hold all Mesos dependencies
(zookeeper, libprocess, protobuf, stout, etc.). Further,
libprocess/3rdparty was to hold all libprocess dependencies (which may in
turn be Mesos dependencies as well).

As I understand, the original motivation was to emphasize that libprocess
is an independent project which depends on "stout", which in turn is also
an independent project.

However, in the current code base, we don't strictly follow the 3rdparty
structure. For example, stout has a dependency on picojson and
google-protobuf, but we don't put these two packages inside
3rdparty/libprocess/3rdparty/stout/3rdparty/.

In light of these anomalies, I want to propose that we flatten out the
3rdparty directory and put all packages (libprocess, stout, protobuf,
picojson, zookeeper, etc.) at the same level in 3rdparty. We can still use
"--with-XYZ=..." to the full extent as needed.

To take it a step further, I want to propose that we bring libprocess and
stout out of 3rdparty/ and move them at the top level (i.e., make them
peers of src/). That way, all code in 3rdparty/ is stuff from "third"
parties and is used only when "--with-bundled" is defined (by default).
This hierarchy will still allow us to keep libprocess and stout as separate
independent projects.

The motivation for this proposal came when dealing with 3rdparty
dependencies for module development. A module developer needs access to
protobuf, picojson, glog, etc., and for that matter, the exact versions of
these packages that Mesos was compiled with.

We want to solve this problem by installing module-specific 3rdparty
dependencies into "${pkglibdir}/3rdparty" as part of "make install" (if
configured with something like "--enable-install-module-dependencies").
(There is a discussion going on in a separate thread).

Further, as of today, when we install Mesos using "make install", we
install stout headers in "${prefix}/include/stout". However, stout has
dependencies on picojson[1] and google-protobuf headers which may not be
present on the machine. This leaves stout, and in turn libprocess and Mesos
headers, fairly broken. I understand that this issue is somewhat orthogonal
to the main issue being discussed in this mail, but I wanted to put it out
since it's related.

Any thoughts, comments, concerns are most welcome!

Best,
Kapil


[1]: Picojson issue was resolved as part of
https://reviews.apache.org/r/41424/ which installs picojson.h into the
include-dir.

--047d7b5d2fd2d5a34a052b5f505c--