drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Rogers <prog...@maprtech.com>
Subject Drill-on-YARN pull requests
Date Mon, 11 Jul 2016 05:31:26 GMT
Hi All,

The Drill-on-YARN work (DRILL-1170) is drawing to a close and is ready to be merged into master.
The project resulted in about 80+ new files, 10K+ lines of code, so this will be a bit of
an effort to merge. 

Drill-on-YARN is actually independent of the rest of Drill. Except for changes to the Drill
launch scripts (DRILL-4581), no Drill code was harmed in the making of this feature; all code
is new. The best approach would be if Drill were set up for semi-independent subprojects.
But, since it is not, the DoY build is integrated into Drill’s own build as a Maven module.

To make the task more manageable I’ll divide the work into about six requests, split mostly
along subsystem boundaries. Each will build. But, (somewhat obviously), the feature won’t
run until all subsystems are merged.

Testing consists of unit tests for those items that lend themselves to such tests. YARN is
a tool for launching distributed apps via generated shell scripts, so it is a bit difficult
to test full YARN integration in a unit test. Instead, we’ll rely on system integration
tests for that aspect.

If you need more background, please consult the documents attached to DRILL-1170. These include
an overview and a draft user guide.

Thanks ahead of time for reviewing the changes. Please let me know if I can provide other
materials to make the task easier.

- Paul

View raw message