drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1170) YARN support for Drill
Date Wed, 06 Apr 2016 20:54:25 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229083#comment-15229083

Paul Rogers commented on DRILL-1170:

Good progress is being made. Our tentative goal is the Drill 1.8 release for an initial integration.
The goal is:

YARN support in Drill 1.8 enables admins to migrate their existing Drill cluster to run under
YARN. The admin simply identifies the nodes on which Drill should run, identifies the required
container sizes, and brings up the Drill cluster under YARN. YARN manages resource allocations
for Drill alongside those of other YARN applications. Drill-on-YARN monitors Drill-bits and
automatically restarts any that fail.

We'll have "experimental" support for starting/stopping Drill-bits. Starting bits is easy.
Stopping is a bit of a challenge because we lack DRILL-2656.

> YARN support for Drill
> ----------------------
>                 Key: DRILL-1170
>                 URL: https://issues.apache.org/jira/browse/DRILL-1170
>             Project: Apache Drill
>          Issue Type: New Feature
>            Reporter: Neeraja
>            Assignee: Paul Rogers
>             Fix For: Future
> This is a tracking item to make Drill work with YARN.
> Below are few requirements/needs to consider.
> - Drill should run as an YARN based application, side by side with other YARN enabled
applications (on same nodes or different nodes). Both memory and CPU resources of Drill should
be controlled in this mechanism.
> - As an YARN enabled application, Drill resource consumption should be adaptive to the
load on the cluster. For ex: When there is no load on the Drill , Drill should consume no
resources on the cluster.  As the load on Drill increases, resources permitting, usage should
grow proportionally.
> - Low latency is a key requirement for Apache Drill along with support for multiple users
(concurrency in 100s-1000s). This should be supported when run as YARN application as well.

This message was sent by Atlassian JIRA

View raw message