Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 13 Nov 2017 10:38:00 +0000 (UTC)
From: "Chen Qingcha (JIRA)" <jira@apache.org>
To: yarn-dev@hadoop.apache.org
Message-ID: <JIRA.13117897.1510569461000.220097.1510569480166@Atlassian.JIRA>
In-Reply-To: <JIRA.13117897.1510569461000@Atlassian.JIRA>
References: <JIRA.13117897.1510569461000@Atlassian.JIRA> <JIRA.13117897.1510569461982@jira-lw-us.apache.org>
Subject: [jira] [Created] (YARN-7481) Gpu locality support for Better AI
 scheduling
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Mon, 13 Nov 2017 10:38:04 -0000

Chen Qingcha created YARN-7481:
----------------------------------

             Summary: Gpu locality support for Better AI scheduling
                 Key: YARN-7481
                 URL: https://issues.apache.org/jira/browse/YARN-7481
             Project: Hadoop YARN
          Issue Type: New Feature
          Components: api, RM, yarn
    Affects Versions: 2.7.2
            Reporter: Chen Qingcha
             Fix For: 2.7.2


We enhance Hadoop with GPU support for better AI job scheduling.=20
Currently, YARN-3926 also supports GPU scheduling, which treats GPU as coun=
table resource.=20
However, GPU placement is very important to deep learning job for better ef=
ficiency.
 For example, a 2-GPU job runs on gpu {0,1} could be faster than run on gpu=
 {0, 7}, if GPU 0 and 1 are under the same PCI-E switch while 0 and 7 are n=
ot.
 We add the GPU support to Hadoop 2.7.2 to enable GPU locality scheduling, =
which support fine-grained GPU placement. A 64-bits bitmap is added to yarn=
 Resource, which indicates both GPU usage and locality information in a nod=
e=C2=A0(up to 64 GPUs per node). '1' means available and '0' otherwise in t=
he corresponding position of the bit.=C2=A0=C2=A0=20


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org