Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <259456805.1216392152417.JavaMail.jira@brutus>
Date: Fri, 18 Jul 2008 07:42:32 -0700 (PDT)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3412) Refactor the scheduler out of the
 JobTracker
In-Reply-To: <53274895.1211066635619.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HADOOP-3412?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D126=
14737#action_12614737 ]=20

Hadoop QA commented on HADOOP-3412:
-----------------------------------

+1 overall.  Here are the results of testing the latest attachment=20
  http://issues.apache.org/jira/secure/attachment/12386376/JobScheduler-v12=
.1.patch
  against trunk revision 677872.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified test=
s.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of java=
c compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number=
 of release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2901/t=
estReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2=
901/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/=
2901/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2901=
/console

This message is automatically generated.

> Refactor the scheduler out of the JobTracker
> --------------------------------------------
>
>                 Key: HADOOP-3412
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3412
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Brice Arnould
>            Assignee: Brice Arnould
>            Priority: Minor
>             Fix For: 0.19.0
>
>         Attachments: JobScheduler-v10.patch, JobScheduler-v11.patch, JobS=
cheduler-v12.1.patch, JobScheduler-v12.patch, JobScheduler-v9.1.patch, JobS=
cheduler-v9.2.patch, JobScheduler-v9.patch, JobScheduler.patch, JobSchedule=
r_v2.patch, JobScheduler_v3.patch, JobScheduler_v3b.patch, JobScheduler_v4.=
patch, JobScheduler_v5.patch, JobScheduler_v6.1.patch, JobScheduler_v6.2.pa=
tch, JobScheduler_v6.3.patch, JobScheduler_v6.4.patch, JobScheduler_v6.patc=
h, JobScheduler_v7.1.patch, JobScheduler_v7.patch, JobScheduler_v8.patch, R=
ackAwareJobScheduler.java, SimpleResourceAwareJobScheduler.java
>
>
> First I would like warn you that my proposition is assumed to be very nai=
ve. I just hope that reading it won't make you lose time.
> h4. The aim
> It seems to me that improving Hadoop scheduling could be very profitable.=
 But, it is hard to implement and compare schedulers, because the schedulin=
g logic is mixed within the rest of the JobTracker.
> This bug is the first step of an attempt to improve the Hadoop scheduler.=
 It re-implements the current scheduling algorithm in a separate class call=
ed JobScheduler. This new class is instantiated in the JobTracker.
> h4. Bug fixed as a side effects
> This patch probably cannot be submited as it is.
> A first difficulty is that it does not have exactly the same behaviour th=
an the current JobTracker. More precisely, it doesn't re-implement things l=
ike code that seems to be never called or concurency problems.
> I wrote TOCONFIRM where my proposition differ from the current implementa=
tion, so you can find them easily.
> I know that fixing bugs silently is bad. So, independently of what you de=
cide about this patch, I will open issues for bugs that you confirm.
> h4. Other side effects
> Another side effect of this patch is to add documentation about each step=
 of the scheduling. I hope that it will help future improvement by lowering=
 the level required to contribute to the scheduler.
> It also reduces the complexity and the granularity of the JobTracker (mak=
ing it more parallel).
> h4. The future
> If you feel that this is a step the right direction, I will try to propos=
e a JobSchedulerInterface that many JobSchedulers could implement and to pr=
opose alternatives to the current =C2=AB FifoJobScheduler =C2=BB.  If some =
of you have ideas about that please tell ^^ I will also open issues for thi=
ngs marked as FIXME in the patch.

--=20
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.