Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 17 Mar 2016 09:09:33 +0000 (UTC)
From: "Vrushali C (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12857216.1439992846000.46664.1458205773684@Atlassian.JIRA>
In-Reply-To: <JIRA.12857216.1439992846000@Atlassian.JIRA>
References: <JIRA.12857216.1439992846000@Atlassian.JIRA>
 <JIRA.12857216.1439992846272@arcas>
Subject: [jira] [Updated] (YARN-4062) Add the flush and compaction
 functionality via coprocessors and scanners for flow run table
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/YARN-4062?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vrushali C updated YARN-4062:
-----------------------------
    Attachment: YARN-4062-YARN-2928.08.patch


Attaching patch v8 to address Sangjin=E2=80=99s review comments.=20

bq. (FlowRunCoprocessor.java) l.268: Logging of request.isMajor() seems a b=
it cryptic. It would print strings like "Compactionrequest=3D ... true for =
ObserverContext ...". Should we do something like request.isMajor() ? "majo=
r compaction" : "minor compaction" instead? And did you mean it to be an IN=
FO logging statement?

Updated the log message. I would like to log it at the INFO level. Compacti=
ons have traditionally taken up cycles on region servers such that they hav=
e been blocked from other operations, hence it would be good to know the de=
tails of this request that has come in.=20

bq. (FlowRunColumnPrefix.java) l.43: if we're removing the aggregation oper=
ation from the only enum entry, should we remove the aggregation operation =
(aggOp) completely from this class?
As we chatted offline, the aggOp operation should be part of any ColumnPref=
ix for this table. I have filed YARN-4786 for enhancements to add in a FINA=
L attribute (among other things).

bq. =09l.381-382: should we throw an exception as this is not possible (cur=
rently)?
Hmm. Not sure if throwing an exception is the right thing here. Thinking ou=
t loud, in the case that we add in another cell agg operation but have a di=
fferent scanner invoked, throwing an exception won=E2=80=99t be correct. =
=20

thanks
Vrushali
bq. =09=E2=96=AA=09l.220: could you elaborate on why this change is needed?=
 I'm generally not too clear on the difference betweencellLimit and limit.=
=20

There was a warning generated by check style I think, which is why I modifi=
ed the code. cellLimit and limit are differently named variables since prev=
iously some functions had an argument which was called limit and check styl=
e flagged that in the maven jenkins build.


> Add the flush and compaction functionality via coprocessors and scanners =
for flow run table
> -------------------------------------------------------------------------=
------------------
>
>                 Key: YARN-4062
>                 URL: https://issues.apache.org/jira/browse/YARN-4062
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4062-YARN-2928.04.patch, YARN-4062-YARN-2928.05=
.patch, YARN-4062-YARN-2928.06.patch, YARN-4062-YARN-2928.07.patch, YARN-40=
62-YARN-2928.08.patch, YARN-4062-YARN-2928.1.patch, YARN-4062-feature-YARN-=
2928.01.patch, YARN-4062-feature-YARN-2928.02.patch, YARN-4062-feature-YARN=
-2928.03.patch
>
>
> As part of YARN-3901, coprocessor and scanner is being added for storing =
into the flow_run table. It also needs a flush & compaction processing in t=
he coprocessor and perhaps a new scanner to deal with the data during flush=
ing and compaction stages.=20


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)