hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4062) Add the flush and compaction functionality via coprocessors and scanners for flow run table
Date Thu, 17 Mar 2016 09:09:33 GMT

     [ https://issues.apache.org/jira/browse/YARN-4062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vrushali C updated YARN-4062:
    Attachment: YARN-4062-YARN-2928.08.patch

Attaching patch v8 to address Sangjin’s review comments. 

bq. (FlowRunCoprocessor.java) l.268: Logging of request.isMajor() seems a bit cryptic. It
would print strings like "Compactionrequest= ... true for ObserverContext ...". Should we
do something like request.isMajor() ? "major compaction" : "minor compaction" instead? And
did you mean it to be an INFO logging statement?

Updated the log message. I would like to log it at the INFO level. Compactions have traditionally
taken up cycles on region servers such that they have been blocked from other operations,
hence it would be good to know the details of this request that has come in. 

bq. (FlowRunColumnPrefix.java) l.43: if we're removing the aggregation operation from the
only enum entry, should we remove the aggregation operation (aggOp) completely from this class?
As we chatted offline, the aggOp operation should be part of any ColumnPrefix for this table.
I have filed YARN-4786 for enhancements to add in a FINAL attribute (among other things).

bq. 	l.381-382: should we throw an exception as this is not possible (currently)?
Hmm. Not sure if throwing an exception is the right thing here. Thinking out loud, in the
case that we add in another cell agg operation but have a different scanner invoked, throwing
an exception won’t be correct.  

bq. 	▪	l.220: could you elaborate on why this change is needed? I'm generally not too clear
on the difference betweencellLimit and limit. 

There was a warning generated by check style I think, which is why I modified the code. cellLimit
and limit are differently named variables since previously some functions had an argument
which was called limit and check style flagged that in the maven jenkins build.

> Add the flush and compaction functionality via coprocessors and scanners for flow run
> -------------------------------------------------------------------------------------------
>                 Key: YARN-4062
>                 URL: https://issues.apache.org/jira/browse/YARN-4062
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4062-YARN-2928.04.patch, YARN-4062-YARN-2928.05.patch, YARN-4062-YARN-2928.06.patch,
YARN-4062-YARN-2928.07.patch, YARN-4062-YARN-2928.08.patch, YARN-4062-YARN-2928.1.patch, YARN-4062-feature-YARN-2928.01.patch,
YARN-4062-feature-YARN-2928.02.patch, YARN-4062-feature-YARN-2928.03.patch
> As part of YARN-3901, coprocessor and scanner is being added for storing into the flow_run
table. It also needs a flush & compaction processing in the coprocessor and perhaps a
new scanner to deal with the data during flushing and compaction stages. 

This message was sent by Atlassian JIRA

View raw message