hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6094) Update the coprocessor to be a dynamically loaded one
Date Tue, 17 Jan 2017 19:33:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-6094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826665#comment-15826665
] 

Sangjin Lee commented on YARN-6094:
-----------------------------------

Thanks for the patch [~vrushalic]!

As for the unit test failures, they are related with the changes in the patch. {{DataGeneratorForTests}}
assumes the presence of the {{timelineservice}} jar under its {{target}} directory, but that
is true only if {{mvn package}} was done on that module. The unit tests don't require that.
We need a different way to get at the location of the jar (coming from the maven local cache).

Question: I see we're making a change to upload and use the jar that has the co-processor
from hdfs, rather than adding it to the hbase install. Is that change needed for the dynamic
co-processor? What is the benefit of doing that over using it out of the local install?

(FlowRunCoprocessor.java)
- maybe I'm bit on the paranoid side, but checking whether it is operating on the right table
seems like a reasonable sanity check nonetheless; do we need to remove this check?

(DataGeneratorForTest.java)
- Do we need to move the package for this and {{TestFlowDataGenerator}}? I'm curious as to
why that move is necessary. The diffs are now deleting the old files and creating the new
files so it's not so easy to see what has changed.


> Update the coprocessor to be a dynamically loaded one
> -----------------------------------------------------
>
>                 Key: YARN-6094
>                 URL: https://issues.apache.org/jira/browse/YARN-6094
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>         Attachments: YARN-6094.001.patch, YARN-6094-YARN-5355.001.patch, YARN-6094-YARN-5355.002.patch
>
>
> The timeline service v2 code base on yarn now uses hbase 1.2.4 after YARN-5976. 
> With this version of hbase, system classes (starting with org.apache.hadoop) can be loaded
as table level coprocessors. Hence we should update the timeline service coprocessor to be
a dynamically loaded one instead of static loading. 
> This involves code changes as well as documentation updates for deployment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message