aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxim Khutornenko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AURORA-1501) Add a job configuraiton diff RPC
Date Tue, 06 Oct 2015 16:46:27 GMT

     [ https://issues.apache.org/jira/browse/AURORA-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Maxim Khutornenko updated AURORA-1501:
--------------------------------------
    Description: 
{quote}
Problem:
We currently don't have a good user experience around "aurora job
diff" command. The task configs are dumped as "prettified" JSON
strings and diffed with the system diff tool. Anyone who tried to use
it knows it can be very hard to read especially when it comes to
executor data deltas. Also, the implementation is done completely
within the Aurora client making it hard to reuse this feature by other
clients (e.g.: an external deploy coordination tool).

Proposal:
Move the diff logic to the scheduler and expose it via a new
jobConfigDiff thrift API.

Benefits:
- Client will no longer have the custom non-reusable logic moving us
closer towards a "thin client" goal.
- The new RPC can be fully used by any existing or new API clients.
- The diff output will be improved via leveraging third party POJO
and/or JSON diff libraries (1,2,3, etc.).
- The server updater can be partially/fully unified with the new diff
logic further improving the overall DRY-ness.

Concerns:
- The executor data is currently treated as an opaque string blob on
the scheduler side. In reality, it's almost guaranteed to be JSON. In
order to deliver the best UX, the scheduler would have to start
requiring ExecutorConfig.data to be a valid JSON.

(1) -
http://java-object-diff.readthedocs.org/en/latest/getting-started/#getting-started
(2) - http://javers.org/documentation/diff-examples/
(3) - https://github.com/skyscreamer/JSONassert
{quote}
Full background here: http://markmail.org/message/g7dcqrzjpvc4eyuh


  was:
{quote}
Problem:
We currently don't have a good user experience around "aurora job
diff" command. The task configs are dumped as "prettified" JSON
strings and diffed with the system diff tool. Anyone who tried to use
it knows it can be very hard to read especially when it comes to
executor data deltas. Also, the implementation is done completely
within the Aurora client making it hard to reuse this feature by other
clients (e.g.: an external deploy coordination tool).

Proposal:
Move the diff logic to the scheduler and expose it via a new
jobConfigDiff thrift API.

Benefits:
- Client will no longer have the custom non-reusable logic moving us
closer towards a "thin client" goal.
- The new RPC can be fully used by any existing or new API clients.
- The diff output will be improved via leveraging third party POJO
and/or JSON diff libraries (1,2,3, etc.).
- The server updater can be partially/fully unified with the new diff
logic further improving the overall DRY-ness.

Concerns:
- The executor data is currently treated as an opaque string blob on
the scheduler side. In reality, it's almost guaranteed to be JSON. In
order to deliver the best UX, the scheduler would have to start
requiring ExecutorConfig.data to be a valid JSON.

(1) -
http://java-object-diff.readthedocs.org/en/latest/getting-started/#getting-started
(2) - http://javers.org/documentation/diff-examples/
(3) - https://github.com/skyscreamer/JSONassert
{quote}
Full background here: http://markmail.org/message/66snhpctvq3btefv



> Add a job configuraiton diff RPC 
> ---------------------------------
>
>                 Key: AURORA-1501
>                 URL: https://issues.apache.org/jira/browse/AURORA-1501
>             Project: Aurora
>          Issue Type: Epic
>          Components: Client, Scheduler
>            Reporter: Maxim Khutornenko
>
> {quote}
> Problem:
> We currently don't have a good user experience around "aurora job
> diff" command. The task configs are dumped as "prettified" JSON
> strings and diffed with the system diff tool. Anyone who tried to use
> it knows it can be very hard to read especially when it comes to
> executor data deltas. Also, the implementation is done completely
> within the Aurora client making it hard to reuse this feature by other
> clients (e.g.: an external deploy coordination tool).
> Proposal:
> Move the diff logic to the scheduler and expose it via a new
> jobConfigDiff thrift API.
> Benefits:
> - Client will no longer have the custom non-reusable logic moving us
> closer towards a "thin client" goal.
> - The new RPC can be fully used by any existing or new API clients.
> - The diff output will be improved via leveraging third party POJO
> and/or JSON diff libraries (1,2,3, etc.).
> - The server updater can be partially/fully unified with the new diff
> logic further improving the overall DRY-ness.
> Concerns:
> - The executor data is currently treated as an opaque string blob on
> the scheduler side. In reality, it's almost guaranteed to be JSON. In
> order to deliver the best UX, the scheduler would have to start
> requiring ExecutorConfig.data to be a valid JSON.
> (1) -
> http://java-object-diff.readthedocs.org/en/latest/getting-started/#getting-started
> (2) - http://javers.org/documentation/diff-examples/
> (3) - https://github.com/skyscreamer/JSONassert
> {quote}
> Full background here: http://markmail.org/message/g7dcqrzjpvc4eyuh



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message