flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Hogan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-8180) Refactor driver outputs
Date Thu, 30 Nov 2017 21:28:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-8180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Greg Hogan updated FLINK-8180:
------------------------------
    Description: 
The change in 1.4 of algorithm results from Tuples to POJOs broke the writing of results as
csv. Testing this was and is a challenge so was not done. There are many additional improvements
which can be made based on recent improvements to the Gelly framework.

Result hash and analytic results should always be printed to the screen. Results can optionally
be written to stdout or to a file. In the latter case the result hash and analytic results
(and schema) will also be written to a top-level file.

The "verbose" output strings can be replaced with json which is just as human-readable but
also machine readable. In addition to csv and json it may be simple to support xml, etc. Computed
fields will be optionally printed to screen or file (currently these are always printed to
screen but never to file).

Testing will be simplified since formats are now a separate concern from the stream.

Jackson is available to Gelly as a dependency provided in the Flink distribution but we may
want to build Gelly as a fat jar in order to include additional modules (which may require
a direct dependency on Jackson, which would require checkstyle suppressions around the unshaded
jackson imports).

  was:
The change in 1.4 of algorithm results from Tuples to POJOs broke the writing of results as
csv. Testing this was and is a challenge so was not done. There are many additional improvements
which can be made based on recent improvements to the Gelly framework.

Result hash and analytic results should always be printed to the screen. Results can optionally
be written to stdout or to a file. In the latter case the result hash and analytic results
(and schema) will also be written to a top-level file.

The "verbose" output strings can be replaced with json which is just as human-readable but
also machine readable. In addition to csv and json it may be simple to support xml, etc. Computed
fields will be optionally printed to screen or file (currently these are always printed to
screen but never to file).

Testing will be simplified since formats are now a separate concern from the stream.

Jackson is available to Gelly as a dependency provided in the Flink distribution but we may
want to build Gelly as a fat jar in order to include additional modules (which may require
a direct dependency on Jackson, which would fail the checkstyle requirement to use the shaded
package).


> Refactor driver outputs
> -----------------------
>
>                 Key: FLINK-8180
>                 URL: https://issues.apache.org/jira/browse/FLINK-8180
>             Project: Flink
>          Issue Type: Improvement
>          Components: Gelly
>    Affects Versions: 1.5.0
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
>             Fix For: 1.5.0
>
>
> The change in 1.4 of algorithm results from Tuples to POJOs broke the writing of results
as csv. Testing this was and is a challenge so was not done. There are many additional improvements
which can be made based on recent improvements to the Gelly framework.
> Result hash and analytic results should always be printed to the screen. Results can
optionally be written to stdout or to a file. In the latter case the result hash and analytic
results (and schema) will also be written to a top-level file.
> The "verbose" output strings can be replaced with json which is just as human-readable
but also machine readable. In addition to csv and json it may be simple to support xml, etc.
Computed fields will be optionally printed to screen or file (currently these are always printed
to screen but never to file).
> Testing will be simplified since formats are now a separate concern from the stream.
> Jackson is available to Gelly as a dependency provided in the Flink distribution but
we may want to build Gelly as a fat jar in order to include additional modules (which may
require a direct dependency on Jackson, which would require checkstyle suppressions around
the unshaded jackson imports).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message