flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2090) toString of CollectionInputFormat takes long time when the collection is huge
Date Tue, 09 Aug 2016 10:30:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413355#comment-15413355
] 

ASF GitHub Bot commented on FLINK-2090:
---------------------------------------

Github user StephanEwen commented on the issue:

    https://github.com/apache/flink/pull/2323
  
    Looks good, merging this...


> toString of CollectionInputFormat takes long time when the collection is huge
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-2090
>                 URL: https://issues.apache.org/jira/browse/FLINK-2090
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Till Rohrmann
>            Assignee: Ivan Mushketyk
>            Priority: Minor
>
> The {{toString}} method of {{CollectionInputFormat}} calls {{toString}} on its underlying
{{Collection}}. Thus, {{toString}} is called for each element of the collection. If the {{Collection}}
contains many elements or the individual {{toString}} calls for each element take a long time,
then the string generation can take a considerable amount of time. [~mikiobraun] noticed that
when he inserted several jBLAS matrices into Flink.
> The {{toString}} method is mainly used for logging statements in {{DataSourceNode}}'s
{{computeOperatorSpecificDefaultEstimates}} method and in {{JobGraphGenerator.getDescriptionForUserCode}}.
I'm wondering whether it is necessary to print the complete content of the underlying {{Collection}}
or if it's not enough to print only the first 3 elements in the {{toString}} method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message