flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1389) Allow setting custom file extensions for files created by the FileOutputFormat
Date Fri, 23 Jan 2015 11:02:34 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289089#comment-14289089
] 

ASF GitHub Bot commented on FLINK-1389:
---------------------------------------

Github user rmetzger commented on the pull request:

    https://github.com/apache/flink/pull/301#issuecomment-71178244
  
    Hadoops `FileOutputFormat` has the following method
    ```java
     public static String getUniqueName(JobConf conf, String name) {
        int partition = conf.getInt(JobContext.TASK_PARTITION, -1);
        if (partition == -1) {
          throw new IllegalArgumentException(
            "This method can only be called from within a Job");
        }
    
        String taskType = (conf.getBoolean(JobContext.TASK_ISMAP, true)) ? "m" : "r";
    
        NumberFormat numberFormat = NumberFormat.getInstance();
        numberFormat.setMinimumIntegerDigits(5);
        numberFormat.setGroupingUsed(false);
    
        return name + "-" + taskType + "-" + numberFormat.format(partition);
      }
    ```
    So users have to overwrite the method if they want to use a custom filename.
    
    I'll remove the configuration-based approach.


> Allow setting custom file extensions for files created by the FileOutputFormat
> ------------------------------------------------------------------------------
>
>                 Key: FLINK-1389
>                 URL: https://issues.apache.org/jira/browse/FLINK-1389
>             Project: Flink
>          Issue Type: New Feature
>            Reporter: Robert Metzger
>            Assignee: Robert Metzger
>            Priority: Minor
>
> A user requested the ability to name avro files with the "avro" extension.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message