crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mārtiņš Kalvāns (JIRA) <>
Subject [jira] [Created] (CRUNCH-502) OutputFormat has inconsistent context state in interface functions
Date Mon, 02 Mar 2015 16:00:08 GMT
Mārtiņš Kalvāns created CRUNCH-502:

             Summary: OutputFormat has inconsistent context state in interface functions
                 Key: CRUNCH-502
             Project: Crunch
          Issue Type: Bug
          Components: IO
    Affects Versions: 0.12.0
            Reporter: Mārtiņš Kalvāns

I created example project to demonstrate problematic behaviour:

1. FormatBundle config is not populated to Configuration in checkOutputSpecs:
15/03/02 15:40:24 INFO bugreport.MyOutputFormat: my.config.key (checkOutputSpecs): null
15/03/02 15:40:24 ERROR bugreport.MyOutputFormat: Wrong my.config.key value in checkOutputSpecs!

2. TaskAttemptContext. getTaskAttemptID().toString() is different in getRecordWriter and getOutputCommitter:
2015-03-02 15:40:38,960 INFO [main] com.spotify.crunch.bugreport.MyOutputFormat: TaskAttemptID
(getOutputCommitter): attempt_1422406067005_0121_m_000000_0
2015-03-02 15:40:39,789 INFO [main] com.spotify.crunch.bugreport.MyOutputFormat: TaskAttemptID
(getRecordWriter): attempt_1422406067005_out0_0121_m_000000_0

This message was sent by Atlassian JIRA

View raw message