crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-553) From.formattedFile may cause records to be dropped.
Date Tue, 28 Jul 2015 02:16:04 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Micah Whitacre updated CRUNCH-553:
----------------------------------
    Assignee: Josh Wills

> From.formattedFile may cause records to be dropped.
> ---------------------------------------------------
>
>                 Key: CRUNCH-553
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-553
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>    Affects Versions: 0.11.0, 0.12.0
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.13.0
>
>         Attachments: CRUNCH-553.patch
>
>
> From the mailing list, a user reported a bug in which they were using multiple instances
of From.formattedFile TableSources and were seeing records getting dropped at random from
different runs of their jobs. I created a simple test that replicated the behavior and found
the source of the problem in the planner: a confusion between a BaseInputTable and the BaseInputCollection
objects that does most of the work to actually configure the input table data that resulted
from BaseInputTable's equals() method not checking to see if an object was of its same class
before performing the comparison on the underlying BaseInputCollection instance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message