crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-553) From.formattedFile may cause records to be dropped.
Date Tue, 28 Jul 2015 01:49:04 GMT
Josh Wills created CRUNCH-553:
---------------------------------

             Summary: From.formattedFile may cause records to be dropped.
                 Key: CRUNCH-553
                 URL: https://issues.apache.org/jira/browse/CRUNCH-553
             Project: Crunch
          Issue Type: Bug
          Components: IO
    Affects Versions: 0.12.0, 0.11.0
            Reporter: Josh Wills
             Fix For: 0.13.0


>From the mailing list, a user reported a bug in which they were using multiple instances
of From.formattedFile TableSources and were seeing records getting dropped at random from
different runs of their jobs. I created a simple test that replicated the behavior and found
the source of the problem in the planner: a confusion between a BaseInputTable and the BaseInputCollection
objects that does most of the work to actually configure the input table data that resulted
from BaseInputTable's equals() method not checking to see if an object was of its same class
before performing the comparison on the underlying BaseInputCollection instance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message