crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Whiting (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-454) Replace use of Scala Iterable with TraversableOnce where SingleUseIterable is the concrete type
Date Mon, 04 Aug 2014 14:29:12 GMT
David Whiting created CRUNCH-454:
------------------------------------

             Summary: Replace use of Scala Iterable with TraversableOnce where SingleUseIterable
is the concrete type
                 Key: CRUNCH-454
                 URL: https://issues.apache.org/jira/browse/CRUNCH-454
             Project: Crunch
          Issue Type: Improvement
          Components: Scrunch
            Reporter: David Whiting
            Priority: Minor


The Scrunch version of combine accepts a function Iterable[V] => V . This causes a lot
of unexpected behaviour because the iterable that is wrapped is actually a SingleUseIterable,
and much of Scala's collection function implementations actually try and access the underlying
iterator multiple times if they know that it's possible. 

The fix implemented here is to call iterator() on the underlying SingleUseIterable and expose
it instead as a TraversableOnce.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message