crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Created] (CRUNCH-448) Smart-ish case class serialization for Scrunch
Date Mon, 21 Jul 2014 16:44:38 GMT
Josh Wills created CRUNCH-448:

             Summary: Smart-ish case class serialization for Scrunch
                 Key: CRUNCH-448
             Project: Crunch
          Issue Type: Improvement
          Components: Scrunch
            Reporter: Josh Wills
         Attachments: CRUNCH-448.patch

I took advantage of some of the new reflection features in Scala 2.10 to make automatic serialization
of case classes much cooler, viz.,

1) We can automatically map any case class to a TupleN and properly infer PTypes for Scala
types (options, lists, etc.), just as we do for regular PCollections.
2) For Avro serialization, we will also capture the names of the case class' fields and map
them to the names of the fields in the Avro record we generate, which makes debugging intermediate
outputs much easier.

This message was sent by Atlassian JIRA

View raw message