incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-70) Simplify Pipeline API
Date Fri, 21 Sep 2012 20:39:07 GMT


Josh Wills commented on CRUNCH-70:

When you write Avro records to a TextOutputFormat, they come out w/values like "AvroWrapper@hexcode",
so we had to force convert to a Writable type to be compatible with TextOutputFormat. I *think*
that got fixed in CRUNCH-52, so that code can probably go.

My default preference is for not removing API functions unless it's part of an overall redesign
(e.g., in the context of untangling the various components of Crunch core) that buys us something.
I think I could squint and make a case for removing readTextFile and writeTextFile anymore,
since doing read(From.textFile(...)) or write(To.textFile(...)) exist now, but I would argue
for keeping enableDebug. It's way too useful.
> Simplify Pipeline API
> ---------------------
>                 Key: CRUNCH-70
>                 URL:
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: CRUNCH-70-20120919.txt
> Today Pipeline interface has the following APIs which really belong to a utils class:
>  - readTextFile
>  - writeTextFile
>  - enableDebug
> The implementation of these APIs is the same in both the Pipeline-types present today
and are most likely going to be the same if ever we have one more impl.
> I propose we move these to a util/lib to make the core interface cleaner.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message