Return-Path: 
 <crunch-dev-return-1568-apmail-incubator-crunch-dev-archive=incubator.apache.org@incubator.apache.org>
X-Original-To: apmail-incubator-crunch-dev-archive@minotaur.apache.org
Delivered-To: apmail-incubator-crunch-dev-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 7C462D876
	for <apmail-incubator-crunch-dev-archive@minotaur.apache.org>;
 Sat,  1 Dec 2012 11:42:02 +0000 (UTC)
Received: (qmail 53766 invoked by uid 500); 1 Dec 2012 11:42:02 -0000
Delivered-To: apmail-incubator-crunch-dev-archive@incubator.apache.org
Received: (qmail 53038 invoked by uid 500); 1 Dec 2012 11:42:00 -0000
Mailing-List: contact crunch-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:crunch-dev-help@incubator.apache.org>
List-Unsubscribe: <mailto:crunch-dev-unsubscribe@incubator.apache.org>
List-Post: <mailto:crunch-dev@incubator.apache.org>
List-Id: <crunch-dev.incubator.apache.org>
Reply-To: crunch-dev@incubator.apache.org
Delivered-To: mailing list crunch-dev@incubator.apache.org
Received: (qmail 52938 invoked by uid 99); 1 Dec 2012 11:41:58 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Dec 2012 11:41:58 +0000
Date: Sat, 1 Dec 2012 11:41:58 +0000 (UTC)
From: "Matthias Friedrich (JIRA)" <jira@apache.org>
To: crunch-dev@incubator.apache.org
Message-ID: <1434515938.48837.1354362118233.JavaMail.jiratomcat@arcas>
In-Reply-To: <1201490380.49563.1350359223177.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (CRUNCH-97) Add helpers for parsing
 PCollection<String> instances
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


     [ https://issues.apache.org/jira/browse/CRUNCH-97?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Friedrich updated CRUNCH-97:
-------------------------------------

    Attachment: CRUNCH-97-Tokenizer-v1.patch

Here's how it would look with Tokenizers. This is just a demo, I would clean it up if we decided to go this way. I'd also have to design a nice call syntax for Parse with tokenizers.

What I don't like about it: It makes nested collections harder to implement (see the ugly hack for Extractors.xcollect()). We also have to pass column positions to ExtractorS - that's the price we pay for flexibility.
                
> Add helpers for parsing PCollection<String> instances
> -----------------------------------------------------
>
>                 Key: CRUNCH-97
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-97
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.5.0
>
>         Attachments: CRUNCH-97.patch, CRUNCH-97-take2.patch, CRUNCH-97-Tokenizer-v1.patch, CRUNCH-97v3.patch
>
>
> We should make it a bit easier to parse delimited text files into specific data types (e.g., ints, floats, etc.) or combinations of types-- e.g., pairs of strings and ints, a Tuple3 of booleans, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira