falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-102) Add integration tests for feed entity parser with table defined
Date Sun, 08 Sep 2013 05:21:51 GMT

    [ https://issues.apache.org/jira/browse/FALCON-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761197#comment-13761197

Venkatesh Seetharam commented on FALCON-102:

bq. Tests pass with FALCON-96
This jira blocks on FALCON-96 :-)

bq. If we pass the type & path, then the consumer can choose to do whatever they want
with it.
This is what I have refactored now. Will attach the patch soon. I did that as part of retention
path but...

bq. This is still a bit unclear to me. What is the significance of Q_START or CLOSE in uri.
When do they show up and why do we need to replace them. Are these specific to the thrift
urls that catalog needs ?
Sorry if I wasn't clear the last time.

Table URI would contain a dated partition key and a File System URI a dated data path. For
Table URI: catalog:clicksdb:clicks#ds=${YEAR}-${MONTH}-${DAY};region=us
FS URI: hdfs://localhost:8020/data/YYYY/feed1/mmHH/dd/MM/?{YEAR}-?{MONTH}-?{DAY}/more/?{YEAR}

The URI is parsed using java.net.URI instead of rolling my own parser. This is where URI cannot
parse: {noformat}"{" or "}"{noformat}. Hence, the need for the stupid ugly string replacement.
Makes sense? 

OTOH, I'm open to suggestions and can't think of anything better. I think all this becomes
clear with how this is being used in Retention.
> Add integration tests for feed entity parser with table defined
> ---------------------------------------------------------------
>                 Key: FALCON-102
>                 URL: https://issues.apache.org/jira/browse/FALCON-102
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.3
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>         Attachments: FALCON-102.patch
> Having issues to get webhcat up and running. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message