falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-102) Add integration tests for feed entity parser with table defined
Date Sun, 08 Sep 2013 05:21:51 GMT

    [ https://issues.apache.org/jira/browse/FALCON-102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13761197#comment-13761197
] 

Venkatesh Seetharam commented on FALCON-102:
--------------------------------------------

bq. Tests pass with FALCON-96
This jira blocks on FALCON-96 :-)

bq. If we pass the type & path, then the consumer can choose to do whatever they want
with it.
This is what I have refactored now. Will attach the patch soon. I did that as part of retention
path but...

bq. This is still a bit unclear to me. What is the significance of Q_START or CLOSE in uri.
When do they show up and why do we need to replace them. Are these specific to the thrift
urls that catalog needs ?
Sorry if I wasn't clear the last time.

Table URI would contain a dated partition key and a File System URI a dated data path. For
Ex:
{code}
Table URI: catalog:clicksdb:clicks#ds=${YEAR}-${MONTH}-${DAY};region=us
FS URI: hdfs://localhost:8020/data/YYYY/feed1/mmHH/dd/MM/?{YEAR}-?{MONTH}-?{DAY}/more/?{YEAR}
{code}

The URI is parsed using java.net.URI instead of rolling my own parser. This is where URI cannot
parse: {noformat}"{" or "}"{noformat}. Hence, the need for the stupid ugly string replacement.
Makes sense? 

OTOH, I'm open to suggestions and can't think of anything better. I think all this becomes
clear with how this is being used in Retention.
                
> Add integration tests for feed entity parser with table defined
> ---------------------------------------------------------------
>
>                 Key: FALCON-102
>                 URL: https://issues.apache.org/jira/browse/FALCON-102
>             Project: Falcon
>          Issue Type: Sub-task
>    Affects Versions: 0.3
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>         Attachments: FALCON-102.patch
>
>
> Having issues to get webhcat up and running. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message