falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Balu Vellanki (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-763) Support feed listing for CatalogStorage
Date Tue, 26 Jan 2016 23:30:40 GMT

    [ https://issues.apache.org/jira/browse/FALCON-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15118270#comment-15118270
] 

Balu Vellanki commented on FALCON-763:
--------------------------------------

[~Praveen] : Create a hcatalog feed entity. A sample entity would look like this,

{code}
<feed description="input" name="hcat-in" xmlns="uri:falcon:feed:0.1">
    <groups>input</groups>

    <frequency>minutes(5)</frequency>
    <timezone>UTC</timezone>
    <late-arrival cut-off="hours(1)"/>

    <clusters>
        <cluster name="hcat-local">
            <validity start="2016-01-01T00:00Z" end="2016-01-02T00:00Z"/>
            <retention limit="hours(24)" action="delete"/>
        </cluster>
    </clusters>

    <table uri="catalog:default:in_table#ds=${YEAR}-${MONTH}-${DAY}-${HOUR}-${MINUTE}"
/>

    <ACL owner="ambari-qa" group="users" permission="0x755"/>
    <schema location="/" provider="/"/>
</feed>
{code}

To test the API, create table in_table under default database with partition name as ds. Create
partitions "2016-01-01-00", "2016-01-01-05", "2016-01-01-10", "2016-01-01-20" and insert some
data into these partitions. 

Now use the following api.  https://falcon.apache.org/restapi/FeedInstanceListing.html , the
CLI would look something like this.

{code}
$FALCON_HOME/bin/falcon instance -type feed -name hcat-in -listing -start "2016-01-01T00:00Z"
-end "2016-01-01T00:20Z" 
{code}

The result should show you MISSING for "2016-01-01-15" and available for the rest of the instances
"2016-01-01-00", "2016-01-01-05", "2016-01-01-10", "2016-01-01-20" 

> Support feed listing for CatalogStorage
> ---------------------------------------
>
>                 Key: FALCON-763
>                 URL: https://issues.apache.org/jira/browse/FALCON-763
>             Project: Falcon
>          Issue Type: Sub-task
>          Components: client, feed
>    Affects Versions: 0.8
>            Reporter: Srikanth Sundarrajan
>            Assignee: Balu Vellanki
>         Attachments: FALCON-763-v1.patch, FALCON-763-v2.patch, FALCON-763.patch
>
>
> Support feed listing capability as outlined in FALCON-761 for FileSystemStorage



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message