apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chinmay Kolhatkar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2293) Apex SQL: Add sql line cli to sql support of apex
Date Fri, 11 Nov 2016 12:54:58 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657010#comment-15657010

Chinmay Kolhatkar commented on APEXMALHAR-2293:

[~thw], [~davidyan] Here is my understanding about this jira. This jira has following asks:
1. Make the endpoints configurable from outside... Something like a JSon configuration which
can be passed to Endpoint interface and individual endpoints can read and configure it.
2. Have a registry of some kind which is accessible externally for registering different types
of endpoints.
3. Use above concept in sql cli.

I propose following changes:
1. Add a method called "getSchema" which will return the URI for given endpoint.
1. Create a EndpointRegistry class which has a map of URI -> Endoint. URI can be like kafka,
file, jdbc etc...
2. In the static block of this class there will be used ServiceLoader of java to find all
the classes in classpath that implements Endpoint interface. Once found these classes can
be added to the Map provided.
3. This way if anyone wants to add an Endpoint externally, all one has to do is define the
class and add to classpath.
4. Alternativey EndpointRegistry can have method like reload which can reload the schema.
5. A new method should be added to Endpoint which takes a String a property for that class.
This string can then be interpreted by Endpoint to configure the property.
6. Similar can be done for MessageFormats as well.
7. SQL cli can then use above APIs to dynamically define the table. (Example of sql cli is
available with storm-sql)

Advantage of this:
1. Endpoints could get automatically registered with malhar-sql.
2. One of create custom endpoints in own project and just use it with malhar-sql.
3. Endpoints can completely configured externally.

Please share your opinion.


> Apex SQL: Add sql line cli to sql support of apex
> -------------------------------------------------
>                 Key: APEXMALHAR-2293
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2293
>             Project: Apache Apex Malhar
>          Issue Type: New Feature
>          Components: sql
>            Reporter: Chinmay Kolhatkar
>            Assignee: Chinmay Kolhatkar
> Add cli like support to malhar-sql.
> The support can be done similar to: https://github.com/apache/storm/blob/master/external/sql/README.md
> It should support following type of statements:
> 1. CREATE EXTERNAL TABLE with kafka, file, jdbc specifications
> 5. INSERT and SELECT statements

This message was sent by Atlassian JIRA

View raw message