kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3552) Data Source SDK to ingest data from different JDBC sources
Date Tue, 11 Sep 2018 14:56:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610748#comment-16610748
] 

Billy Liu commented on KYLIN-3552:
----------------------------------

+1. Besides the SDK and API, could you consider one full-featured data source adapter for
the early adoption.  

> Data Source SDK to ingest data from different JDBC sources
> ----------------------------------------------------------
>
>                 Key: KYLIN-3552
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3552
>             Project: Kylin
>          Issue Type: New Feature
>          Components: Query Engine, RDBMS Source
>            Reporter: youcheng.zhang
>            Assignee: youcheng.zhang
>            Priority: Major
>             Fix For: v2.6.0
>
>
> Kylin already supports JDBC source, but it takes much efforts to develop an implementation
to a new source engine, like supporting metadata sync, cube build and query pushdown. It’s
mainly because the SQL dialects and jdbc implementations between source engines are quite
different.
> We propose to develop a data source sdk, which provides APIs to help developers handle
these dialect differences and easily implement an adaptor for one source engine. With this
SDK, users can achieve followings from a JDBC source:
>  # synchronize metadata and data from jdbc source.
>  # Build cube from jdbc source.
>  # query pushdown to jdbc source engine when cube is unmatched. 
> The SDK will define some interfaces of adaptors. To implement a new source, developers
only need to implement an adaptor for the source.
> To keep compatibly, we will keep the existing JDBC source module, and implement a new
JDBC source module as the caller of the SDK. Besides, we will also provide a pushdown implementation
class as the caller of SDK for pushdown scenarios.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message