sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1616) Sqoop data type to Kite Avro data type convert
Date Mon, 01 Dec 2014 23:24:13 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230672#comment-14230672

Jarek Jarcec Cecho commented on SQOOP-1616:

This is slightly off-topic, but I wanted to add my thoughts. The original design that drove
IDF is to decouple the connector code and it's internal data representation. We worked with
assumption that there will be exactly one internal representation that each connector will
use (in memory). One connector can of course support multiple output formats, but we did not
intended to use the IDF infrastructure to do such conversions and hence the single {{getIDF()}}
call in {{SqoopConnector}}.

Having said that I believe that Kite internally uses Avro objects to represents data in memory
and hence using Avro IDF might make sense.

> Sqoop data type to Kite Avro data type convert
> ----------------------------------------------
>                 Key: SQOOP-1616
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1616
>             Project: Sqoop
>          Issue Type: Sub-task
>          Components: connectors
>            Reporter: Qian Xu
>            Assignee: Qian Xu
>            Priority: Minor
>             Fix For: 1.99.5
> Should add more data type convert support

This message was sent by Atlassian JIRA

View raw message