sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 25180: SQOOP-1378 - Sqoop2: From/To: Refactor schema
Date Tue, 02 Sep 2014 18:49:33 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25180/#review52047
-----------------------------------------------------------


Good start! A few comments...

Should the IDF persist a TO schema? It seems a little weird since the IDF is carrying one
representation of data (CSV format for now). I'm wondering if IDF should have one schema that
represents the data its carrying. Then, the TO connector or connector schemas could be responsible
for coercing data from the IDF to the connector schema.


common/src/main/java/org/apache/sqoop/schema/Schema.java
<https://reviews.apache.org/r/25180/#comment90778>

    Pull this out to the framework level and persist separately from Schema itself? This should
help choose the matching strategy between connectors?



connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsInitializer.java
<https://reviews.apache.org/r/25180/#comment90792>

    Does the class and method signatures need to change?



connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
<https://reviews.apache.org/r/25180/#comment90793>

    See general comment


- Abraham Elmahrek


On Aug. 29, 2014, 3:01 a.m., Gwen Shapira wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25180/
> -----------------------------------------------------------
> 
> (Updated Aug. 29, 2014, 3:01 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Repository: sqoop-SQOOP-1367
> 
> 
> Description
> -------
> 
> SQOOP-1378 - Sqoop2: From/To: Refactor schema
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/sqoop/job/etl/ExtractorContext.java af03f0a 
>   common/src/main/java/org/apache/sqoop/schema/Schema.java bbebab8 
>   common/src/main/java/org/apache/sqoop/schema/SchemaError.java 7c8c61e 
>   common/src/main/java/org/apache/sqoop/schema/SchemaMatchOption.java PRE-CREATION 
>   connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcFromInitializer.java
fbbe506 
>   connector/connector-generic-jdbc/src/test/java/org/apache/sqoop/connector/jdbc/TestExtractor.java
2b1dec2 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsInitializer.java
d2d12a8 
>   connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
1e8ab52 
>   connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/IntermediateDataFormat.java
91b594e 
>   connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/matcher/AbstractMatcher.java
PRE-CREATION 
>   connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/matcher/LocationMatcher.java
PRE-CREATION 
>   connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/matcher/NameMatcher.java
PRE-CREATION 
>   execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 1d60ba3

>   execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java
e457cff 
> 
> Diff: https://reviews.apache.org/r/25180/diff/
> 
> 
> Testing
> -------
> 
> Nope!
> 
> 
> Thanks,
> 
> Gwen Shapira
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message