sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-470) Create tests for blob support for Avro import
Date Wed, 04 Apr 2012 02:16:23 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245968#comment-13245968
] 

jiraposter@reviews.apache.org commented on SQOOP-470:
-----------------------------------------------------



bq.  On 2012-03-29 22:04:43, Bilung Lee wrote:
bq.  > Look good overall.  It would be better if you could also add some to test out the
area of compression.
bq.  
bq.  Cheolsoo Park wrote:
bq.      Thank you for reviewing my patch. I will add test cases for compression as per your
request.

Hi Bilung, in terms of compression, I think that there are two aspects that we can test: 1)
Avro compression and 2) LobFile compression.

As for 1, I added two more test cases.

As for 2, even though LobFile supports compression, using LobFile with compression codec requires
quite a few changes to various places. Currently, the LobFile writer that is created by LargeObjectLoader
is constructed with no codec parameter as follows:

  public static Writer create(Path p, Configuration conf, boolean isCharData)
      throws IOException {
    return org.apache.sqoop.io.LobFile.create(p, conf, isCharData, null); <-- null is for
codec.
  }

To support LobFile compression, we will have to extend LargeObjectLoader first. I think that
it is better to defer adding tests for LobFile compression until we update LargeObjectLoader.

Please let me know what you think.


- Cheolsoo


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4527/#review6548
-----------------------------------------------------------


On 2012-04-04 01:55:06, Cheolsoo Park wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4527/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2012-04-04 01:55:06)
bq.  
bq.  
bq.  Review request for Sqoop, Arvind Prabhakar and Bilung Lee.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Add Avro blob import tests as part of thrid party test suite since Hsqldb does not support
BLOB/CLOB type.
bq.  
bq.  The changes include:
bq.  
bq.  - abstract LobAvroImportTestCase class
bq.    1) testBlobAvroImportInline: test inline import
bq.    2) testBlobAvroImportExternal: test external file import
bq.    3) testBlobAvroImportMultiCols: test multiple columns import
bq.  
bq.  - OracleLobAvroImportTest class
bq.  - MySQLLobAvroImportTest class
bq.  
bq.  
bq.  This addresses bug SQOOP-470.
bq.      https://issues.apache.org/jira/browse/SQOOP-470
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    /src/test/com/cloudera/sqoop/manager/MySQLLobAvroImportTest.java PRE-CREATION 
bq.    /src/test/com/cloudera/sqoop/manager/OracleLobAvroImportTest.java PRE-CREATION 
bq.    /src/test/com/cloudera/sqoop/testutil/LobAvroImportTestCase.java PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/4527/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Tested with Oracle XE 10g and MySQL 5.1
bq.  
bq.  Ran ant test
bq.  Ran ant test -Dthirdparty=true
bq.  Ran ant checkstyle
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Cheolsoo
bq.  
bq.


                
> Create tests for blob support for Avro import
> ---------------------------------------------
>
>                 Key: SQOOP-470
>                 URL: https://issues.apache.org/jira/browse/SQOOP-470
>             Project: Sqoop
>          Issue Type: Sub-task
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>
> Create tests for blob support for Avro import

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message