sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bilung Lee" <bleeapa...@gmail.com>
Subject Re: Review Request: SQOOP-470 Create tests for blob support for Avro import
Date Wed, 04 Apr 2012 20:44:34 GMT


> On 2012-03-29 22:04:43, Bilung Lee wrote:
> > Look good overall.  It would be better if you could also add some to test out the
area of compression.
> 
> Cheolsoo Park wrote:
>     Thank you for reviewing my patch. I will add test cases for compression as per your
request.
> 
> Cheolsoo Park wrote:
>     Hi Bilung, in terms of compression, I think that there are two aspects that we can
test: 1) Avro compression and 2) LobFile compression.
>     
>     As for 1, I added two more test cases.
>     
>     As for 2, even though LobFile supports compression, using LobFile with compression
codec requires quite a few changes to various places. Currently, the LobFile writer that is
created by LargeObjectLoader is constructed with no codec parameter as follows:
>     
>       public static Writer create(Path p, Configuration conf, boolean isCharData)
>           throws IOException {
>         return org.apache.sqoop.io.LobFile.create(p, conf, isCharData, null); <--
null is for codec.
>       }
>     
>     To support LobFile compression, we will have to extend LargeObjectLoader first. I
think that it is better to defer adding tests for LobFile compression until we update LargeObjectLoader.
>     
>     Please let me know what you think.

Thanks for adding the tests!  Yes, tests for Avro compression would be good enough for current
implementation.  The other can come later as needed.


- Bilung


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4527/#review6548
-----------------------------------------------------------


On 2012-04-04 01:55:06, Cheolsoo Park wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4527/
> -----------------------------------------------------------
> 
> (Updated 2012-04-04 01:55:06)
> 
> 
> Review request for Sqoop, Arvind Prabhakar and Bilung Lee.
> 
> 
> Summary
> -------
> 
> Add Avro blob import tests as part of thrid party test suite since Hsqldb does not support
BLOB/CLOB type.
> 
> The changes include:
> 
> - abstract LobAvroImportTestCase class
>   1) testBlobAvroImportInline: test inline import
>   2) testBlobAvroImportExternal: test external file import
>   3) testBlobAvroImportMultiCols: test multiple columns import
> 
> - OracleLobAvroImportTest class
> - MySQLLobAvroImportTest class
> 
> 
> This addresses bug SQOOP-470.
>     https://issues.apache.org/jira/browse/SQOOP-470
> 
> 
> Diffs
> -----
> 
>   /src/test/com/cloudera/sqoop/manager/MySQLLobAvroImportTest.java PRE-CREATION 
>   /src/test/com/cloudera/sqoop/manager/OracleLobAvroImportTest.java PRE-CREATION 
>   /src/test/com/cloudera/sqoop/testutil/LobAvroImportTestCase.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/4527/diff
> 
> 
> Testing
> -------
> 
> Tested with Oracle XE 10g and MySQL 5.1
> 
> Ran ant test
> Ran ant test -Dthirdparty=true
> Ran ant checkstyle
> 
> 
> Thanks,
> 
> Cheolsoo
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message