flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Oracle 11g number serialization: classcast problem
Date Wed, 23 Mar 2016 09:41:41 GMT
On 23.03.2016 10:38, Chesnay Schepler wrote:
> On 23.03.2016 10:04, Stefano Bortoli wrote:
>> I had a look at the JDBC input format, and it does indeed interpret 
>> BIGDECIMAL and NUMERIC values as double.
> This sounds more like a bug actually. Feel free to open a JIRA for this.
Actually, this was done so that every field is a primitive or String.
>> The status of the JDBCInputFormat is not adequate for real world use 
>> case, as for example does not deal with NULL values.
> This was already reported in FLINK-3471. To clarify, for NULL fields 
> the format fails only if the type is either DECIMAL, NUMERIC, DATE, 
> TIME, TIMESTAMP, or SQLXML. Other types should default to 0, empty 
> string or false; which actually isn't intended behavior, but caused by 
> JDBC itself.
> Defaulting to some value seems the only way to deal with this issue, 
> since we can't store null in a Tuple.
> I wasn't sure what value DATE, TIME, TIMESTAMP and SQLXML should 
> default to, as such i didn't change them yet. I also just dislike the 
> fact that a straight copy from A to B will not produce the same table.
>> However, with little effort we fixed few stuff and now we are getting 
>> to something usable. We are actually trying to do something a-la 
>> sqoop, therefore given a boundary query, we create the splits, and 
>> then assign it to the input format to read the database with 
>> configurable parallelism. We are still working on it. If we get to 
>> something stable and working, we'll gladly share it.
>> saluti,
>> Stefano
>> 2016-03-22 15:46 GMT+01:00 Chesnay Schepler <chesnay@apache.org 
>> <mailto:chesnay@apache.org>>:
>>     The JDBC formats don't make any assumption as to what DB backend
>>     is used.
>>     A JDBC float in general is returned as a double, since that was
>>     the recommended mapping i found when i wrote the formats.
>>     Is the INT returned as a double as well?
>>     Note: The (runtime) output type is in no way connected to the
>>     TypeInfo you pass when constructing the format.
>>     On 21.03.2016 14 <tel:21.03.2016%2014>:16, Stefano Bortoli wrote:
>>         Hi squirrels,
>>         I working on a flink job connecting to a Oracle DB. I started
>>         from the JDBC example for Derby, and used the TupleTypeInfo
>>         to configure the fields of the tuple as it is read.
>>         The record of the example has 2 INT, 1 FLOAT and 2 VARCHAR.
>>         Apparently, using Oracle, all the numbers are read as Double,
>>         causing a ClassCast exception. Of course I can fix it by
>>         changing the TupleTypeInfo, but I wonder whether there is
>>         some assumption for Oracle and Numbers.
>>         Thanks a lot for your support!
>>         saluti,
>>         Stefano

View raw message