Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 285D3200D2B for ; Thu, 2 Nov 2017 09:27:05 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 26CB1160BE5; Thu, 2 Nov 2017 08:27:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 45B301609EE for ; Thu, 2 Nov 2017 09:27:04 +0100 (CET) Received: (qmail 78880 invoked by uid 500); 2 Nov 2017 08:27:03 -0000 Mailing-List: contact issues-help@carbondata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.apache.org Delivered-To: mailing list issues@carbondata.apache.org Received: (qmail 78871 invoked by uid 99); 2 Nov 2017 08:27:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Nov 2017 08:27:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id A35631A11B4 for ; Thu, 2 Nov 2017 08:27:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id gtP3BjH3OeaU for ; Thu, 2 Nov 2017 08:27:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id CC1485F566 for ; Thu, 2 Nov 2017 08:27:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 52606E061C for ; Thu, 2 Nov 2017 08:27:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0AE6823F05 for ; Thu, 2 Nov 2017 08:27:00 +0000 (UTC) Date: Thu, 2 Nov 2017 08:27:00 +0000 (UTC) From: "Vandana Yadav (JIRA)" To: issues@carbondata.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CARBONDATA-1664) Abnormal behavior of timestamp data type in carbondata MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 02 Nov 2017 08:27:05 -0000 [ https://issues.apache.org/jira/browse/CARBONDATA-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Yadav updated CARBONDATA-1664: -------------------------------------- Description: Abnormal behavior of timestamp data type in carbondata Steps to Reproduce: 1) Create Table: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB") 2)Load Data: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table uniqdata OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') 3) Execute Query: a) select DOB from UNIQDATA where DOB ='1970-01-01 10:00:03.0' or DOB = '1970-01-04 01:00:03.0'; output: +------------------------+--+ | DOB | +------------------------+--+ | 1970-01-01 10:00:03.0 | | 1970-01-04 01:00:03.0 | +------------------------+--+ b) select DOB from UNIQDATA where DOB in ('1970-01-01 10:00:03.0','1970-01-04 01:00:03.0'); output: +------+--+ | DOB | +------+--+ +------+--+ c)select DOB from UNIQDATA where DOB in (cast('1970-01-01 10:00:03.0' as timestamp),cast('1970-01-04 01:00:03.0' as timestamp)); output: +------------------------+--+ | DOB | +------------------------+--+ | 1970-01-01 10:00:03.0 | | 1970-01-04 01:00:03.0 | +------------------------+--+ Abnormality of timestamp datatype: In the select query (a) it fetch the records containing DOB 1970-01-01 10:00:03.0 and 1970-01-04 01:00:03.0 but for query (b) while using IN operator it shows no data and again in the same query when we cast it to timestamp as in query (c) it displays result. There should be a strict type checking for timestamp values. was: Abnormal behavior of timestamp data type in carbondata Steps to Reproduce: 1) Create Table: CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB") 2)Load Data: LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table uniqdata OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') 3) Execute Query: a) select DOB from UNIQDATA where DOB ='1970-01-01 10:00:03.0' or DOB = '1970-01-04 01:00:03.0'; output: +------------------------+--+ | DOB | +------------------------+--+ | 1970-01-01 10:00:03.0 | | 1970-01-04 01:00:03.0 | +------------------------+--+ b) select DOB from UNIQDATA where DOB in ('1970-01-01 10:00:03.0','1970-01-04 01:00:03.0'); +------+--+ | DOB | +------+--+ +------+--+ c)select DOB from UNIQDATA where DOB in (cast('1970-01-01 10:00:03.0' as timestamp),cast('1970-01-04 01:00:03.0' as timestamp)); +------------------------+--+ | DOB | +------------------------+--+ | 1970-01-01 10:00:03.0 | | 1970-01-04 01:00:03.0 | +------------------------+--+ Abnormality of timestamp datatype: In the select query (a) it fetch the records containing DOB 1970-01-01 10:00:03.0 and 1970-01-04 01:00:03.0 but for query (b) while using IN operator it shows no data and again in the same query when we cast it to timestamp as in query (c) it displays result. There should be a strict type checking for timestamp values. > Abnormal behavior of timestamp data type in carbondata > ------------------------------------------------------ > > Key: CARBONDATA-1664 > URL: https://issues.apache.org/jira/browse/CARBONDATA-1664 > Project: CarbonData > Issue Type: Bug > Components: data-query > Affects Versions: 1.3.0 > Environment: spark 2.1 > Reporter: Vandana Yadav > Attachments: 2000_UniqData.csv > > > Abnormal behavior of timestamp data type in carbondata > Steps to Reproduce: > 1) Create Table: > CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES ("TABLE_BLOCKSIZE"= "256 MB") > 2)Load Data: > LOAD DATA INPATH 'HDFS_URL/BabuStore/Data/uniqdata/2000_UniqData.csv' into table uniqdata OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1') > 3) Execute Query: > a) select DOB from UNIQDATA where DOB ='1970-01-01 10:00:03.0' or DOB = '1970-01-04 01:00:03.0'; > output: > +------------------------+--+ > | DOB | > +------------------------+--+ > | 1970-01-01 10:00:03.0 | > | 1970-01-04 01:00:03.0 | > +------------------------+--+ > b) select DOB from UNIQDATA where DOB in ('1970-01-01 10:00:03.0','1970-01-04 01:00:03.0'); > output: > +------+--+ > | DOB | > +------+--+ > +------+--+ > c)select DOB from UNIQDATA where DOB in (cast('1970-01-01 10:00:03.0' as timestamp),cast('1970-01-04 01:00:03.0' as timestamp)); > output: > +------------------------+--+ > | DOB | > +------------------------+--+ > | 1970-01-01 10:00:03.0 | > | 1970-01-04 01:00:03.0 | > +------------------------+--+ > Abnormality of timestamp datatype: > In the select query (a) it fetch the records containing DOB 1970-01-01 10:00:03.0 and 1970-01-04 01:00:03.0 but for query (b) while using IN operator it shows no data and again in the same query when we cast it to timestamp as in query (c) it displays result. > There should be a strict type checking for timestamp values. -- This message was sent by Atlassian JIRA (v6.4.14#64029)