Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0E1C7200C47 for ; Thu, 30 Mar 2017 08:26:56 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 0CA61160B8B; Thu, 30 Mar 2017 06:26:56 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 533F3160B78 for ; Thu, 30 Mar 2017 08:26:55 +0200 (CEST) Received: (qmail 87759 invoked by uid 500); 30 Mar 2017 06:26:54 -0000 Mailing-List: contact issues-help@carbondata.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@carbondata.incubator.apache.org Delivered-To: mailing list issues@carbondata.incubator.apache.org Received: (qmail 87750 invoked by uid 99); 30 Mar 2017 06:26:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Mar 2017 06:26:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E5677C0145 for ; Thu, 30 Mar 2017 06:26:53 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id NIB8G7TayjiA for ; Thu, 30 Mar 2017 06:26:52 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id AC42F60E11 for ; Thu, 30 Mar 2017 06:26:42 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id EE19FE012E for ; Thu, 30 Mar 2017 06:26:41 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 80D0F24062 for ; Thu, 30 Mar 2017 06:26:41 +0000 (UTC) Date: Thu, 30 Mar 2017 06:26:41 +0000 (UTC) From: "tianli (JIRA)" To: issues@carbondata.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (CARBONDATA-833) load data from dataframe,generater data row may be error when delimiterLevel1 or delimiterLevel2 is special character MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 30 Mar 2017 06:26:56 -0000 tianli created CARBONDATA-833: --------------------------------- Summary: load data from dataframe,generater data row may be error when delimiterLevel1 or delimiterLevel2 is special character Key: CARBONDATA-833 URL: https://issues.apache.org/jira/browse/CARBONDATA-833 Project: CarbonData Issue Type: Bug Components: spark-integration Affects Versions: 1.0.0-incubating, 1.1.0-incubating Reporter: tianli Assignee: tianli Fix For: 1.1.0-incubating, 1.0.0-incubating load data from dataframe,generater data row may be error by delimiterLevel1 or delimiterLevel2 is special character because delimiterLevel1 and delimiterLevel2 when carbonLoadModel is create by CarbonUtil.delimiterConverter(), CarbonScalaUtil.getString direct use carbonLoadModel.getComplexDelimiterLevel1 and carbonLoadModel.getComplexDelimiterLevel2 val delimiter = if (level == 1) { delimiterLevel1 } else { delimiterLevel2 } val builder = new StringBuilder() s.foreach { x => builder.append(getString(x, serializationNullFormat, delimiterLevel1, delimiterLevel2, timeStampFormat, dateFormat, level + 1)).append(delimiter) } make primitive data added a more char \ when datatype is complex -- This message was sent by Atlassian JIRA (v6.3.15#6346)