carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-276) Add trim option
Date Wed, 12 Oct 2016 09:31:20 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568190#comment-15568190
] 

ASF GitHub Bot commented on CARBONDATA-276:
-------------------------------------------

Github user sujith71955 commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/200#discussion_r82968468
  
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/csvreaderstep/UnivocityCsvParser.java
---
    @@ -102,8 +102,8 @@ public void initialize() throws IOException {
         parserSettings.setMaxColumns(
             getMaxColumnsForParsing(csvParserVo.getNumberOfColumns(), csvParserVo.getMaxColumns()));
         parserSettings.setNullValue("");
    -    parserSettings.setIgnoreLeadingWhitespaces(false);
    -    parserSettings.setIgnoreTrailingWhitespaces(false);
    +    parserSettings.setIgnoreLeadingWhitespaces(csvParserVo.getTrim());
    --- End diff --
    
    Guys i think if while data loading we are reading from configuration inorder to trim it
or not same we need to do while doing filter also, based on configuration value decide.
    ex:  while loading i enabled trim property, so system will trim and load the data, now
in filter query also if user provides while space then it needs to be trimmed while creating
filter model. This will provide more system consistentency. if user enable trim then we wont
trim it while loading and also while querying.


> Add trim option
> ---------------
>
>                 Key: CARBONDATA-276
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-276
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Lionx
>            Assignee: Lionx
>            Priority: Minor
>
> Fix a bug and add trim option.
> Bug: When string is contains LeadingWhiteSpace or TrailingWhiteSpace, query result is
null. This is because the dictionary ignore the LeadingWhiteSpace and TrailingWhiteSpace and
the csvInput dose not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message