carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-37) Support Date/Time format for Timestamp columns to be defined at column level
Date Wed, 26 Oct 2016 02:46:00 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15607199#comment-15607199
] 

ASF GitHub Bot commented on CARBONDATA-37:
------------------------------------------

Github user QiangCai commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/219#discussion_r85039488
  
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/surrogatekeysgenerator/csvbased/CarbonCSVBasedSeqGenStep.java
---
    @@ -470,6 +474,36 @@ public boolean processRow(StepMetaInterface smi, StepDataInterface
sdi) throws K
                   break;
               }
             }
    +        HashMap<String, String> dateformatsHashMap = new HashMap<String, String>();
    +        if (meta.dateFormat != null) {
    +          String[] dateformats = meta.dateFormat.split(",");
    +          for (String dateFormat:dateformats) {
    +            String[] dateFormatSplits = dateFormat.split(":", 2);
    +            dateformatsHashMap.put(dateFormatSplits[0],dateFormatSplits[1]);
    +            // TODO  verify the dateFormatSplits is valid or not
    +          }
    +        }
    +        directDictionaryGenerators =
    +                new DirectDictionaryGenerator[meta.getDimensionColumnIds().length];
    +        for (int i = 0; i < meta.getDimensionColumnIds().length; i++) {
    +          ColumnSchemaDetails columnSchemaDetails = columnSchemaDetailsWrapper.get(
    +                  meta.getDimensionColumnIds()[i]);
    +          if (columnSchemaDetails.isDirectDictionary()) {
    +            if (dateformatsHashMap.containsKey(columnSchemaDetails.getColumnName()))
{
    +              directDictionaryGenerators[i] =
    +                      DirectDictionaryKeyGeneratorFactory.getDirectDictionaryGenerator(
    +                              columnSchemaDetails.getColumnType(),
    +                              dateformatsHashMap.get(columnSchemaDetails.getColumnName()));
    +            } else {
    +              String dateFormat = CarbonProperties.getInstance()
    +                      .getProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT,
    +                              CarbonCommonConstants.CARBON_TIMESTAMP_DEFAULT_FORMAT);
    +              directDictionaryGenerators[i] =
    +                      DirectDictionaryKeyGeneratorFactory.getDirectDictionaryGenerator(
    +                              columnSchemaDetails.getColumnType(), dateFormat);
    --- End diff --
    
    1. move out CarbonProperties.getInstance().getProperty  from for loop
    2. for defaut  dataformat, use  method getDirectDictionaryGenerator(DataType dataType)


> Support Date/Time format for Timestamp columns to be defined at column level
> ----------------------------------------------------------------------------
>
>                 Key: CARBONDATA-37
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-37
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: Vimal Das Kammath
>            Assignee: Lionx
>
> Carbon support defining the Date/Time format. But the configuration for the same is present
in carbon.properties and hence is global for all tables.
> This global configuration for timestamp format cannot support scenarios where different
tables or different Timestamp columns in the same table.
> Suggest to provide option in the create table DDL itself to define the format for each
Timestamp column. Also provide defaults so that users can create table with Timestamp columns
without having to always define the Date/Time format.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message