carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CARBONDATA-298) 3. Add InputProcessorStep which should iterate recordreader and parse the data as per the data type.
Date Thu, 20 Oct 2016 08:49:58 GMT

    [ https://issues.apache.org/jira/browse/CARBONDATA-298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15591238#comment-15591238
] 

ASF GitHub Bot commented on CARBONDATA-298:
-------------------------------------------

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/240#discussion_r84227827
  
    --- Diff: processing/src/main/java/org/apache/carbondata/processing/newflow/parser/CarbonParserFactory.java
---
    @@ -0,0 +1,40 @@
    +package org.apache.carbondata.processing.newflow.parser;
    +
    +import java.util.List;
    +
    +import org.apache.carbondata.core.carbon.metadata.schema.table.column.CarbonColumn;
    +import org.apache.carbondata.core.carbon.metadata.schema.table.column.CarbonDimension;
    +import org.apache.carbondata.processing.newflow.parser.impl.ArrayParserImpl;
    +import org.apache.carbondata.processing.newflow.parser.impl.PrimitiveParserImpl;
    +import org.apache.carbondata.processing.newflow.parser.impl.StructParserImpl;
    +
    +public class CarbonParserFactory {
    +
    +  public static GenericParser createParser(CarbonColumn carbonColumn, String[] complexDelimiters)
{
    +    return createParser(carbonColumn, complexDelimiters, 0);
    +  }
    +
    +  private static GenericParser createParser(CarbonColumn carbonColumn, String[] complexDelimiters,
    +      int counter) {
    +    switch (carbonColumn.getDataType()) {
    +      case ARRAY:
    +        List<CarbonDimension> listOfChildDimensions =
    +            ((CarbonDimension) carbonColumn).getListOfChildDimensions();
    +        ArrayParserImpl arrayParser = new ArrayParserImpl(complexDelimiters[counter]);
    +        for (CarbonDimension dimension : listOfChildDimensions) {
    +          arrayParser.addChildren(createParser(dimension, complexDelimiters, counter
+ 1));
    +        }
    +        return arrayParser;
    +      case STRUCT:
    +        List<CarbonDimension> dimensions =
    +            ((CarbonDimension) carbonColumn).getListOfChildDimensions();
    +        StructParserImpl parser = new StructParserImpl(complexDelimiters[counter]);
    +        for (CarbonDimension dimension : dimensions) {
    +          parser.addChildren(createParser(dimension, complexDelimiters, counter + 1));
    +        }
    +        return parser;
    +      default:
    --- End diff --
    
    DataType can be Map, I think it is added in DataType already, so we should add a case
branch for Map, otherwise logic is not correct


> 3. Add InputProcessorStep which should iterate recordreader and parse the data as per
the data type.
> ----------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-298
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-298
>             Project: CarbonData
>          Issue Type: Sub-task
>            Reporter: Ravindra Pesala
>             Fix For: 0.2.0-incubating
>
>
> Add InputProcessorStep which should iterate recordreader/RecordBufferedWriter and parse
the data as per the data types.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message