phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kalyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3135) Support loading csv data using apache phoenix flume plugin
Date Tue, 02 Aug 2016 07:23:20 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403529#comment-15403529
] 

Kalyan commented on PHOENIX-3135:
---------------------------------

I just provided the solution for CSV data with flume.

please verify the solution patch:

https://github.com/kalyanhadooptraining/phoenix/commit/04234ce7564326da564898c3c726640db23d9c66

Adding new feature to existing PhoenixSink. Convert CSV Data to Phoenix Table Data

It supports array data type also with proper `array delimiter`

Note:

// mandatory properties
phoenix-agent.sinks.phoenixsink.serializer = csv
phoenix-agent.sinks.phoenixsink.serializer.columns=c1,c2,c3,c4

// optional properties with sample data
phoenix-agent.sinks.phoenixsink.serializer.csvDelimiter = ,
phoenix-agent.sinks.phoenixsink.serializer.csvQuote = \"
phoenix-agent.sinks.phoenixsink.serializer.csvEscape = \\
phoenix-agent.sinks.phoenixsink.serializer.csvArrayDelimiter = ,

// sample data set 1
schema: col1 varchar , col2 double, col3 varchar, col4 integer
input:  kalyan,10.5,abc,1
input:  "kalyan",10.5,"abc",1

// sample data set 1
schema: col1 varchar , col2 double, col3 varchar[], col4 integer[]
input:  kalyan,10.5,"abc,pqr,xyz","1,2,3,4"

any suggestions .. please update me.

> Support loading csv data using apache phoenix flume plugin
> ----------------------------------------------------------
>
>                 Key: PHOENIX-3135
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3135
>             Project: Phoenix
>          Issue Type: New Feature
>         Environment: cloudera 5.4
>            Reporter: Kalyan
>            Priority: Minor
>             Fix For: 4.9.0
>
>
> To work with below sample data sets ... we need support loading csv data using apache
phoenix flume plugin.
> // sample data set 1
> schema: col1 varchar , col2 double, col3 varchar, col4 integer
> input: kalyan,10.5,abc,1
> input: "kalyan",10.5,"abc",1
> // sample data set 1
> schema: col1 varchar , col2 double, col3 varchar[], col4 integer[]
> input: kalyan,10.5,"abc,pqr,xyz","1,2,3,4"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message