apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chaitanya Chebolu <chaita...@datatorrent.com>
Subject Re: Redshift Output Operator
Date Tue, 21 Feb 2017 06:56:02 GMT
Created JIRA for this task: APEXMALHAR-2416

On Mon, Feb 13, 2017 at 4:14 PM, Chaitanya Chebolu <
chaitanya@datatorrent.com> wrote:

> Hi All,
>
>   I am proposing Amazon Redshift output module.
>   Please refer below link about the Redshift: https://aws.amazon.com/
> redshift/
>
>   Primary functionality of this module is load data into Redshift tables
> from data files using copy command. Refer the below link about the copy
> command:
> http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html
>
> Input type to this module is byte[].
>
>   I am proposing the below design:
> 1) Write the tuples into EMR/S3. By default, it writes to S3.
> 2) Once the file is rolled, upload the file into Redshift using copy
> command.
>
> Please share your thoughts on design.
>
> Regards,
> Chaitanya
>



-- 

*Chaitanya*

Software Engineer

E: chaitanya@datatorrent.com | Twitter: @chaithu1403

www.datatorrent.com  |  apex.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message