apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chaitanya Chebolu <chaita...@datatorrent.com>
Subject Redshift Output Operator
Date Mon, 13 Feb 2017 10:44:33 GMT
Hi All,

  I am proposing Amazon Redshift output module.
  Please refer below link about the Redshift:
https://aws.amazon.com/redshift/

  Primary functionality of this module is load data into Redshift tables
from data files using copy command. Refer the below link about the copy
command:
http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html

Input type to this module is byte[].

  I am proposing the below design:
1) Write the tuples into EMR/S3. By default, it writes to S3.
2) Once the file is rolled, upload the file into Redshift using copy
command.

Please share your thoughts on design.

Regards,
Chaitanya

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message