flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suma Cherukuri <Suma_Cheruk...@symantec.com>
Subject Question about Apache Flink Use Case
Date Fri, 22 Jul 2016 19:03:26 GMT

Good Afternoon!

I work as an engineer at Symantec. My team works on Multi-tenant Event Processing System.
Just a high level background, our customers write data to kafka brokers though agents like
logstash and we process the events and save the log data in Elastic Search and S3.

Use Case: We have a use case where in we write batches of events to S3 when file size limitation
of 1MB (specific to our case) or a certain time threshold is reached.  We are planning on
merging the number of files specific to a folder into one single file based on either time
limit such as every 24 hrs.

 We were considering various options available today and would like to know if Apache Flink
can be used to serve the purpose.

Looking forward to hearing from you.

Thank you
Suma Cherukuri

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message