Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
From: Kostas Kloudas <k.kloudas@data-artisans.com>
Message-Id: <B8851AC4-82A0-460A-91E2-CB1A152A6BC3@data-artisans.com>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_F1F89931-C9A4-4597-B3AF-FB367FF62518"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: When using Flink for CEP, can the data in Cassandra database be
 used for state
Date: Thu, 9 Nov 2017 15:01:17 +0100
In-Reply-To: <CAMjGjA8koUSWOXWBHBX0rBpK72CJj0sHiNMidxKGB1HiNFPmtQ@mail.gmail.com>
Cc: user@flink.apache.org
To: shyla deshpande <deshpandeshyla@gmail.com>
References: <CAMjGjA8koUSWOXWBHBX0rBpK72CJj0sHiNMidxKGB1HiNFPmtQ@mail.gmail.com>
archived-at: Thu, 09 Nov 2017 14:01:25 -0000


--Apple-Mail=_F1F89931-C9A4-4597-B3AF-FB367FF62518
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hi Shyla,

Happy to hear that you are experimenting with CEP!

For enriching your input stream with data from Cassandra (or whichever =
external storage system) you could use:
* either the AsyncIO functionality offered by Flink =
(https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/as=
yncio.html =
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/as=
yncio.html>)
* or, iff all your database fits in memory, you could write a =
ProcessFunction =
(https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/pr=
ocess_function.html =
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/pr=
ocess_function.html>) which loads the database in memory in the open() =
method, and then uses the data accordingly.

Afterwards, you can use the resulting (enriched) DataStream to feed it =
into CEP for further processing.

Hope this helps!
Kostas

> On Nov 9, 2017, at 12:08 AM, shyla deshpande =
<deshpandeshyla@gmail.com> wrote:
>=20
> Hello all,
>=20
> I am new to Flink.
>=20
> We have our data in Cassandra database. We have a use case for CEP.=20
> I am checking out if Flink fits well for us. =20
>=20
> When processing the event data, I may want to pull data for the =
cassandra database like the user profile and join with the event data.
>=20
> Is there a way to do that?  I appreciate your help.=20
>=20
> Thanks


--Apple-Mail=_F1F89931-C9A4-4597-B3AF-FB367FF62518
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">Hi Shyla,<div class=3D""><br class=3D""></div><div =
class=3D"">Happy to hear that you are experimenting with CEP!</div><div =
class=3D""><br class=3D""></div><div class=3D"">For enriching your input =
stream with data from Cassandra (or whichever external storage system) =
you could use:</div><div class=3D"">* either the AsyncIO functionality =
offered by Flink (<a =
href=3D"https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/st=
ream/asyncio.html" =
class=3D"">https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev=
/stream/asyncio.html</a>)</div><div class=3D"">* or, <b class=3D"">iff</b>=
 all your database fits in memory, you could write a ProcessFunction (<a =
href=3D"https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/st=
ream/process_function.html" =
class=3D"">https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev=
/stream/process_function.html</a>) which loads the database in memory in =
the open() method, and then uses the data accordingly.</div><div =
class=3D""><br class=3D""></div><div class=3D"">Afterwards, you can use =
the resulting (enriched) DataStream to feed it into CEP for further =
processing.</div><div class=3D""><br class=3D""></div><div class=3D"">Hope=
 this helps!</div><div class=3D"">Kostas</div><div class=3D""><br =
class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On =
Nov 9, 2017, at 12:08 AM, shyla deshpande &lt;<a =
href=3D"mailto:deshpandeshyla@gmail.com" =
class=3D"">deshpandeshyla@gmail.com</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"ltr" =
class=3D""><div class=3D"">Hello all,</div><div class=3D""><br =
class=3D""></div><div class=3D"">I am new to Flink.</div><div =
class=3D""><br class=3D""></div>We have our data in Cassandra database. =
We have a use case for CEP.&nbsp;<div class=3D"">I am checking out if =
Flink fits well for us.&nbsp;&nbsp;<br class=3D""><br =
class=3D""></div><div class=3D"">When processing the event data, I may =
want to pull data for the cassandra database like the user profile and =
join with the event data.</div><div class=3D""><br class=3D""></div><div =
class=3D"">Is there a way to do that?&nbsp; I appreciate your =
help.&nbsp;</div><div class=3D""><br class=3D""></div><div =
class=3D"">Thanks</div></div>
</div></blockquote></div><br class=3D""></div></body></html>=

--Apple-Mail=_F1F89931-C9A4-4597-B3AF-FB367FF62518--