spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <>
Subject Re: Using Spark for portfolio manager app
Date Sat, 19 Sep 2015 09:23:01 GMT
If you want to be able to let your users query their portfolio then you may
want to think about storing the current state of the portfolios in
hbase/phoenix or alternatively a cluster of relationaldatabases can make
sense. For the rest you may use Spark.

Le sam. 19 sept. 2015 à 4:43, Thúy Hằng Lê <> a écrit :

> Hi all,
> I am going to build a financial application for Portfolio Manager, where
> each portfolio contains a list of stocks, the number of shares purchased,
> and the purchase price.
> Another source of information is stocks price from market data. The
> application need to calculate real-time gain or lost of each stock in each
> portfolio ( compared to the purchase price).
> I am new with Spark, i know using Spark Streaming I can aggregate
> portfolio possitions in real-time, for example:
>             user A contains:
>                       - 100 IBM stock with transactionValue=$15000
>                       - 500 AAPL stock with transactionValue=$11400
> Now given the stock prices change in real-time too, e.g if IBM price at
> 151, i want to update the gain or lost of it: gainOrLost(IBM) = 151*100 -
> 15000 = $100
> My questions are:
>          * What is the best method to combine 2 real-time streams(
> transaction made by user and market pricing data) in Spark.
>          * How can I use real-time Adhoc SQL again portfolio's positions,
> is there any way i can do SQL on the output of Spark Streamming.
>          For example,
>               select sum(gainOrLost) from portfolio where user='A';
>          * What are prefered external storages for Spark in this use case.
>          * Is spark is right choice for my use case?

View raw message