ctakes-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jay vyas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CTAKES-331) Add persistence layer to SparkStreaming
Date Mon, 17 Nov 2014 14:49:33 GMT

    [ https://issues.apache.org/jira/browse/CTAKES-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214699#comment-14214699
] 

jay vyas commented on CTAKES-331:
---------------------------------

Update on this, ive minimized a spark streaming app that i can use as a test bed for solr
and cassandra ETL.

ill paste the code when i get a chance to put it in a branch shortly later tonite

> Add persistence layer to SparkStreaming
> ---------------------------------------
>
>                 Key: CTAKES-331
>                 URL: https://issues.apache.org/jira/browse/CTAKES-331
>             Project: cTAKES
>          Issue Type: Improvement
>          Components: ctakes-clinical-pipeline
>            Reporter: jay vyas
>
> With the ability to grab tweets and process them scalable w/ SparkStreaming, we now should
get a persistence layer - so that we can query data after it is ingested.
> I can create a sink interfaces w/ a few options (solr,cassandra,...) for local processing,
and then we can refactor the CTakes portion of the pipeline to run asynchronously to ingest.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message