ctakes-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jay vyas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CTAKES-331) Add persistence layer to SparkStreaming
Date Mon, 17 Nov 2014 14:49:33 GMT

    [ https://issues.apache.org/jira/browse/CTAKES-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214699#comment-14214699

jay vyas commented on CTAKES-331:

Update on this, ive minimized a spark streaming app that i can use as a test bed for solr
and cassandra ETL.

ill paste the code when i get a chance to put it in a branch shortly later tonite

> Add persistence layer to SparkStreaming
> ---------------------------------------
>                 Key: CTAKES-331
>                 URL: https://issues.apache.org/jira/browse/CTAKES-331
>             Project: cTAKES
>          Issue Type: Improvement
>          Components: ctakes-clinical-pipeline
>            Reporter: jay vyas
> With the ability to grab tweets and process them scalable w/ SparkStreaming, we now should
get a persistence layer - so that we can query data after it is ingested.
> I can create a sink interfaces w/ a few options (solr,cassandra,...) for local processing,
and then we can refactor the CTakes portion of the pipeline to run asynchronously to ingest.

This message was sent by Atlassian JIRA

View raw message