spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tathagata Das (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-24234) create the bottom-of-task RDD with row buffer
Date Mon, 21 May 2018 19:59:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tathagata Das reassigned SPARK-24234:
-------------------------------------

    Assignee: Jose Torres

> create the bottom-of-task RDD with row buffer
> ---------------------------------------------
>
>                 Key: SPARK-24234
>                 URL: https://issues.apache.org/jira/browse/SPARK-24234
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Structured Streaming
>    Affects Versions: 2.4.0
>            Reporter: Jose Torres
>            Assignee: Jose Torres
>            Priority: Major
>             Fix For: 3.0.0
>
>
> [https://docs.google.com/document/d/1IL4kJoKrZWeyIhklKUJqsW-yEN7V7aL05MmM65AYOfE/edit#heading=h.8t3ci57f7uii]
>  
> This probably ought to be an abstraction of ContinuousDataSourceRDD and ContinuousQueuedDataReader.
These classes do pretty much exactly what's needed, except the buffer is filled from the
remote data source instead of a remote Spark task.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message