lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-9684) Add schedule Streaming Expression
Date Fri, 30 Dec 2016 21:22:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15788344#comment-15788344
] 

Joel Bernstein edited comment on SOLR-9684 at 12/30/16 9:21 PM:
----------------------------------------------------------------

One of the things I think would be interesting would be to include a cost based scheduler,
which we could build into this implementation.

Each expression implements a cost() method which could be used to determine which tasks to
schedule together. But this is going to take more thought and probably involve walking the
parse tree to find which collections are involved in the expression.

Currently also the cost() method is not implemented so we'd have to put some thought into
how expressions calculate cost. Fairly soon we'll have to calculate cost for many expressions
to support the Calcite cost based join optimizer. 


was (Author: joel.bernstein):
One of the things I think would be interesting would be to include a cost based scheduler,
which we could build into this implementation.

Each expression implements a cost() method which could be used to determine which tasks to
schedule together. But this is going to take more thought and probably involve walking the
parse tree to find which collections are involved in the expression.

Currently also the cost() method is not implemented so we'd have to put some thought into
how expressions calculate cost.

> Add schedule Streaming Expression
> ---------------------------------
>
>                 Key: SOLR-9684
>                 URL: https://issues.apache.org/jira/browse/SOLR-9684
>             Project: Solr
>          Issue Type: New Feature
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Joel Bernstein
>            Assignee: Joel Bernstein
>             Fix For: master (7.0), 6.4
>
>         Attachments: SOLR-9684.patch, SOLR-9684.patch, SOLR-9684.patch
>
>
> SOLR-9559 adds a general purpose *parallel task executor* for streaming expressions.
The executor() function executes a stream of tasks and doesn't have any concept of task priority.
> The scheduler() function wraps two streams, a high priority stream and a low priority
stream. The scheduler function emits tuples from the high priority stream first, and then
the low priority stream.
> The executor() function can then wrap the scheduler function to see tasks in priority
order.
> Pseudo syntax:
> {code}
> daemon(executor(schedule(topic(tasks, q="priority:high"), topic(tasks, q="priority:low"))))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message