edgent-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dale LaBossiere (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (QUARKS-195) Metrics.{counter,rateMeter}() shouldn't use TStream.pipe()
Date Wed, 08 Jun 2016 18:16:21 GMT

    [ https://issues.apache.org/jira/browse/QUARKS-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321118#comment-15321118

Dale LaBossiere commented on QUARKS-195:

At a high level historically we've had/supported a position that there are cases where an
API client may want / need to extend quarks.oplets.core oplets or fully implement quarks.oplet.Oplet.
  The TStream oplet based methods were introduced as necessary to make that possible.  

I'm comfortable with "don't preclude API clients from using oplets when they want to" and
adding TStream.peek(Peek) is just another step in that direction.

As to the rest of your observation / question, I think it boils down to: Oplets add more lifecycle
control and info access than is available to a function.

I see the metrics related Peek oplets are leveraging the Oplet's lifecycle methods / info:
- Oplet.initialize() and OpletContext are used to get access to the runtime ControlService
and register metric's MXBean control.  Info needed for that registration includes the jobId
and opletId (at least indirectly via use of OpletContext.uniqify(); the StreamScope Peek oplet
makes direct use of those ids)
- Oplet.close() is being used to unregister the metric's Control

The absence of similar capabilities at the functional level, for those function implementations
that need it, was noted on the wiki: https://cwiki.apache.org/confluence/display/QUARKS/Experiences+with+ControlService
 it would be good to work through what's there.

> Metrics.{counter,rateMeter}() shouldn't use TStream.pipe()
> ----------------------------------------------------------
>                 Key: QUARKS-195
>                 URL: https://issues.apache.org/jira/browse/QUARKS-195
>             Project: Quarks
>          Issue Type: Bug
>          Components: API
>            Reporter: Dale LaBossiere
>            Assignee: Dale LaBossiere
>            Priority: Minor
> Add `TStream.peek(Peek<T>)` and change `Metrics.counter(TStream)` and `Metrics.rateMeter(TStream)`
to use it instead of `pipe()`.
> The author of CounterOp and RateMeter implemented the functionality as Peek Oplets not
as Consumer functions. Lacking a TStream.peek(Peek), TStream.pipe() must be used to add the
Peek oplets to the topology.
> The runtime treats TStream.peek(Consumer) generated Peek oplets rather differently than
pipe related oplets (added via pipe() or indirectly via pipe-ish functional methods): see
Connector.connect() vs Connector.peek(), and TStream.peek() returns "this" whereas the addition
of pipe oplets returns a new TStream. 
> The use of pipe() in this case is partially responsible for the effect reported in QUARKS-189.

> Adding TStream.peek(Peek) enables users/authors of Peek oplets to get the same peek-ish
behavior as their functional peeker brethen.  It continues to flesh out the general ability
of API clients to implement and add oplets to the topology.
> The growing number of "oplet" based analogs to the "function" based methods makes me
wonder if the oplet ones should be broken out into another interface that TStream implements
(`OpletTStream`?).  It would contain the current `pipe(Pipe)`, `fanin(FanIn,List)`, `sink(Sink)`,
and the new `peek(Peek)`, and any others that may be needed in the future  - e.g., a `split(Split)`
and/or one that can handle multiple iports and oports.
> Instead, TStream.pipe() (ConnectorStream.pipe()) could be modified to deal with Pipe
oplet args in the desired manner and document that Pipe oplets receive this special treatment
and that pipe() returns "this" for them instead of a new TStream.  Adding TStream.peek(Peek)
seems like a clearer alternative, and perhaps oplet.core.Peek should not extend Pipe?

This message was sent by Atlassian JIRA

View raw message