spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From brkyvz <...@git.apache.org>
Subject [GitHub] spark pull request: [SPARK-14555] Second cut of Python API for Str...
Date Tue, 26 Apr 2016 00:00:25 GMT
Github user brkyvz commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12673#discussion_r61010762
  
    --- Diff: python/pyspark/sql/streaming.py ---
    @@ -87,6 +87,84 @@ def stop(self):
             self._jcq.stop()
     
     
    +class ContinuousQueryManager(object):
    +    """A class to manage all the :class:`ContinuousQuery` ContinuousQueries active
    +    on a :class:`SQLContext`.
    +
    +    .. note:: Experimental
    +
    +    .. versionadded:: 2.0
    +    """
    +
    +    def __init__(self, jcqm):
    +        self._jcqm = jcqm
    +
    +    @ignore_unicode_prefix
    +    @property
    +    @since(2.0)
    +    def active(self):
    +        """Returns a list of active queries associated with this SQLContext
    +
    +        >>> cq = df.write.format('memory').queryName('this_query').startStream()
    +        >>> cqm = sqlContext.streams
    +        >>> # get the list of active continuous queries
    +        >>> [q.name for q in cqm.active]
    +        [u'this_query']
    +        >>> cq.stop()
    +        """
    +        return [ContinuousQuery(jcq) for jcq in self._jcqm.active()]
    +
    +    @since(2.0)
    +    def get(self, name):
    +        """Returns an active query from this SQLContext or throws exception if an active
query
    +        with this name doesn't exist.
    +
    +        >>> df.write.format('memory').queryName('this_query').startStream()
    +        >>> cq = sqlContext.streams.get('this_query')
    +        >>> cq.isActive
    +        True
    +        >>> cq.stop()
    +        """
    +        if name is None or type(name) != str or len(name.strip()) == 0:
    +            raise ValueError("The name for the query must be a non-empty string. Got:
%s" % name)
    +        return ContinuousQuery(self._jcqm.get(name))
    --- End diff --
    
    Don't we already register the exception capturer? The thrown exception is an `IllegalArgumentException`
and it should be captured right? Do I need to do anything extra?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message