spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] brkyvz commented on issue #26225: [SPARK-29568][SS] Stop existing running streams when a new stream is launched
Date Wed, 13 Nov 2019 02:19:31 GMT
brkyvz commented on issue #26225: [SPARK-29568][SS] Stop existing running streams when a new
stream is launched
URL: https://github.com/apache/spark/pull/26225#issuecomment-553207189
 
 
   If stopActiveRunOnRestart is false, this piece of code is not even
   executed. Another error is thrown earlier.
   
   On Tue, Nov 12, 2019, 6:04 PM Tathagata Das <notifications@github.com>
   wrote:
   
   > *@tdas* commented on this pull request.
   > ------------------------------
   >
   > In
   > sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala
   > <https://github.com/apache/spark/pull/26225#discussion_r345538033>:
   >
   > >
   > +    activeRunOpt.foreach(_.stop())
   > +
   > +    activeQueriesSharedLock.synchronized {
   > +      // We still can have a race condition when two concurrent instances try to
start the same
   > +      // stream, while a third one was already active. In this case, we throw a
   > +      // ConcurrentModificationException.
   > +      val oldActiveQuery = sparkSession.sharedState.activeStreamingQueries.put(
   > +        query.id, query.streamingQuery) // we need to put the StreamExecution, not
the wrapper
   > +      if (oldActiveQuery != null) {
   > +        throw new ConcurrentModificationException(
   > +          "Another instance of this query was just started by a concurrent session.")
   >
   > This is not the correct error message when stopActiveRunOnRestart is
   > false.
   > If the active run was stopped, then this error message is correct.
   > If the active run was not stopped, then this error will be thrown and
   > therefore should simply say that there is an active run (run id ...).
   >
   > In other words, this can stay as the same message as it was in Spark
   > 2.4,.... may be improved by adding the run id.
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/spark/pull/26225?email_source=notifications&email_token=ABIAE67GGGFBE62ES54DPNDQTNODTA5CNFSM4JD6YV42YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCLKS3FA#pullrequestreview-315960724>,
   > or unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/ABIAE65ERNLXRWXYE3U7HK3QTNODTANCNFSM4JD6YV4Q>
   > .
   >
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message