spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Assigned] (SPARK-23674) Add Spark ML Listener for Tracking ML Pipeline Status
Date Wed, 14 Mar 2018 18:00:00 GMT


Apache Spark reassigned SPARK-23674:

    Assignee:     (was: Apache Spark)

> Add Spark ML Listener for Tracking ML Pipeline Status
> -----------------------------------------------------
>                 Key: SPARK-23674
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 2.3.0
>            Reporter: Mingjie Tang
>            Priority: Major
> Currently, Spark provides status monitoring for different components of Spark, like spark
history server, streaming listener, sql listener and etc. The use case would be (1) front
UI to track the status of training coverage rate during iteration, then DS can understand
how the job converge when training, like K-means, Logistic and other linear regression model. 
(2) tracking the data lineage for the input and output of training data.  
> In this proposal, we hope to provide Spark ML pipeline listener to track the status of
Spark ML pipeline status includes: 
>  # ML pipeline create and saved 
>  # ML pipeline model created, saved and load  
>  # ML model training status monitoring  

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message