spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewan Leith <ewan.le...@realitymine.com>
Subject Spark streaming - update configuration while retaining write ahead log data?
Date Tue, 15 Mar 2016 09:12:53 GMT
Has anyone seen a way of updating the Spark streaming job configuration while retaining the
existing data in the write ahead log?

e.g. if you've launched a job without enough executors and a backlog has built up in the WAL,
can you increase the number of executors without losing the WAL data?

Thanks,
Ewan

Mime
View raw message