flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Derek VerLee <derekver...@gmail.com>
Subject local disk cleanup after crash
Date Thu, 07 Mar 2019 21:45:37 GMT
<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>I think that effort is put in to have task managers clean up
      their folders, however I have noticed that in some cases local
      folders are not cleaned up and can build up, eventually causing
      problems due to a full disk.  As far as I know this only happens
      with crashes and other out-of-happy-path scenarios.<br>
    </p>
    <p>I am thinking of writing a script to clean up local folders that
      runs before task-manager starts between restarts in the case of a
      crash.</p>
    <p>Assuming local recovery is not configured, what should I delete
      and what should I leave around?<br>
    </p>
    <p>What should I keep if local recovery is configured?</p>
    <p><br>
    </p>
    <p>Under the "taskmanager.tmp.dirs" I see:</p>
    <p>blobStore-*<br>
      flink-dist-cache-*<br>
      flink-io-*<br>
      localState/*<br>
      rocksdb-lib-*</p>
    <p><br>
    </p>
    <p>Thanks<br>
    </p>
  </body>
</html>

Mime
View raw message