systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <>
Subject MLContext scratch space cleanup
Date Tue, 25 Apr 2017 18:35:31 GMT
A recent issue, described in SYSTEMML-1466, made me think about the cleanup
semantics of our temporary scratch_space when coming through the new
MLContext API. For our main compilation chain (hadoop/spark_submit), the
semantics are very clear: we delete the entire script specific directory
before and after execution. However, for MLContext it is not as easy
because temporary variables are potentially handed out as results but we
need the cleanup because otherwise temporary writes fail. Checking for
existing files is also not possible, as this might even lead to incorrect
results. Could somebody please clarify the current cleanup semantics and
point me to the relevant code?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message