Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
Date: Thu, 7 Apr 2016 15:56:50 -0700
Message-ID: 
 <CAJQx5Ny3ZTqWP=R8UCd+R=8MZ2jdBWDOGQbXe6tzyPt2gJngdw@mail.gmail.com>
Subject: MLlib ALS MatrixFactorizationModel.save fails consistently
From: Colin Woodbury <colingw@gmail.com>
To: user <user@spark.apache.org>
Content-Type: multipart/alternative; boundary=001a1147ec326248a0052fecfe67

--001a1147ec326248a0052fecfe67
Content-Type: text/plain; charset=UTF-8

Hi all,

I've implemented most of a content recommendation system for a client.
However, whenever I attempt to save a MatrixFactorizationModel I've
trained, I see one of four outcomes:

1. Despite "save" being wrapped in a "try" block, I see a massive stack
trace quoting some java.io classes. The Model isn't written.
2. Same as the above, but the Model *is* written. It's unusable however, as
it's missing many of the files it should have, particularly in the
"product" folder.
3. Same as the above, but sbt crashes completely.
4. No massive stack trace, and the Model seems to be written. Upon being
loaded by another Spark context and fed a user ID, it claims the user isn't
present in the Model.

Case 4 is pretty rare. I see these failures both locally and when I test on
a Google Cloud instance with much better resources.

Note that `ALS.trainImplicit` and `model.save` are being called from within
a Future. Could it be possible that Play threads are closing before Spark
can finish, thus interrupting it somehow?

We are running Spark 1.6.1 within Play 2.4 and Scala 2.11. All these
failures have occurred while in Play's Dev mode in SBT.

Thanks for any insight you can give.

--001a1147ec326248a0052fecfe67
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div>Hi all,<br><br></div>I&#39;ve implemented m=
ost of a content recommendation system for a client. However, whenever I at=
tempt to save a MatrixFactorizationModel I&#39;ve trained, I see one of fou=
r outcomes:<br><br></div><div>1. Despite &quot;save&quot; being wrapped in =
a &quot;try&quot; block, I see a massive stack trace quoting some <a href=
=3D"http://java.io">java.io</a> classes. The Model isn&#39;t written.<br></=
div><div>2. Same as the above, but the Model *is* written. It&#39;s unusabl=
e however, as it&#39;s missing many of the files it should have, particular=
ly in the &quot;product&quot; folder.<br></div><div>3. Same as the above, b=
ut sbt crashes completely.<br></div><div>4. No massive stack trace, and the=
 Model seems to be written. Upon being loaded by another Spark context and =
fed a user ID, it claims the user isn&#39;t present in the Model.<br></div>=
<div><br></div><div>Case 4 is pretty rare. I see these failures both locall=
y and when I test on a Google Cloud instance with much better resources.<br=
><br></div><div>Note that `ALS.trainImplicit` and `model.save` are being ca=
lled from within a Future. Could it be possible that Play threads are closi=
ng before Spark can finish, thus interrupting it somehow? <br></div><div><b=
r></div>We are running Spark 1.6.1 within Play 2.4 and Scala 2.11. All thes=
e failures have occurred while in Play&#39;s Dev mode in SBT.<br><br></div>=
Thanks for any insight you can give.<br></div>

--001a1147ec326248a0052fecfe67--