Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
References: <1508214193418-0.post@n3.nabble.com> <CAF7ADNp3C+Meqk-EGHmJ=p8WDecVvbQ-C_Gxqa_4ZdcktYD-pg@mail.gmail.com>
In-Reply-To: <CAF7ADNp3C+Meqk-EGHmJ=p8WDecVvbQ-C_Gxqa_4ZdcktYD-pg@mail.gmail.com>
From: Bago Amirbekian <bago@databricks.com>
Date: Wed, 01 Nov 2017 23:42:52 +0000
Message-ID: <CAKP5QoA0wj=3n-N+F+i1et2_teoDUzx6xFQDbChq+rJRDa7PFA@mail.gmail.com>
Subject: Re: HashingTFModel/IDFModel in Structured Streaming
To: Joseph Bradley <joseph@databricks.com>
Cc: Davis Varghese <verghesd@gmail.com>, "dev@spark.apache.org" <dev@spark.apache.org>
Content-Type: multipart/alternative; boundary="001a114d83eab24ab2055cf46e10"
archived-at: Wed, 01 Nov 2017 23:43:08 -0000

--001a114d83eab24ab2055cf46e10
Content-Type: text/plain; charset="UTF-8"

Davis I'm looking into this. If you could include some code that I can use
to reproduce the error & the stack trace it would be really helpful.

On Fri, Oct 20, 2017 at 11:01 AM Joseph Bradley <joseph@databricks.com>
wrote:

> Hi Davis,
> We've started tracking these issues under this umbrella:
> https://issues.apache.org/jira/browse/SPARK-21926
> I'm hoping we can fix some of these for 2.3.
> Thanks,
> Joseph
>
> On Mon, Oct 16, 2017 at 9:23 PM, Davis Varghese <verghesd@gmail.com>
> wrote:
>
>>  I have built a ML pipeline model on a static twitter data for sentiment
>> analysis. When I use the model on a structured stream, it always throws
>> "Queries with streaming sources must be executed with
>> writeStream.start()".
>> This particular model doesn't contain any documented "unsupported"
>> operations. It only calls the transform() method of the stages. Anyone
>> have
>> encountered the issue? if the model doesn't contain
>> HashingTFModel/IDFModel,
>> it works fine, but then I can not create feature vectors from the tweet.
>>
>> Thanks,
>> Davis
>>
>>
>>
>> --
>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
>
> --
>
> Joseph Bradley
>
> Software Engineer - Machine Learning
>
> Databricks, Inc.
>
> [image: http://databricks.com] <http://databricks.com/>
>

--001a114d83eab24ab2055cf46e10
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Davis I&#39;m looking into this. If you could include some=
 code that I can use to reproduce the error &amp; the stack trace it would =
be really helpful.</div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On =
Fri, Oct 20, 2017 at 11:01 AM Joseph Bradley &lt;<a href=3D"mailto:joseph@d=
atabricks.com">joseph@databricks.com</a>&gt; wrote:<br></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;p=
adding-left:1ex"><div dir=3D"ltr">Hi Davis,<div>We&#39;ve started tracking =
these issues under this umbrella:=C2=A0<a href=3D"https://issues.apache.org=
/jira/browse/SPARK-21926" target=3D"_blank">https://issues.apache.org/jira/=
browse/SPARK-21926</a></div><div>I&#39;m hoping we can fix some of these fo=
r 2.3.</div><div>Thanks,</div><div>Joseph</div></div><div class=3D"gmail_ex=
tra"></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Mon=
, Oct 16, 2017 at 9:23 PM, Davis Varghese <span dir=3D"ltr">&lt;<a href=3D"=
mailto:verghesd@gmail.com" target=3D"_blank">verghesd@gmail.com</a>&gt;</sp=
an> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex">=C2=A0I have built a ML pipeli=
ne model on a static twitter data for sentiment<br>
analysis. When I use the model on a structured stream, it always throws<br>
&quot;Queries with streaming sources must be executed with writeStream.star=
t()&quot;.<br>
This particular model doesn&#39;t contain any documented &quot;unsupported&=
quot;<br>
operations. It only calls the transform() method of the stages. Anyone have=
<br>
encountered the issue? if the model doesn&#39;t contain HashingTFModel/IDFM=
odel,<br>
it works fine, but then I can not create feature vectors from the tweet.<br=
>
<br>
Thanks,<br>
Davis<br>
<br>
<br>
<br>
--<br>
Sent from: <a href=3D"http://apache-spark-developers-list.1001551.n3.nabble=
.com/" rel=3D"noreferrer" target=3D"_blank">http://apache-spark-developers-=
list.1001551.n3.nabble.com/</a><br>
<br>
---------------------------------------------------------------------<br>
To unsubscribe e-mail: <a href=3D"mailto:dev-unsubscribe@spark.apache.org" =
target=3D"_blank">dev-unsubscribe@spark.apache.org</a><br>
<br>
</blockquote></div><br><br clear=3D"all"><div><br></div></div><div class=3D=
"gmail_extra">-- <br><div class=3D"m_8907555974755158822gmail_signature" da=
ta-smartmail=3D"gmail_signature"><div dir=3D"ltr"><p style=3D"font-size:sma=
ll;margin-top:0pt;margin-bottom:0pt"><font color=3D"#000000" face=3D"Arial"=
><span style=3D"font-size:12.6667px;line-height:15.2px;white-space:pre-wrap=
">Joseph Bradley</span></font></p><p style=3D"font-size:small;margin-top:0p=
t;margin-bottom:0pt"><font color=3D"#000000" face=3D"Arial"><span style=3D"=
font-size:12.6667px;line-height:15.2px;white-space:pre-wrap">Software Engin=
eer - Machine Learning</span></font></p><p dir=3D"ltr" style=3D"font-size:1=
2.8px;line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style=3D"font=
-size:12.6667px;font-family:arial;color:rgb(0,0,0);vertical-align:baseline;=
white-space:pre-wrap">Databricks, Inc.</span></p><p dir=3D"ltr" style=3D"fo=
nt-size:12.8px;line-height:1.2;margin-top:0pt;margin-bottom:0pt"><a href=3D=
"http://databricks.com/" style=3D"font-size:12.8px;color:rgb(17,85,204)" ta=
rget=3D"_blank"><img src=3D"https://databricks.com/wp-content/uploads/2016/=
11/db-bug-email-sig-16px.png" width=3D"16" height=3D"16" alt=3D"http://data=
bricks.com"></a><br></p></div></div>
</div></blockquote></div>

--001a114d83eab24ab2055cf46e10--