spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Judy Nash <judyn...@exchange.microsoft.com>
Subject RE: New Metrics Sink class not packaged in spark-assembly jar
Date Tue, 10 Feb 2015 07:06:26 GMT
Thanks Patrick! That was the issue.
Built the jars on windows env with mvn and forgot to run make-distributions.ps1  afterward,
so was looking at old jars.

From: Patrick Wendell [mailto:pwendell@gmail.com]
Sent: Monday, February 9, 2015 10:43 PM
To: Judy Nash
Cc: dev@spark.apache.org
Subject: Re: New Metrics Sink class not packaged in spark-assembly jar

Actually, to correct myself, the assembly jar is in assembly/target/scala-2.11 (I think).

On Mon, Feb 9, 2015 at 10:42 PM, Patrick Wendell <pwendell@gmail.com<mailto:pwendell@gmail.com>>
wrote:
Hi Judy,

If you have added source files in the sink/ source folder, they should appear in the assembly
jar when you build. One thing I noticed is that you are looking inside the "/dist" folder.
That only gets populated if you run "make-distribution". The normal development process is
just to do "mvn package" and then look at the assembly jar that is contained in core/target.

- Patrick

On Mon, Feb 9, 2015 at 10:02 PM, Judy Nash <judynash@exchange.microsoft.com<mailto:judynash@exchange.microsoft.com>>
wrote:
Hello,

Working on SPARK-5708<https://issues.apache.org/jira/browse/SPARK-5708> - Add Slf4jSink
to Spark Metrics Sink.

Wrote a new Slf4jSink class (see patch attached), but the new class is not packaged as part
of spark-assembly jar.

Do I need to update build config somewhere to have this packaged?

Current packaged class:
[cid:image001.png@01D044BC.8FE515C0]

Thought I must have missed something basic but can't figure out why.

Thanks!
Judy

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org<mailto:dev-unsubscribe@spark.apache.org>
For additional commands, e-mail: dev-help@spark.apache.org<mailto:dev-help@spark.apache.org>



Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message