spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information
Date Fri, 08 Apr 2016 14:52:25 GMT

     [ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Graves updated SPARK-14279:
----------------------------------
    Description: 
Right now the spark-submit --version and other parts of the code pick up version information
from a static SPARK_VERSION. We would want to  pick the version from the pom.version probably
stored inside a properties file. Also, it might be nice to have other details like branch,
build information and other specific details when having a spark-submit --version

Note, the motivation is to more easily tie this to automated continuous integration and deployment
and to easily have traceability.

Part of this is right now you have to manually change a java file to change the version that
comes out when you run spark-submit --version. With continuous integration the build numbers
could be something like 1.6.1.X (where X increments on each change) and I want to see the
exact version easily. Having to manually change a java file makes that hard. obviously that
should make the apache spark releases easier as you don't have to manually change this file
as well.

The other important part for me is the git information. This easily lets me trace it back
to exact commits. We have a multi-tenant YARN cluster and users can run many different versions
at once. I want to be able to see exactly which version they are running. The reason to know
exact version can range from helping debug some problem to making sure someone didn't hack
something in Spark to cause bad things (generally they should use approved version), etc.



  was:

Right now the spark-submit --version and other parts of the code pick up version information
from a static SPARK_VERSION. We would want to  pick the version from the pom.version probably
stored inside a properties file. Also, it might be nice to have other details like branch,
build information and other specific details when having a spark-submit --version

the motivation is to more easily tie this to automated continuous integration and deployment
and to easily have traceability.

Part of this is right now you have to manually change a java file to change the version that
comes out when you run spark-submit --version. With continuous integration the build numbers
could be something like 1.6.1.X (where X increments on each change) and I want to see the
exact version easily. Having to manually change a java file makes that hard. obviously that
should make the apache spark releases easier as you don't have to manually change this file
as well.

The other important part for me is the git information. This easily lets me trace it back
to exact commits. We have a multi-tenant YARN cluster and users can run many different versions
at once. I want to be able to see exactly which version they are running. The reason to know
exact version can range from helping debug some problem to making sure someone didn't hack
something in Spark to cause bad things (generally they should use approved version), etc.




> Improve the spark build to pick the version information from the pom file and add git
commit information
> --------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-14279
>                 URL: https://issues.apache.org/jira/browse/SPARK-14279
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>            Reporter: Sanket Reddy
>            Assignee: Sanket Reddy
>            Priority: Minor
>
> Right now the spark-submit --version and other parts of the code pick up version information
from a static SPARK_VERSION. We would want to  pick the version from the pom.version probably
stored inside a properties file. Also, it might be nice to have other details like branch,
build information and other specific details when having a spark-submit --version
> Note, the motivation is to more easily tie this to automated continuous integration and
deployment and to easily have traceability.
> Part of this is right now you have to manually change a java file to change the version
that comes out when you run spark-submit --version. With continuous integration the build
numbers could be something like 1.6.1.X (where X increments on each change) and I want to
see the exact version easily. Having to manually change a java file makes that hard. obviously
that should make the apache spark releases easier as you don't have to manually change this
file as well.
> The other important part for me is the git information. This easily lets me trace it
back to exact commits. We have a multi-tenant YARN cluster and users can run many different
versions at once. I want to be able to see exactly which version they are running. The reason
to know exact version can range from helping debug some problem to making sure someone didn't
hack something in Spark to cause bad things (generally they should use approved version),
etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message