spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivaram Venkataraman <>
Subject Re: Should spark-ec2 get its own repo?
Date Mon, 20 Jul 2015 19:44:17 GMT
Technically I think the project ends in 2017 and I think we will figure out
a transition for AMPLab repositories when the project ends. I think it
should be pretty simple to transfer ownership to a new organization if /
when the time comes around.


On Mon, Jul 20, 2015 at 12:03 PM, Reynold Xin <> wrote:

> Is amplab the right owner, given its ending next year? Maybe we should
> create spark-ec2, or spark-project instead?
> On Mon, Jul 20, 2015 at 12:01 PM, Shivaram Venkataraman <
>> wrote:
>> I've created and added an initial
>> set of committers. Note that this is not a fork of the existing
>> and users will need to fork from here. This
>> is mostly to avoid the base-fork in pull requests being set incorrectly
>> etc.
>> I'll be migrating some PRs / closing them in the old repo and will also
>> update the README in that repo.
>> Thanks
>> Shivaram
>> On Fri, Jul 17, 2015 at 3:00 PM, Sean Owen <> wrote:
>>> On Fri, Jul 17, 2015 at 6:58 PM, Shivaram Venkataraman
>>> <> wrote:
>>> > I am not sure why the ASF JIRA can be only used to track one set of
>>> > artifacts that are packaged and released together. I agree that
>>> marking a
>>> > fix version as 1.5 for a change in another repo doesn't make a lot of
>>> sense,
>>> > but we could just not use fix versions for the EC2 issues ?
>>> *shrug* it just seems harder and less natural to use ASF JIRA. What's
>>> the benefit? I agree it's not a big deal either way but it's a small
>>> part of the problem we're solving in the first place. I suspect that
>>> one way or the other, there would be issues filed both places, so this
>>> probably isn't worth debating.
>>> > My concerns are less about it being pushed out etc. For better or
>>> worse we
>>> > have had EC2 scripts be a part of the Spark distribution from a very
>>> early
>>> > stage (from version 0.5.0 if my git history reading is correct).  So
>>> users
>>> > will assume that any error with EC2 scripts belong to the Spark
>>> project. In
>>> > addition almost all the contributions to the EC2 scripts come from
>>> Spark
>>> > developers and so keeping the issues in the same mailing list / JIRA
>>> seems
>>> > natural. This I guess again relates to the question of managing issues
>>> for
>>> > code that isn't part of the Spark release artifact.
>>> Yeah good question -- Github doesn't give you a mailing list. I think
>>> dev@ would still be where it's discussed which is ... again 'part of
>>> the problem' but as you say, probably beneficial. It's a pretty low
>>> traffic topic anyway.
>>> > I'll create the amplab/spark-ec2 repo over the next couple of days
>>> unless
>>> > there are more comments on this thread. This will at least alleviate
>>> some of
>>> > the naming confusion over using a repository in mesos and I'll give
>>> Sean,
>>> > Nick, Matthew commit access to it. I am still not convinced about
>>> moving the
>>> > issues over though.
>>> I won't move the issues. Maybe time tells whether one approach is
>>> better, or that it just doesn't matter.
>>> However it'd be a great opportunity to review and clear stale EC2 issues.

View raw message