asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <>
Subject Re: Migration of git repository
Date Mon, 08 Jun 2015 02:29:36 GMT

If it is pushed to any wider audience than roughly the dev@ list, it is a release. That definitely
includes maven central.  Artifacts in maven are convenience binaries and this not a release
but they should be traceable to an exact source release. 

Sent from my iPhone

> On Jun 7, 2015, at 19:10, Till Westmann <> wrote:
> Hmm, good point. It doesn’t have to. One question might be if we can push it to some
maven repository, if it’s not an official release. 
> But I think that should also be fine as long as we don’t push it to a repository that
claims to contain official releases. 
> Some mentor input might be helpful on this as well :)
> Cheers,
> Till
>> On Jun 7, 2015, at 6:53 PM, Ildar Absalyamov <> wrote:
>> Does version bump always mean full-fledged Apache release? We need the former just
to resolve compile time dependencies.
>>> On Jun 7, 2015, at 18:49, Till Westmann <> wrote:
>>> In principle I agree with this, but creating a new release will be a little more
involved that just running maven, when we do this at the ASF.
>>> To publish a new release we will have to vet and vote on the release. This takes
at least 72 hours  in the best case if we’re a TLP, the first release candidate is great,
and have enough people to vote. While we’re still in the incubator, releasing will take
a little longer as we also have to get enough votes for the release in the incubator.
>>> As I proposed earlier, it would be really good to go through the full release
process once, before we decide how to structure our processes and infrastructure.
>>> Cheers,
>>> Till
>>>> On Jun 4, 2015, at 6:37 PM, Ildar Absalyamov <>
>>>> I am with Chris on repository separation and I think that the solution to
the issue of Hyracks commits breaking Asterix build is using release Hyracks versions instead
of snapshot ones. Yes, that will create a frequent Hyracks releases (we will have to release
it each time there is a change which spans both Hyracks & Asterix) and we have abandoned
this practice a while ago, but it seems that’s the only way to separate projects logically.
>>>> Here are few examples to clear the picture. In all examples Hyracks version
is 4.5.6-Snapshot, Asterix version is 1.2.3-Snapshot (but it depends on previous release version
Hyracks 4.5.5):
>>>> 1) The changes span both Asterix & Hyracks.
>>>> First make sure that Asterix could depend on Hyracks 4.5.6-Snapshot without
API conflicts & switch Asterix dependency to 4.5.6-Snapshot.
>>>> Submit Gerrit review, once it is done as a part of git-asf script commit
changes, bump Hyracks version to 4.5.6, make Asterix depend on 4.5.6 and bump Hyracks to 4.5.7-Snapshot
right after.
>>>> 2) The changes are located only in Hyracks. Regular review and commit (with
snapshot version) without any version bump.
>>>> 3) The changes are located only in Asterix. Regular review and commit (with
snapshot version) without any version bump.
>>>> In this scenario Hyracks commit can never make Asterix build fail (since
it depends on a stable release) and it’s the responsibility of the first person, whose commits
spans both repos to make sure that the changes in snapshot Hyracks version are properly merged.
>>>> Regarding the Yingyi’s issue with Gerrit topics: could we modify git-gerrit
script so it would submit both Asterix & Hyracks reviews (granted that the latter is needed),
and link them together, setting the proper topic? Gerrit seems to have API for changing that,
>>>>> On Jun 4, 2015, at 15:45, Mike Carey <> wrote:
>>>>> Just a quick high-level note from our nearest equivalent of the pointy-haired
Dilbert guy (aka me):  What would be nice is to have Hyracks changes kick off tests of all
"supported client projects" - AsterixDB, VXQuery, maybe also Pregelix, IMRU, and possibly
others in the future.  I don't think we'll ever prevent such downstream things from being
broken unless we run their tests - so I would suggest that we need a mechanism to keep Hyracks
changes from being permitted to happen without verifying the ongoing integrity of all "blessed"
(priority 1) affected projects....  We could have an agreed upon list of such projects and
tests for each....  It would be nice to have a "quick check" (hello world still works, basics
are working) that was synchronously blocking of such changes, and at least a daily verification
that all's totally well (AFAWK) for them all.
>>>>> Not sure how this affects the still two-sided discussion...  :-)
>>>>> Cheers,
>>>>> Mike
>>>>>> On 6/2/15 10:00 AM, Chris Hillery wrote:
>>>>>>> On Mon, Jun 1, 2015 at 9:46 PM, Yingyi Bu <>
>>>>>>> In my opinion,  merging the repository doesn't break the separation
>>>>>>> hyracks and asterixdb, because the dependencies are controlled
by mvn pom
>>>>>>> files.
>>>>>> That wasn't the separation I was talking about. I meant API separation.
>>>>>> it is now, when we make a change to both Asterix and Hyracks, we
are forced
>>>>>> to consider the API implications, or at least they are put out there
in a
>>>>>> very clear way that we need to look at. If we merge them, people
>>>>>> (rightly) treat the whole thing as one product, and there will be
no brakes
>>>>>> on making wide-ranging API changes.
>>>>>> (As an aside: I don't trust Maven's pom files to do a good job of
>>>>>> the dependency management clean. In fact I trust it to do precisely
>>>>>> opposite, by making it both easier to screw up the dependencies and
>>>>>> to update them in future.)
>>>>>> Again, my point is this: If we truly believe that Hyracks is a re-usable
>>>>>> component, it should be treated as such from source to build to delivery.
>>>>>> By merging in Asterix, we are saying that Asterix is "more equal"
>>>>>> others Hyracks clients, to the point that we're tacitly willing to
>>>>>> those other clients in favor of simplifying Asterix development.
If that is
>>>>>> a fair and true statement, well, then, sure, let's merge them.
>>>>>> 1) It forces those hyracks-only changes to pass asterixdb regression
>>>>>>> tests.  Currently hyracks-only change are not verified by asterixdb
>>>>>> This is a good point, I will admit. However, I think this same goal
can be
>>>>>> met in other ways. My strong preference would be to create a set
of true
>>>>>> API tests inside of Hyracks, which both document and test the external
>>>>>> Hyracks API. That will make API-breaking changes in future much easier
>>>>>> spot, and also make it clear when Asterix is using internal APIs
that it
>>>>>> should not.
>>>>>>> 2) On my local machine,  I don't need to always install hyracks
and then
>>>>>>> verify asterixdb from time to time.  Especially, switching branches
>>>>>>> painful because the installed hyracks snapshot is overwritten
from time to
>>>>>>> time.
>>>>>> I haven't tried working on multiple Hyracks branches at the same
time, so I
>>>>>> haven't experienced this. This seems like a working method error,
>>>>>> If you're working with two things that are "the same version" (even
>>>>>> that's a snapshot version), you'll need to use separate Maven repositories
>>>>>> to install them. In fact, merging the two git repositories would
do nothing
>>>>>> to fix this problem, will it? If the proposal is to put the two source
>>>>>> repositories in the same git repo but otherwise leave them untouched,
>>>>>> nothing would change in the build process. It's possible I'm missing
>>>>>> something there, though.
>>>>>>> 3) I only need to make one code review request and one jenkins
>>>>>>> Currently I need to manually change the topic of my asterixdb
gerrit CL
>>>>>>> every time before I update my hyracks CL, and then manually schedule
>>>>>>> jenkins to run a new asterixdb job.  If I forget to schedule
the jenkins
>>>>>>> job, the asterixdb CL is still shown to be "verified by jenkins".
>>>>>> This is a problem, but it's a problem in commit validation, not in
>>>>>> source. Modifying the source to work around these issues is still
a bad
>>>>>> idea IMHO.
>>>>>> The "change-topic" issue could be fixed with a bit of development
>>>>>> (have the topic point to a change, rather than a specific patchset
on the
>>>>>> change, so you only need to set it once, for instance).
>>>>>> As for manually scheduling Asterix Jenkins jobs, that sounds like
it's only
>>>>>> a problem where your Hyracks change breaks an existing public API.
>>>>>> would be obviated by having true API testing inside of Hyracks, which
>>>>>> something that we should have regardless of any decisions about source
>>>>>> locations.
>>>>>> In summary / repeating myself again: yes, we have some problems because
>>>>>> Hyracks and Asterix are in seperate repositories. But those problems
>>>>>> pointing out true issues with our development and processes. Merging
>>>>>> repositories isn't fixing those problems, it's sweeping them under
the rug.
>>>>>> Long term we would be much better off to identify, isolate, and fix
>>>>>> problems themselves.
>>>>>> Ceej
>>>>>> aka Chris Hillery
>>>> Best regards,
>>>> Ildar
>> Best regards,
>> Ildar

View raw message