Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C4505200BFA for ; Thu, 12 Jan 2017 22:56:57 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id C2C97160B40; Thu, 12 Jan 2017 21:56:57 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2C188160B29 for ; Thu, 12 Jan 2017 22:56:56 +0100 (CET) Received: (qmail 36481 invoked by uid 500); 12 Jan 2017 21:56:55 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 36471 invoked by uid 99); 12 Jan 2017 21:56:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2017 21:56:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id A1012C1D91 for ; Thu, 12 Jan 2017 21:56:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.69 X-Spam-Level: * X-Spam-Status: No, score=1.69 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, T_KAM_HTML_FONT_INVALID=0.01, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id mIn8ij8cn_yW for ; Thu, 12 Jan 2017 21:56:52 +0000 (UTC) Received: from mail-it0-f46.google.com (mail-it0-f46.google.com [209.85.214.46]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 5512E5FAED for ; Thu, 12 Jan 2017 21:56:51 +0000 (UTC) Received: by mail-it0-f46.google.com with SMTP id r185so24125177ita.0 for ; Thu, 12 Jan 2017 13:56:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=xWozyYYpZnMPqt+0VY3KXTmliNuytDn+cZjFfaDQdOs=; b=CdswGz0B60os9R+9BfG1dleE/V8+RA/yWKFxAUVb/ebVMzPA6wOW4sDZ3FkxFf22YF MjygkVK6YcYAgIaWwHrrbEf76NRBYURmzDsq7w7P3KKvCxwCp6aunmB2FTjWfSQXBtva pi8YO4du0PdqnSV/h2cp0sHn6/7/gN3f92zB+q/+PIisabI3o6EPt26SiOOiEEzAmW4G +O9QAwS6+S26rF58Us2CWLaLPJ5qZfp/YfmlRCcEoGgKZlLvg+DtFThJu/U1PTUrWbzm TZo9omMaLKesHFhOyZAcEoT+rnSSNdHHK0FcniN8kGzgKTtUT97IT6wTnVc+pvw2dGH6 huRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=xWozyYYpZnMPqt+0VY3KXTmliNuytDn+cZjFfaDQdOs=; b=YSoMnMJfhDGEgNnJuIeBZnsrXehOh2sJZpK9MO6D420a8Or8IpLNl6/OMHDLbiMYz0 7fhbX+9RYP3i1MMXsHo3A4QTErf3LQsdY5PRs1v7FOljtHn8g5LCEbkwoN81viiEkI9B LwBWJvOX8H8IaEgzvgXLYaE50OrknZWFv/bzmY7/SvT30mULqDZaGnmrf9qKCz/xxePB gN5h7cJG46r+tIEFM3V3J5A2wyHF2GbuVbJ+7+EwCiJH1inji/VhFI7MpNLD3HbT7BVf up1U98zNZy8thxVH+LBr2xrbp5P3b6wp60pMiPOBawodZdqFDSCT11X2U7IgljJytK2p Ha4g== X-Gm-Message-State: AIkVDXL8bopxvIwiy/tTZi7KJDys6+MaYW5O0m39IJjI8HnbYQR+9enPArmBkRB83+x2XA== X-Received: by 10.36.53.69 with SMTP id k66mr321153ita.18.1484258209743; Thu, 12 Jan 2017 13:56:49 -0800 (PST) Received: from ?IPv6:2001:18e8:2:28b9:a4d:ce0e:d1ce:b039? ([2001:18e8:2:28b9:a4d:ce0e:d1ce:b039]) by smtp.gmail.com with ESMTPSA id p20sm1761368itc.8.2017.01.12.13.56.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Jan 2017 13:56:48 -0800 (PST) Subject: Re: Database Version Control for Airavata To: dev@airavata.apache.org References: <14C32E1E-D398-4FA7-9E10-865948C1BCF2@iu.edu> <03d4e5d3-2b73-2585-3acc-6507fb5ef8b3@gmail.com> <548DCE1B-F2D1-4F92-B6FB-19474B4F2CC2@apache.org> From: Anuj Message-ID: Date: Thu, 12 Jan 2017 16:56:47 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <548DCE1B-F2D1-4F92-B6FB-19474B4F2CC2@apache.org> Content-Type: multipart/alternative; boundary="------------9D961F8E75A3E13613E23B02" archived-at: Thu, 12 Jan 2017 21:56:58 -0000 This is a multi-part message in MIME format. --------------9D961F8E75A3E13613E23B02 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit hello Suresh, I had gone through the documentation for FlywayDB, Though the docs are not very clear, here are my observations * FlywayDB doesn't generate delta Scripts like DBDiff does * It is mainly used to run the migration scripts which are already created by some other means, like the one's we generated using DBDiff * https://flywaydb.org/documentation/migration/sql - We just point the FlywayDB to a classpath where the scripts are stored, it will run them accordingly. please correct me if I am wrong, FlywayDB can be used along with DBDiff or Liquibase to bring in automation. Thanks and best regards, *Anuj Bhandar* MS Computer Science Indiana University Bloomington +1 812 361 9414 On 01/12/2017 04:34 PM, Suresh Marru wrote: > Hi Anuj, > > Have you looked at https://flywaydb.org/ ? Its maven integration > caught my attention. Just curios how it compared with the liquibase or > dbdiff you have discussed below. > > Suresh > >> On Dec 27, 2016, at 1:31 PM, Miller, Mark > > wrote: >> >> Thanks Anuj, For taking the time to send that, it was super helpful, >> and answered my questions. >> Mark >> *From:*Anuj Bhandar [mailto:bhandar.anuj@gmail.com] >> *Sent:*Tuesday, December 27, 2016 10:23 AM >> *To:*dev@airavata.apache.org >> *Subject:*Re: Database Version Control for Airavata >> >> HI Mark, >> >> Liquibase is a Database Version control tool, it gives you the >> ability to rollback any changes (commit) on DataBase Schema just like >> GIT, but it will only be useful for future builds, i.e. if you >> implement Liquibase in current build, make changes on top of that, >> then those changes can be rolled back, in short the version to be >> rolled back should be present in the repository. Works just like GIT >> for source code. >> >> But if you are referring to Airavata's scenario, i.e. we have two >> version of DB schema's, one for Development and the other for >> Production, then a tool >> like*DBDiff*(https://github.com/DBDiff/DBDiff) compares the two >> different versions and generates Delta SQL scripts, which can be run >> on the DB schema's, DBDiff has the ability to generate both UP and >> Down Migration scripts. >> >> I hope that I have answered the question, please revert back if you >> need further details. >> >> Thanks and best regards, >> >> *Anuj Bhandar* >> MS Computer Science >> Indiana University Bloomington >> +1 812 361 9414 >> >> On 12/27/2016 9:19 AM, Christie, Marcus Aaron wrote: >> >> Mark, >> Anuj might know more about this but I thought I would chime in. I >> believe this is something that database migration tools can do. I >> haven’t tried that though. Here’s Liquibase’s support for >> rollback scripts: >> http://www.liquibase.org/documentation/rollback.html >> Marcus >> >> On Dec 25, 2016, at 8:49 AM, Miller, Mark > > wrote: >> Hi Anuj, >> Very interesting. I had a question, rather than a vote. We >> have found (at CIPRES) that we really wish we had the ability >> to make DB schema changes and test them and roll them back if >> there is an issue with what we tried. I wonder if this is >> something the new or existing DBs can do in AIravata? >> Mark >> *From:*Pierce, Marlon [mailto:marpierc@iu.edu] >> *Sent:*Friday, December 23, 2016 10:11 AM >> *To:*dev@airavata.apache.org >> *Subject:*Re: Database Version Control for Airavata >> +1 for looking into Liquibase. I’d be interested to hear >> other opinions. >> >> Marlon >> *From:*Anuj Bhandar > > >> *Reply-To:*"dev@airavata.apache.org >> " > > >> *Date:*Friday, December 23, 2016 at 12:55 PM >> *To:*"dev@airavata.apache.org >> " > > >> *Subject:*Database Version Control for Airavata >> Hello Dev, >> Merry Christmas ! Hope you guys are doing good. >> As a part of the Airavata 0.17 release, we are faced with a >> industry standard problem of migrating our old master >> database to the new Development database, the effort needed >> to manually write migration SQL scripts is huge. >> Hence for the current release, I'm relying on DBDiff >> (http://dbdiff.github.io/DBDiff/) for generating Delta SQL >> scripts for the entire database and manually write scripts >> for complex DB schema changes. >> To address the issue in future releases, I propose to >> implement a database version control system like Liquibase >> (http://www.liquibase.org/) for Airavata, this will help >> automate the creation of delta scripts and also adding >> additional functionality like backward compatibility between >> releases. >> Please feel to suggest changes and also comment about the >> products I'm using to accomplish these tasks. >> Thanks and best regards, >> *Anuj Bhandar* >> MS Computer Science >> Indiana University Bloomington >> +1 812 361 9414 >> > --------------9D961F8E75A3E13613E23B02 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

hello Suresh,

I had gone through the documentation for FlywayDB, Though the docs are not very clear, here are my observations

    * FlywayDB doesn't generate delta Scripts like DBDiff does

    * It is mainly used to run the migration scripts which are already created by some other means, like the one's we generated using DBDiff

    * https://flywaydb.org/documentation/migration/sql - We just point the FlywayDB to a classpath where the scripts are stored, it will run them accordingly.

please correct me if I am wrong, FlywayDB can be used along with DBDiff or Liquibase to bring in automation.

Thanks and best regards,

Anuj Bhandar
MS Computer Science
Indiana University Bloomington
+1 812 361 9414


On 01/12/2017 04:34 PM, Suresh Marru wrote:
Hi Anuj,

Have you looked at https://flywaydb.org/ ? Its maven integration caught my attention. Just curios how it compared with the liquibase or dbdiff you have discussed below.

Suresh

On Dec 27, 2016, at 1:31 PM, Miller, Mark <mmiller@sdsc.edu> wrote:

Thanks Anuj, For taking the time to send that, it was super helpful, and answered my questions.
Mark
From: Anuj Bhandar [mailto:bhandar.anuj@gmail.com] 
Sent: Tuesday, December 27, 2016 10:23 AM
To: dev@airavata.apache.org
Subject: Re: Database Version Control for Airavata
 

HI Mark,

Liquibase is a Database Version control tool, it gives you the ability to rollback any changes (commit) on DataBase Schema just like GIT, but it will only be useful for future builds, i.e. if you implement Liquibase in current build, make changes on top of that, then those changes can be rolled back, in short the version to be rolled back should be present in the repository. Works just like GIT for source code.

But if you are referring to Airavata's scenario, i.e. we have two version of DB schema's, one for Development and the other for Production, then a tool like DBDiff (https://github.com/DBDiff/DBDiff) compares the two different versions and generates Delta SQL scripts, which can be run on the DB schema's, DBDiff has the ability to generate both UP and Down Migration scripts.

I hope that I have answered the question, please revert back if you need further details.

Thanks and best regards,

Anuj Bhandar
MS Computer Science
Indiana University Bloomington
+1 812 361 9414

On 12/27/2016 9:19 AM, Christie, Marcus Aaron wrote:
Mark, 
 
Anuj might know more about this but I thought I would chime in. I believe this is something that database migration tools can do. I haven’t tried that though. Here’s Liquibase’s support for rollback scripts: http://www.liquibase.org/documentation/rollback.html
 
Marcus
 
On Dec 25, 2016, at 8:49 AM, Miller, Mark <mmiller@sdsc.edu> wrote:
 
Hi Anuj,
 
Very interesting. I had a question, rather than a vote. We have found (at CIPRES) that we really wish we had the ability to make DB schema changes and test them and roll them back if there is an issue with what we tried. I wonder if this is something the new or existing DBs can do in AIravata?
 
Mark
 
From: Pierce, Marlon [mailto:marpierc@iu.edu] 
Sent: Friday, December 23, 2016 10:11 AM
To: dev@airavata.apache.org
Subject: Re: Database Version Control for Airavata
 
+1 for looking into Liquibase. I’d be interested to hear other opinions.

Marlon
 
From: Anuj Bhandar <bhandar.anuj@gmail.com>
Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
Date: Friday, December 23, 2016 at 12:55 PM
To: "dev@airavata.apache.org" <dev@airavata.apache.org>
Subject: Database Version Control for Airavata
 
Hello Dev,
Merry Christmas ! Hope you guys are doing good.
As a part of the Airavata 0.17 release, we are faced with a industry standard problem of migrating our old master database to the new Development database, the effort needed to manually write migration SQL scripts is huge.
Hence for the current release, I'm relying on DBDiff (http://dbdiff.github.io/DBDiff/) for generating Delta SQL scripts for the entire database and manually write scripts for complex DB schema changes.
To address the issue in future releases, I propose to implement a database version control system like Liquibase (http://www.liquibase.org/) for Airavata, this will help automate the creation of delta scripts and also adding additional functionality like backward compatibility between releases.
Please feel to suggest changes and also comment about the products I'm using to accomplish these tasks.
Thanks and best regards,
Anuj Bhandar
MS Computer Science
Indiana University Bloomington
+1 812 361 9414
 
 


--------------9D961F8E75A3E13613E23B02--