Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
From: Bolke de Bruin <bdbruin@gmail.com>
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)
Date: Thu, 26 Jan 2017 08:32:14 +0100
Subject: Re: Airflow 1.8.0 BETA 3
Message-Id: <BA5E77FF-5232-4058-9208-4269121435B3@gmail.com>
References: <FC36989F-72F8-45D6-9334-6B4EB114E66D@gmail.com> <CALCMntt4NspxgKmUSMwCyG2v2CS4xL5uu0-2Rp0qkcy3aefogw@mail.gmail.com> <BE1FC56D-6B27-4C28-B3D2-0E27DE10F5BB@gmail.com>
In-Reply-To: <BE1FC56D-6B27-4C28-B3D2-0E27DE10F5BB@gmail.com>
To: dev@airflow.incubator.apache.org
archived-at: Thu, 26 Jan 2017 07:32:52 -0000

Simple workaround: downgrade to beta 1 first.=20

Sent from my iPhone

> On 26 Jan 2017, at 07:34, Bolke de Bruin <bdbruin@gmail.com> wrote:
>=20
> Mmm that is due to the reverting of one changes to the db. Need to look in=
to that how to fix it.=20
>=20
> Sent from my iPhone
>=20
>> On 26 Jan 2017, at 00:51, Alex Van Boxel <alex@vanboxel.be> wrote:
>>=20
>> I do seem to have a problem upgrading the MySQL database with the last
>> commit:
>>=20
>>=20
>> 2017-01-25T23:41:55.662572654Z
>> /usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/ut=
il/messaging.py:69:
>> UserWarning: Revision 1a5a9e6bf2b5 referenced from 1a5a9e6bf2b5 ->
>> 127d2bf2dfa7 (head), Add dag_id/state index on dag_run table is not prese=
nt
>> 2017-01-25T23:41:55.662613670Z   warnings.warn(msg)
>> 2017-01-25T23:41:55.664560884Z Traceback (most recent call last):
>> 2017-01-25T23:41:55.664582238Z   File "/usr/local/bin/airflow", line 4, i=
n
>> <module>
>> 2017-01-25T23:41:55.664758457Z
>> __import__('pkg_resources').run_script('airflow=3D=3D1.8.0b1+apache.incub=
ating',
>> 'airflow')
>> 2017-01-25T23:41:55.664776553Z   File
>> "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
>> 739, in run_script
>> 2017-01-25T23:41:55.664937644Z
>> self.require(requires)[0].run_script(script_name, ns)
>> 2017-01-25T23:41:55.664952001Z   File
>> "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
>> 1494, in run_script
>> 2017-01-25T23:41:55.665072840Z     exec(code, namespace, namespace)
>> 2017-01-25T23:41:55.665086884Z   File
>> "/usr/local/lib/python2.7/site-packages/airflow-1.8.0b1+apache.incubating=
-py2.7.egg/EGG-INFO/scripts/airflow",
>> line 28, in <module>
>> 2017-01-25T23:41:55.665211755Z     args.func(args)
>> 2017-01-25T23:41:55.665225157Z   File
>> "/usr/local/lib/python2.7/site-packages/airflow-1.8.0b1+apache.incubating=
-py2.7.egg/airflow/bin/cli.py",
>> line 931, in upgradedb
>> 2017-01-25T23:41:55.665326691Z     db_utils.upgradedb()
>> 2017-01-25T23:41:55.665346979Z   File
>> "/usr/local/lib/python2.7/site-packages/airflow-1.8.0b1+apache.incubating=
-py2.7.egg/airflow/utils/db.py",
>> line 292, in upgradedb
>> 2017-01-25T23:41:55.665449297Z     command.upgrade(config, 'heads')
>> 2017-01-25T23:41:55.665462333Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/c=
ommand.py",
>> line 174, in upgrade
>> 2017-01-25T23:41:55.665521466Z     script.run_env()
>> 2017-01-25T23:41:55.665527232Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/base.py",
>> line 416, in run_env2017-01-25T23:41:55.665638879Z
>> util.load_python_file(self.dir, 'env.py')
>> 2017-01-25T23:41:55.665656437Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/u=
til/pyfiles.py",
>> line 93, in load_python_file
>> 2017-01-25T23:41:55.665682760Z     module =3D load_module_py(module_id, p=
ath)
>> 2017-01-25T23:41:55.665699257Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/u=
til/compat.py",
>> line 79, in load_module_py
>> 2017-01-25T23:41:55.665725036Z     mod =3D imp.load_source(module_id, pat=
h,
>> fp)
>> 2017-01-25T23:41:55.665751433Z   File
>> "/usr/local/lib/python2.7/site-packages/airflow-1.8.0b1+apache.incubating=
-py2.7.egg/airflow/migrations/env.py",
>> line 88, in <module>
>> 2017-01-25T23:41:55.665778219Z     run_migrations_online()
>> 2017-01-25T23:41:55.665787820Z   File
>> "/usr/local/lib/python2.7/site-packages/airflow-1.8.0b1+apache.incubating=
-py2.7.egg/airflow/migrations/env.py",
>> line 83, in run_migrations_online
>> 2017-01-25T23:41:55.665811353Z     context.run_migrations()
>> 2017-01-25T23:41:55.665818933Z   File "<string>", line 8, in run_migratio=
ns
>> 2017-01-25T23:41:55.666707547Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/r=
untime/environment.py",
>> line 797, in run_migrations
>> 2017-01-25T23:41:55.666839945Z     self.get_context().run_migrations(**kw=
)
>> 2017-01-25T23:41:55.666859336Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/r=
untime/migration.py",
>> line 305, in run_migrations
>> 2017-01-25T23:41:55.666970791Z     for step in self._migrations_fn(heads,=

>> self):
>> 2017-01-25T23:41:55.666992265Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/c=
ommand.py",
>> line 163, in upgrade
>> 2017-01-25T23:41:55.667021804Z     return script._upgrade_revs(revision,
>> rev)
>> 2017-01-25T23:41:55.667032496Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/base.py",
>> line 329, in _upgrade_revs
>> 2017-01-25T23:41:55.667144295Z     revs =3D list(revs)
>> 2017-01-25T23:41:55.667157407Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/revision.py",
>> line 641, in _iterate_revisions
>> 2017-01-25T23:41:55.667302027Z     requested_lowers =3D
>> self.get_revisions(lower)
>> 2017-01-25T23:41:55.667315913Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/revision.py",
>> line 298, in get_revisions
>> 2017-01-25T23:41:55.667390590Z     return sum([self.get_revisions(id_elem=
)
>> for id_elem in id_], ())
>> 2017-01-25T23:41:55.667411867Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/revision.py",
>> line 300, in get_revisions
>> 2017-01-25T23:41:55.667419226Z     resolved_id, branch_label =3D
>> self._resolve_revision_number(id_)
>> 2017-01-25T23:41:55.667434110Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/revision.py",
>> line 433, in _resolve_revision_number
>> 2017-01-25T23:41:55.667576273Z     self._revision_map
>> 2017-01-25T23:41:55.667590641Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/u=
til/langhelpers.py",
>> line 240, in __get__
>> 2017-01-25T23:41:55.667648561Z     obj.__dict__[self.__name__] =3D result=
 =3D
>> self.fget(obj)
>> 2017-01-25T23:41:55.667658598Z   File
>> "/usr/local/lib/python2.7/site-packages/alembic-0.8.9-py2.7.egg/alembic/s=
cript/revision.py",
>> line 151, in _revision_map
>> 2017-01-25T23:41:55.667680311Z     down_revision =3D map_[downrev]
>> 2017-01-25T23:41:55.667746364Z KeyError: '1a5a9e6bf2b5'
>>=20
>>=20
>>=20
>>=20
>>=20
>>=20
>>> On Wed, Jan 25, 2017 at 11:15 PM Bolke de Bruin <bdbruin@gmail.com> wrot=
e:
>>>=20
>>> Hi All,
>>>=20
>>> I have made the THIRD beta of Airflow 1.8.0 available at:
>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <
>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/> , public keys=

>>> are available at
>>> https://dist.apache.org/repos/dist/release/incubator/airflow/ <
>>> https://dist.apache.org/repos/dist/release/incubator/airflow/> . It is
>>> tagged with a local version =E2=80=9Capache.incubating=E2=80=9D so it al=
lows upgrading from
>>> earlier releases. This beta is available for testing in a more productio=
n
>>> like setting (acceptance environment?).
>>>=20
>>> I would like to encourage everyone  to try it out, to report back any
>>> issues so we get to a rock solid release of 1.8.0. When reporting issues=
 a
>>> test case or even a fix is highly appreciated.
>>>=20
>>> Issues cleared:
>>>=20
>>> * Manual trigger not working
>>> * Performance issues with MySQL
>>> * Postgres auto-commit support left over to the driver
>>> * Poison pill taken while task has exited
>>> * Keep cgroups optional
>>> * Funcsigs pinned to 1.0.0
>>>=20
>>> Issue(s) remaining (blocker for RC):
>>> * Cgroups not py3 compatible
>>>=20
>>> If all goes well we should have a Release Candidate on Feb 2. Thanks for=

>>> reporting issues and keep on testing please :). Moving towards RC I tend=
 to
>>> like small bug fixes only. When we mark RC (do we need to vote on this?)=

>>> the procedure becomes even more strict. Please remember that the FINAL
>>> release is dependent on a vote on the IPMC mailinglist.
>>>=20
>>> Cheers
>>> Bolke
>>=20
>> --=20
>> _/
>> _/ Alex Van Boxel