From users-return-551-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Tue Mar 23 20:02:57 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id DD4601804BB for ; Tue, 23 Mar 2021 21:02:57 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 955156426E for ; Tue, 23 Mar 2021 20:02:56 +0000 (UTC) Received: (qmail 87801 invoked by uid 500); 23 Mar 2021 20:02:55 -0000 Mailing-List: contact users-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@airflow.apache.org Delivered-To: mailing list users@airflow.apache.org Received: (qmail 87791 invoked by uid 99); 23 Mar 2021 20:02:55 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Mar 2021 20:02:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id B97861FF468 for ; Tue, 23 Mar 2021 20:02:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id Y5sBVTT4YiuZ for ; Tue, 23 Mar 2021 20:02:53 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::22e; helo=mail-lj1-x22e.google.com; envelope-from=theclouddataengineer@gmail.com; receiver= Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com [IPv6:2a00:1450:4864:20::22e]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 4EC3C7FC65 for ; Tue, 23 Mar 2021 20:02:53 +0000 (UTC) Received: by mail-lj1-x22e.google.com with SMTP id r20so27297656ljk.4 for ; Tue, 23 Mar 2021 13:02:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=JzGsCs6Pkr8Wo3Vrc0c9V/Zs7PQtR6BbOT5whuzxr6U=; b=Sp8Jv/0AomahEMfp6hu8rAj8diFkFaCcJBVKdnr4BSyRzkjQnrqCmum14ZxNHIE578 OtI+yZGD6U1HgojJB9t0zto3YZaFp4Cyz3yAQI0QYZvvuhFJOWUr3+u/KNitZ6F7Q8Qv A2fHv33DWCt5lMz+IIswOnXdbQPWhrSc69H4/XIIrGsu5DyhwTO2uFu6k/b7fKzbOSDq g2pfMPYiDmbwkUFcAVOrXcMnrR3Z4cjdA7XH57DND45Mk/7DU9Rn8tJ0XcEpDRzBEoPo Mli8UpYZz7400HjMqE4u9N4iU5xRY1zgngcZuv/XID16DiLwZNPTKuBafYhKAclMVNc9 EdfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=JzGsCs6Pkr8Wo3Vrc0c9V/Zs7PQtR6BbOT5whuzxr6U=; b=Gsv6pt/rPzR6ZH77m6s5LqyussYaL9jrlJQ9aZMZc1gwEdENp5+T9Doojh1PdYOMmu 1XENyzYQlkps/4MvtRd2ZdyV3fvs7NFZkjenc54q6oeC/mTqxf+wQAnEHGZOwBPRyGrX AVim9h3y3SZocpGmYf0eKkNb5jSmCJDp0Rm5ENU7CEmQn247oIEno6F6xE0Sj9StFKgf aISYLuraXQXYQrJpEn9LgWKY0Hl1PzRtnOBz4ypINrNLV8Fi5QcOsGOAlwA/mSCu2xua EPuSUxfbm7deULygLtFWICdZ1D/QznGFUyxwXn5pfGjjCdA0wQWo/FPOPqdBtHDiLVKb xBIw== X-Gm-Message-State: AOAM5328JgC7jjrskDrcH7rDAj7ooIwHrMLsDxHMjlcWiHmLJ86Rtk4d LgVGa0jOorCbmotvkuVyoHR0mcYqiNAfVWgn7seLOLv/FXta8A== X-Google-Smtp-Source: ABdhPJwy4NwvdW0zT5INAA3O6ye1kIpdXXjYVfVTZkkkogDzy0GmLpAkTPsoTPjfSP8AZ4xtVuTtHBvSkqasc+GYF+Q= X-Received: by 2002:a2e:bc25:: with SMTP id b37mr4122494ljf.342.1616529772264; Tue, 23 Mar 2021 13:02:52 -0700 (PDT) MIME-Version: 1.0 From: William Glennon Date: Tue, 23 Mar 2021 16:02:41 -0400 Message-ID: Subject: airflow db init fips enabled? To: users@airflow.apache.org Content-Type: multipart/alternative; boundary="000000000000d8429005be39aa12" --000000000000d8429005be39aa12 Content-Type: text/plain; charset="UTF-8" Hi, I am working on a project that has CentOS 7 fips enabled configuration using Python 3.6.8. Airflow version 2.0.0 If I disable fips, I can get "airflow db init" to run correctly. However, with fips enabled I get the traceback below when I run "airflow db init". With fips enabled you can't use md5 hash. However, that is what is happening in the Airflow installation I believe. Just one example is to look at flask_caching/backends/filesystemcache.py file. Is there a way to get Airflow working without disabling fips? I did some research but I have not come across anything yet. Thanks in advance for your help! (I also opened an issue on Github for this.) Traceback (most recent call last): File "/home/airflow/airflowpoc/bin/airflow", line 8, in sys.exit(main()) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/__main__.py", line 40, in main args.func(args) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command return func(*args, **kwargs) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb db.initdb() File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/utils/db.py", line 549, in initdb upgradedb() File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/utils/db.py", line 688, in upgradedb command.upgrade(config, 'heads') File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/command.py", line 294, in upgrade script.run_env() File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/script/base.py", line 481, in run_env util.load_python_file(self.dir, "env.py") File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file module = load_module_py(module_id, path) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/util/compat.py", line 182, in load_module_py spec.loader.exec_module(module) File "", line 678, in exec_module File "", line 219, in _call_with_frames_removed File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/migrations/env.py", line 108, in run_migrations_online() File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online context.run_migrations() File "", line 8, in run_migrations File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/runtime/environment.py", line 813, in run_migrations self.get_context().run_migrations(**kw) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/runtime/migration.py", line 560, in run_migrations step.migration_fn(**kw) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 310, in upgrade remap_permissions() File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 287, in remap_permissions appbuilder = create_app(config={'FAB_UPDATE_PERMS': False}).appbuilder File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/www/app.py", line 105, in create_app Cache(app=flask_app, config={'CACHE_TYPE': 'filesystem', 'CACHE_DIR': '/tmp'}) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/__init__.py", line 155, in __init__ self.init_app(app, config) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/__init__.py", line 210, in init_app self._set_cache(app, config) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/__init__.py", line 237, in _set_cache app, config, cache_args, cache_options File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/backends/__init__.py", line 68, in filesystem return FileSystemCache(*args, **kwargs) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/backends/filesystemcache.py", line 80, in __init__ self._update_count(value=len(self._list_dir())) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/backends/filesystemcache.py", line 108, in _list_dir for name in (self._fs_count_file,) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/backends/filesystemcache.py", line 108, in for name in (self._fs_count_file,) File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/backends/filesystemcache.py", line 149, in _get_filename hash = self._hash_method(key).hexdigest() ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips --000000000000d8429005be39aa12 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

I am working on a project=C2=A0that= has CentOS 7 fips enabled configuration using Python 3.6.8.
Airf= low version 2.0.0
If I disable fips, I can get "airflow db i= nit" to run correctly.
However, with fips enabled I get the = traceback below when I run "airflow db init".

With = fips enabled you can't use md5 hash. However, that is what is happening= in the Airflow installation I believe.=C2=A0
Just one example is= to look at=C2=A0flask_caching/backends/filesystemcache.py file.
=
Is there a way to get Airflow working without disabling=C2= =A0fips?
I did some research but I have not come across anything= =C2=A0yet.

Thanks in advance for your help!
<= div>(I also opened an issue on Github for this.)


Traceback (most recent call last):
=C2=A0 File "/= home/airflow/airflowpoc/bin/airflow", line 8, in <module>
=C2= =A0 =C2=A0 sys.exit(main())
=C2=A0 File "/home/airflow/airflowpoc/l= ib64/python3.6/site-packages/airflow/__main__.py", line 40, in main=C2=A0 =C2=A0 args.func(args)
=C2=A0 File "/home/airflow/airflowpo= c/lib64/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, i= n command
=C2=A0 =C2=A0 return func(*args, **kwargs)
=C2=A0 File &quo= t;/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/cli/comman= ds/db_command.py", line 31, in initdb
=C2=A0 =C2=A0 db.initdb()
= =C2=A0 File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/ai= rflow/utils/db.py", line 549, in initdb
=C2=A0 =C2=A0 upgradedb()=C2=A0 File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/= airflow/utils/db.py", line 688, in upgradedb
=C2=A0 =C2=A0 command.= upgrade(config, 'heads')
=C2=A0 File "/home/airflow/airflow= poc/lib64/python3.6/site-packages/alembic/command.py", line 294, in up= grade
=C2=A0 =C2=A0 script.run_env()
=C2=A0 File "/home/airflow/= airflowpoc/lib64/python3.6/site-packages/alembic/script/base.py", line= 481, in run_env
=C2=A0 =C2=A0 util.load_python_file(self.dir, "env= .py")
=C2=A0 File "/home/airflow/airflowpoc/lib64/python3.6/si= te-packages/alembic/util/pyfiles.py", line 97, in load_python_file
= =C2=A0 =C2=A0 module =3D load_module_py(module_id, path)
=C2=A0 File &qu= ot;/home/airflow/airflowpoc/lib64/python3.6/site-packages/alembic/util/comp= at.py", line 182, in load_module_py
=C2=A0 =C2=A0 spec.loader.exec_= module(module)
=C2=A0 File "<frozen importlib._bootstrap_externa= l>", line 678, in exec_module
=C2=A0 File "<frozen impor= tlib._bootstrap>", line 219, in _call_with_frames_removed
=C2=A0= File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/= migrations/env.py", line 108, in <module>
=C2=A0 =C2=A0 run_m= igrations_online()
=C2=A0 File "/home/airflow/airflowpoc/lib64/pyth= on3.6/site-packages/airflow/migrations/env.py", line 102, in run_migra= tions_online
=C2=A0 =C2=A0 context.run_migrations()
=C2=A0 File "= ;<string>", line 8, in run_migrations
=C2=A0 File "/home= /airflow/airflowpoc/lib64/python3.6/site-packages/alembic/runtime/environme= nt.py", line 813, in run_migrations
=C2=A0 =C2=A0 self.get_context(= ).run_migrations(**kw)
=C2=A0 File "/home/airflow/airflowpoc/lib64/= python3.6/site-packages/alembic/runtime/migration.py", line 560, in ru= n_migrations
=C2=A0 =C2=A0 step.migration_fn(**kw)
=C2=A0 File "= /home/airflow/airflowpoc/lib64/python3.6/site-packages/airflow/migrations/v= ersions/2c6edca13270_resource_based_permissions.py", line 310, in upgr= ade
=C2=A0 =C2=A0 remap_permissions()
=C2=A0 File "/home/airflow= /airflowpoc/lib64/python3.6/site-packages/airflow/migrations/versions/2c6ed= ca13270_resource_based_permissions.py", line 287, in remap_permissions=
=C2=A0 =C2=A0 appbuilder =3D create_app(config=3D{'FAB_UPDATE_PERMS= ': False}).appbuilder
=C2=A0 File "/home/airflow/airflowpoc/lib= 64/python3.6/site-packages/airflow/www/app.py", line 105, in create_ap= p
=C2=A0 =C2=A0 Cache(app=3Dflask_app, config=3D{'CACHE_TYPE': &= #39;filesystem', 'CACHE_DIR': '/tmp'})
=C2=A0 File &= quot;/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/_= _init__.py", line 155, in __init__
=C2=A0 =C2=A0 self.init_app(app,= config)
=C2=A0 File "/home/airflow/airflowpoc/lib64/python3.6/site= -packages/flask_caching/__init__.py", line 210, in init_app
=C2=A0 = =C2=A0 self._set_cache(app, config)
=C2=A0 File "/home/airflow/airf= lowpoc/lib64/python3.6/site-packages/flask_caching/__init__.py", line = 237, in _set_cache
=C2=A0 =C2=A0 app, config, cache_args, cache_options<= br>=C2=A0 File "/home/airflow/airflowpoc/lib64/python3.6/site-packages= /flask_caching/backends/__init__.py", line 68, in filesystem
=C2=A0= =C2=A0 return FileSystemCache(*args, **kwargs)
=C2=A0 File "/home/= airflow/airflowpoc/lib64/python3.6/site-packages/flask_caching/backends/fil= esystemcache.py", line 80, in __init__
=C2=A0 =C2=A0 self._update_c= ount(value=3Dlen(self._list_dir()))
=C2=A0 File "/home/airflow/airf= lowpoc/lib64/python3.6/site-packages/flask_caching/backends/filesystemcache= .py", line 108, in _list_dir
=C2=A0 =C2=A0 for name in (self._fs_co= unt_file,)
=C2=A0 File "/home/airflow/airflowpoc/lib64/python3.6/si= te-packages/flask_caching/backends/filesystemcache.py", line 108, in &= lt;listcomp>
=C2=A0 =C2=A0 for name in (self._fs_count_file,)
=C2= =A0 File "/home/airflow/airflowpoc/lib64/python3.6/site-packages/flask= _caching/backends/filesystemcache.py", line 149, in _get_filename
= =C2=A0 =C2=A0 hash =3D self._hash_method(key).hexdigest()
ValueError: er= ror:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips<= /div>
--000000000000d8429005be39aa12--