airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <ja...@potiuk.com>
Subject Re: Airflow-Init stuck at initialization
Date Mon, 16 Aug 2021 17:56:17 GMT
Another thing to try - just run airflow image with `bash` parameter
(`docker run -it <image> bash`) and see if you can connect to your
host/port (there is `nc` tool installed in the production image). I believe
when you try it you will hang on trying to connect to your database.

I think there is one reason why it could fail - in case your host name is
wrongly parsed from the URL (because it contains restricted characters) -
this could happen.

This is what the connectivity check does (this is where it seems to hang) -
copied it with explanation on what is happening (gethosbyname will also
work for IP addresses BTW). You might try to reproduce this.

    # Checks if it is possible to connect to the host using netcat.
    #
    # We want to avoid misleading messages and perform only forward lookup
of the service IP address.
    # Netcat when run without -n performs both forward and reverse lookup
and fails if the reverse
    # lookup name does not match the original name even if the host is
reachable via IP. This happens
    # randomly with docker-compose in GitHub Actions.
    # Since we are not using reverse lookup elsewhere, we can perform
forward lookup in python
    # And use the IP in NC and add '-n' switch to disable any DNS use.
    # Even if this message might be harmless, it might hide the real reason
for the problem
    # Which is the long time needed to start some services, seeing this
message might be totally misleading
    # when you try to analyse the problem, that's why it's best to avoid it,
ip=$(python -c "import socket; print(socket.gethostbyname('${host}'))")
nc -zvvn "${ip}" "${port}"


J.


On Mon, Aug 16, 2021 at 7:31 PM Anthony Joyce <
anthony.joyce@omicronmedia.com> wrote:

> Thanks for the suggestions. Will do. And yes, I am using the official
> apache-airflow image.
>
> Anthony
>
> On Aug 16, 2021, at 1:29 PM, Daniel Standish <dpstandish@gmail.com> wrote:
>
> Let me offer a suggestion.
>
> I think it's helpful to simplify things and isolate components.  To that
> end what you could do is scrap docker for a moment.  Create a
> python virtualenv in your host.  Install airflow into this virtualenv. Set
> AIRFLOW__CORE__SQL_ALCHEMY_CONN as appropriate and run airflow in the
> virtualenv.  Then you'll be able to confirm the issue is docker and not
> some other connectivity issue.
>
> Next move on to docker, perhaps without compose.  One thing you could do
> is get a terminal in a running container and try to connect to this server
> e.g. using telnet or psql (you might have to install these).  Then you'll
> figure out what connection string will actually work and it's just a matter
> of setting up the env vars correctly for airflow to connect in the right
> way.
>
> Also, are you using the official apache-airflow image?  If not, I would
> recommend that.
>
>
>
>
> On Mon, Aug 16, 2021 at 10:22 AM Anthony Joyce <
> anthony.joyce@omicronmedia.com> wrote:
>
>> Hi Daniel,
>>
>> Thanks for the suggestion! I’m new at Docker so apologies if some of the
>> configs are incorrect. Still learning.
>>
>> I removed service Postgres since I am using my local db. Still stuck on
>> *airflow-init*. There are no log entries that explain what it is doing
>> or what it is waiting for, but obviously *airflow-init *is waiting for
>> something. I’ve since restarted docker service, flushed iptables chains
>> relating to Docker, restarted again. Still stuck.
>>
>> Anthony
>>
>> On Aug 16, 2021, at 1:11 PM, Daniel Standish <dpstandish@gmail.com>
>> wrote:
>>
>> If you are using an already existing postgres, then why are you running
>> postgres service in compose?
>>
>> If that's *not* the database you are connecting to, then you should
>> remove it from your compose file. (conversely if you *are* trying to use
>> compose postgres, then we might have some configuration suggestions to
>> offer)
>> ---
>> Separately, on mac compose, for the container to access host i think you
>> might need to add svc.localhost or something like that... there are SO
>> posts on this topic eg
>> https://stackoverflow.com/questions/24319662/from-inside-of-a-docker-container-how-do-i-connect-to-the-localhost-of-the-mac
>>
>>
>>
>

-- 
+48 660 796 129

Mime
View raw message