airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anil Kulkarni <anil...@gmail.com>
Subject Re: Airflow-Init stuck at initialization
Date Tue, 17 Aug 2021 15:34:35 GMT
Hi Anthony,

Can you please share your docker compose file?
The file in my repo works fine for me
https://github.com/anilkulkarni87/airflow-docker

It was getting stuck owing to the user permission and was resolved.

On Tue, Aug 17, 2021 at 5:58 AM Anthony Joyce <
anthony.joyce@omicronmedia.com> wrote:

> Thanks Jarek.
>
> The commands below can grab the IP from the hostname, but nc -zvvn
> definitely hangs when trying to connect to my host IP on the Postgres port.
> This helps me narrow it down a bit. Thank you for that suggestion!
>
> Anthony
>
> On Aug 16, 2021, at 1:56 PM, Jarek Potiuk <jarek@potiuk.com> wrote:
>
> Another thing to try - just run airflow image with `bash` parameter
> (`docker run -it <image> bash`) and see if you can connect to your
> host/port (there is `nc` tool installed in the production image). I believe
> when you try it you will hang on trying to connect to your database.
>
> I think there is one reason why it could fail - in case your host name is
> wrongly parsed from the URL (because it contains restricted characters) -
> this could happen.
>
> This is what the connectivity check does (this is where it seems to hang)
> - copied it with explanation on what is happening (gethosbyname will also
> work for IP addresses BTW). You might try to reproduce this.
>
>     # Checks if it is possible to connect to the host using netcat.
>     #
>     # We want to avoid misleading messages and perform only forward lookup
> of the service IP address.
>     # Netcat when run without -n performs both forward and reverse lookup
> and fails if the reverse
>     # lookup name does not match the original name even if the host is
> reachable via IP. This happens
>     # randomly with docker-compose in GitHub Actions.
>     # Since we are not using reverse lookup elsewhere, we can perform
> forward lookup in python
>     # And use the IP in NC and add '-n' switch to disable any DNS use.
>     # Even if this message might be harmless, it might hide the real
> reason for the problem
>     # Which is the long time needed to start some services, seeing this
> message might be totally misleading
>     # when you try to analyse the problem, that's why it's best to avoid
> it,
> ip=$(python -c "import socket; print(socket.gethostbyname('${host}'))")
> nc -zvvn "${ip}" "${port}"
>
>
> J.
>
>
> On Mon, Aug 16, 2021 at 7:31 PM Anthony Joyce <
> anthony.joyce@omicronmedia.com> wrote:
>
>> Thanks for the suggestions. Will do. And yes, I am using the official
>> apache-airflow image.
>>
>> Anthony
>>
>> On Aug 16, 2021, at 1:29 PM, Daniel Standish <dpstandish@gmail.com>
>> wrote:
>>
>> Let me offer a suggestion.
>>
>> I think it's helpful to simplify things and isolate components.  To that
>> end what you could do is scrap docker for a moment.  Create a
>> python virtualenv in your host.  Install airflow into this virtualenv. Set
>> AIRFLOW__CORE__SQL_ALCHEMY_CONN as appropriate and run airflow in the
>> virtualenv.  Then you'll be able to confirm the issue is docker and not
>> some other connectivity issue.
>>
>> Next move on to docker, perhaps without compose.  One thing you could do
>> is get a terminal in a running container and try to connect to this server
>> e.g. using telnet or psql (you might have to install these).  Then you'll
>> figure out what connection string will actually work and it's just a matter
>> of setting up the env vars correctly for airflow to connect in the right
>> way.
>>
>> Also, are you using the official apache-airflow image?  If not, I would
>> recommend that.
>>
>>
>>
>>
>> On Mon, Aug 16, 2021 at 10:22 AM Anthony Joyce <
>> anthony.joyce@omicronmedia.com> wrote:
>>
>>> Hi Daniel,
>>>
>>> Thanks for the suggestion! I’m new at Docker so apologies if some of the
>>> configs are incorrect. Still learning.
>>>
>>> I removed service Postgres since I am using my local db. Still stuck on
>>> *airflow-init*. There are no log entries that explain what it is doing
>>> or what it is waiting for, but obviously *airflow-init *is waiting for
>>> something. I’ve since restarted docker service, flushed iptables chains
>>> relating to Docker, restarted again. Still stuck.
>>>
>>> Anthony
>>>
>>> On Aug 16, 2021, at 1:11 PM, Daniel Standish <dpstandish@gmail.com>
>>> wrote:
>>>
>>> If you are using an already existing postgres, then why are you running
>>> postgres service in compose?
>>>
>>> If that's *not* the database you are connecting to, then you should
>>> remove it from your compose file. (conversely if you *are* trying to
>>> use compose postgres, then we might have some configuration suggestions to
>>> offer)
>>> ---
>>> Separately, on mac compose, for the container to access host i think you
>>> might need to add svc.localhost or something like that... there are SO
>>> posts on this topic eg
>>> https://stackoverflow.com/questions/24319662/from-inside-of-a-docker-container-how-do-i-connect-to-the-localhost-of-the-mac
>>>
>>>
>>>
>>
>
> --
> +48 660 796 129
>
>
> --
Cheers,
Anil Kulkarni
https://anilkulkarni.com/

Mime
View raw message