From dev-return-5382-archive-asf-public=cust-asf.ponee.io@airflow.incubator.apache.org Thu Jun 7 19:31:19 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 04A83180663 for ; Thu, 7 Jun 2018 19:31:18 +0200 (CEST) Received: (qmail 6059 invoked by uid 500); 7 Jun 2018 17:31:17 -0000 Mailing-List: contact dev-help@airflow.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.incubator.apache.org Delivered-To: mailing list dev@airflow.incubator.apache.org Delivered-To: moderator for dev@airflow.incubator.apache.org Received: (qmail 35968 invoked by uid 99); 7 Jun 2018 17:02:24 -0000 X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.869 X-Spam-Level: * X-Spam-Status: No, score=1.869 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=IxhA2RI3a5O+AF5/KO9fKwHNRnjZDvfSU+DyvB1gpv8=; b=pYVHgV/HatkPm3I8iHtJjHCXIgIlEMCbz31La8wEI7j0n7PRwqkmmq4ECG6sG/pKiE BHYoBPUNx7/9kEBmM6fGhzxLW9kWEcNX0//KhxmQIRPR5mlOsJWeJUfyVLsRns2k4GEa THbuW7L9Qu3R0p2qJEk0bpJ3KtTiuyMS0Fe4CJysqY0nwEZpGt1/3lM5g68lNN1hn0T1 z8d9w0sqUZAu0TYzLzXW2wVclypiu7iMngtbThiT4YvZIgEfS0ep1Xrgqj8BL0i+stR7 b3fWOyWfrkwjEp8X3OUExf3gOjT+00aYhrZARVPl75/KYfEgxoQCFX6buiNQOxxVDiRu j29w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=IxhA2RI3a5O+AF5/KO9fKwHNRnjZDvfSU+DyvB1gpv8=; b=KHTZpWqt6Dq09kP2rUq80038a/4p18lu19ModYg+kZIAn7lSfaYrtcIHCDU3kkuIqu fpmHNv6NFAfRNBIbd2ZczDN9Q3qDoExOQIPT5FLg9JjG5tvG/VF2r+qolqInZ8Ei7pZb jawou0DqdoS06rRN31d0OZNaMly1BCHvn/N3CQNrLzfpEVeiF0pRjENCLK/kaiC4Rmqr suMKjd3pswWkHJKHT0NmpmlaAkLP4pxGVKu2vUwDWbOKCAqeXH0J82A4CAX5h4Nusvd0 d1oNpMnYjeAq1JIlOY8xRa73J6ILArXq584S3+tYWjmwDTuBPMZGFhjdasdQgXYcT5yp lrbg== X-Gm-Message-State: APt69E0fACfjMlo+9xZzVmeh8u8w3bx9M3lRshHojLE/+vMch3bxEUc6 xA0MLYrycbOXPbG/iCe+3R+/1POrq7pfG/OQ/JU= X-Google-Smtp-Source: ADUXVKIadcwGUdzteo9iKCTyGfUX0MDjXZSk3HL+CNKoHNgcA4VYaIIe0XWV4KWvTLWYaW8cVq2tWKtmH2b+yIi5nEs= X-Received: by 2002:a24:648c:: with SMTP id t134-v6mr2899453itc.150.1528390938223; Thu, 07 Jun 2018 10:02:18 -0700 (PDT) MIME-Version: 1.0 From: Shikhar Sharma Date: Thu, 7 Jun 2018 22:31:37 +0530 Message-ID: Subject: Airflow: Docker_operator issue To: dev@airflow.apache.org Content-Type: multipart/mixed; boundary="000000000000f3cc40056e103ea1" --000000000000f3cc40056e103ea1 Content-Type: multipart/alternative; boundary="000000000000f3cc3c056e103e9f" --000000000000f3cc3c056e103e9f Content-Type: text/plain; charset="UTF-8" Hi folks, I am trying to create multiple instances(containers) out of single image. I also need a docker volume to have the communication between the containers. This thing is working perfectly fine when I am running the containers manually(through command line in linux) but when I'm trying to orchestrate it through airflow, I am can't see any file inside the volume(my local host directory). PS: Please find the attached dag file and suggest any changes needed. Thanks Regards Shikhar Sharma --000000000000f3cc3c056e103e9f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi folks,

I am trying to cre= ate multiple instances(containers) out of single image. I also need a docke= r volume to have the communication between the containers.

This thing is working perfectly fine when I am running the contain= ers manually(through command line in linux) but when I'm trying to orch= estrate it through airflow, I am can't see any file inside the volume(m= y local host directory).

PS: Please find the attac= hed dag file and suggest any changes needed.

Thank= s

Regards
Shikhar Sharma
--000000000000f3cc3c056e103e9f-- --000000000000f3cc40056e103ea1 Content-Type: text/x-python; charset="US-ASCII"; name="sched_container.py" Content-Disposition: attachment; filename="sched_container.py" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ji4sg0w80 ZnJvbSBhaXJmbG93IGltcG9ydCBEQUcKZnJvbSBhaXJmbG93Lm9wZXJhdG9ycy5kb2NrZXJfb3Bl cmF0b3IgaW1wb3J0IERvY2tlck9wZXJhdG9yCmZyb20gZGF0ZXRpbWUgaW1wb3J0IGRhdGV0aW1l LCB0aW1lZGVsdGEKCmRlZmF1bHRfYXJncyA9IHsKCSdvd25lcic6J1NoaWtoYXIgU2hhcm1hJywK CSdkZXBlbmRzX29uX3Bhc3QnOkZhbHNlLAoJJ3N0YXJ0X2RhdGUnOmRhdGV0aW1lKDIwMTgsNiw0 KSwKfQoKZGFnID0gREFHKCdkb2NrZXJfc2FtcGxlJywgZGVmYXVsdF9hcmdzPWRlZmF1bHRfYXJn cywgc2NoZWR1bGVfaW50ZXJ2YWw9J0BvbmNlJykKCnQxID0gRG9ja2VyT3BlcmF0b3IoYXBpX3Zl cnNpb249JzEuMzcnLCAKCXRhc2tfaWQ9J2RvY2tlcl8xJywgCglpbWFnZT0nbW9kZWxfMTpweXRo b24nLCAKCWNvbW1hbmQ9ICJbJy9iaW4vc2gnLCAnLWMnLCAnY2QgRG9ja2VyLWFzc2lnbm1lbnQt bXVsdGktY29udGFpbmVyJywgJ3B5dGhvbiBleHBsb3JhdG9yeS5weScsICdjZCAuLicsICdta2Rp ciByZXN1bHQnLCAnY3AgL0RvY2tlci1hc3NpZ25tZW50LW11bHRpLWNvbnRhaW5lci8qLnBrbCAv cmVzdWx0J10iLAoJZGFnPWRhZywKCSN0bXBfZGlyPScvaG9tZS9zaGlraGFyL2NvbnRhaW5lcl9v dXRwdXQnLCAKCXZvbHVtZXM9WycvaG9tZS9zaGlraGFyL2NvbnRhaW5lcl9vdXRwdXQ6L3Jlc3Vs dCddKQoKdDIgPSBEb2NrZXJPcGVyYXRvcihhcGlfdmVyc2lvbj0nMS4zNycsIAoJdGFza19pZD0n ZG9ja2VyXzInLCAKCWltYWdlPSdtb2RlbF8xOnB5dGhvbicsIAoJY29tbWFuZD0iWycvYmluL3No JywgJy1jJywgJ2NkIERvY2tlci1hc3NpZ25tZW50LW11bHRpLWNvbnRhaW5lcicsICdnaXQgcHVs bCcsICdweXRob24gbW9kZWxfMS5weScsICdjZCAuLicsICdta2RpciByZXN1bHQnLCAnY3AgL0Rv Y2tlci1hc3NpZ25tZW50LW11bHRpLWNvbnRhaW5lci9tb2RlbF8xX3Njb3JlX2ZpbGUucGtsIC9y ZXN1bHQnXSIsIAoJZGFnPWRhZywgCgkjdG1wX2Rpcj0nL2hvbWUvc2hpa2hhci9jb250YWluZXJf b3V0cHV0JywKCXZvbHVtZXM9WycvaG9tZS9zaGlraGFyL2NvbnRhaW5lcl9vdXRwdXQ6L3Jlc3Vs dCddCgkpCgp0MyA9IERvY2tlck9wZXJhdG9yKGFwaV92ZXJzaW9uPScxLjM3JywgCgl0YXNrX2lk PSdkb2NrZXJfMycsIAoJaW1hZ2U9J21vZGVsXzE6cHl0aG9uJywgCgljb21tYW5kPSJbJy9iaW4v c2gnLCAnLWMnLCAnY2QgRG9ja2VyLWFzc2lnbm1lbnQtbXVsdGktY29udGFpbmVyJywgJ2dpdCBw dWxsJywgJ3B5dGhvbiBtb2RlbF8yLnB5JywgJ2NkIC4uJywgJ21rZGlyIHJlc3VsdCcsICdjcCAv RG9ja2VyLWFzc2lnbm1lbnQtbXVsdGktY29udGFpbmVyL21vZGVsXzJfc2NvcmVfZmlsZS5wa2wg L3Jlc3VsdCddIiwgCglkYWc9ZGFnLCAKCSN0bXBfZGlyPScvaG9tZS9zaGlraGFyL2NvbnRhaW5l cl9vdXRwdXQnLAoJdm9sdW1lcz1bJy9ob21lL3NoaWtoYXIvY29udGFpbmVyX291dHB1dDovcmVz dWx0J10KCSkKCnQ0ID0gRG9ja2VyT3BlcmF0b3IoYXBpX3ZlcnNpb249JzEuMzcnLCAKCXRhc2tf aWQ9J2RvY2tlcl80JywgCglpbWFnZT0nbW9kZWxfMTpweXRob24nLCAKCWNvbW1hbmQ9IlsnL2Jp bi9zaCcsICctYycsICdjZCBEb2NrZXItYXNzaWdubWVudC1tdWx0aS1jb250YWluZXInLCAnZ2l0 IHB1bGwnLCAncHl0aG9uIG91dHB1dC5weScsICdjZCAuLicsICdta2RpciByZXN1bHQnLCAnY3Ag L0RvY2tlci1hc3NpZ25tZW50LW11bHRpLWNvbnRhaW5lci9kYXRhT24qLnR4dCAvcmVzdWx0J10i LCAKCWRhZz1kYWcsIAoJI3RtcF9kaXI9Jy9ob21lL3NoaWtoYXIvY29udGFpbmVyX291dHB1dCcs Cgl2b2x1bWVzPVsnL2hvbWUvc2hpa2hhci9jb250YWluZXJfb3V0cHV0Oi9yZXN1bHQnXQoJKQoK dDIuc2V0X3Vwc3RyZWFtKHQxKQp0My5zZXRfdXBzdHJlYW0odDEpCnQ0LnNldF91cHN0cmVhbSh0 MykKdDQuc2V0X3Vwc3RyZWFtKHQyKQo= --000000000000f3cc40056e103ea1--