airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob De Schutter (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-827) Add scrapyd operator
Date Thu, 02 Feb 2017 12:33:51 GMT
Bob De Schutter created AIRFLOW-827:

             Summary: Add scrapyd operator
                 Key: AIRFLOW-827
             Project: Apache Airflow
          Issue Type: New Feature
          Components: contrib
            Reporter: Bob De Schutter
            Assignee: Bob De Schutter

This operator allows to schedule a spider run on a scrapyd server.
Optionally, the operator can wait for the crawl process to finish
which allows for downstream tasks to use the scraped data.

This message was sent by Atlassian JIRA

View raw message