airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-3099) Errors raised when some blocs are missing in airflow.cfg
Date Fri, 21 Sep 2018 21:16:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16624189#comment-16624189
] 

ASF GitHub Bot commented on AIRFLOW-3099:
-----------------------------------------

kaxil opened a new pull request #3934: [AIRFLOW-3099] Stop Missing Section Errors for optional
sections
URL: https://github.com/apache/incubator-airflow/pull/3934
 
 
   Make sure you have checked _all_ steps below.
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/)
issues and references them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
     - https://issues.apache.org/jira/browse/AIRFLOW-3099
   
   
   ### Description
   
   - [x] Here are some details about my PR, including screenshots of any UI changes:
   
   When we upgrade from a version of airflow to another one and new config bloc are available
or if we delete useless blocs, lots of errors are raised if we don't have some blocs in airflow.cfg
file.
   
   We need to avoid these errors for non-required blocs.
   
   For the record logs (not exhaustive):
   
   ```
   [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option sasl_enabled does not
exist in the config!
   [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option host does not exist
in the config!
   [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option port does not exist
in the config!
   [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option username does not exist
in the config!
   [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option password does not exist
in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option default_hive_mapred_queue
does not exist in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option master does not exist
in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option framework_name does
not exist in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option task_cpu does not exist
in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option task_memory does not
exist in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option checkpoint does not
exist in the config!
   [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option authenticate does not
exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch Option elasticsearch_host
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch Option elasticsearch_log_id_template
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch Option elasticsearch_end_of_log_mark
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option worker_container_repository
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option worker_container_tag
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option delete_worker_pods
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option namespace does
not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option airflow_configmap
does not exist in the config!
   [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option dags_volume_subpath
does not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option dags_volume_claim
does not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option logs_volume_subpath
does not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option logs_volume_claim
does not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_repo does
not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_branch does
not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_user does
not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_password does
not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_subpath does
not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_sync_container_repository
does not exist in the config!
   [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_sync_container_tag
does not exist in the config!
   [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option git_sync_init_container_name
does not exist in the config!
   [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option worker_service_account_name
does not exist in the config!
   [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option image_pull_secrets
does not exist in the config!
   [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option gcp_service_account_keys
does not exist in the config!
   [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option in_cluster does
not exist in the config!{noformat}
   ```
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for this extremely
good reason:
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines, and I have squashed
multiple commits if they address the same issue. In addition, my commits follow the guidelines
from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes how to use
it.
     - When adding new operators/hooks/sensors, the autoclass documentation generation needs
to be added.
   
   ### Code Quality
   
   - [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Errors raised when some blocs are missing in airflow.cfg
> --------------------------------------------------------
>
>                 Key: AIRFLOW-3099
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3099
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: configuration
>    Affects Versions: 1.10.0
>            Reporter: Christophe
>            Assignee: Kaxil Naik
>            Priority: Minor
>             Fix For: 1.10.1
>
>
> When we upgrade from a version of airflow to another one and new config bloc are available
or if we delete useless blocs, lots of errors are raised if we don't have some blocs in airflow.cfg
file.
> We need to avoid these errors for non-required blocs.
>  
> For the record logs (not exhaustive):
>  
> {noformat}
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option sasl_enabled does
not exist in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option host does not exist
in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option port does not exist
in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option username does not
exist in the config!
> [2018-09-21 10:49:37,727] {cli.py:464} ERROR - Section atlas Option password does not
exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section hive Option default_hive_mapred_queue
does not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option master does not exist
in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option framework_name does
not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option task_cpu does not
exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option task_memory does
not exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option checkpoint does not
exist in the config!
> [2018-09-21 10:49:37,728] {cli.py:464} ERROR - Section mesos Option authenticate does
not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch Option elasticsearch_host
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch Option elasticsearch_log_id_template
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section elasticsearch Option elasticsearch_end_of_log_mark
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option worker_container_repository
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option worker_container_tag
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option delete_worker_pods
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option namespace does
not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option airflow_configmap
does not exist in the config!
> [2018-09-21 10:49:37,729] {cli.py:464} ERROR - Section kubernetes Option dags_volume_subpath
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option dags_volume_claim
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option logs_volume_subpath
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option logs_volume_claim
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_repo does
not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_branch does
not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_user does
not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_password
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_subpath
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_sync_container_repository
does not exist in the config!
> [2018-09-21 10:49:37,730] {cli.py:464} ERROR - Section kubernetes Option git_sync_container_tag
does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option git_sync_init_container_name
does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option worker_service_account_name
does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option image_pull_secrets
does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option gcp_service_account_keys
does not exist in the config!
> [2018-09-21 10:49:37,731] {cli.py:464} ERROR - Section kubernetes Option in_cluster does
not exist in the config!{noformat}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message