airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ash Berlin-Taylor <...@apache.org>
Subject Re: Airflow 1.10.1b1 release available - PLEASE TEST
Date Mon, 12 Nov 2018 21:51:50 GMT
Hi Hitesh,

My understanding was that the only official place an Apache release can happen from is https://www.apache.org/dist/incubator/airflow/
<https://www.apache.org/dist/incubator/airflow/> - so anything else is by definition
not an official Apache release.

So i guess it depends on what we mean by "Apache" release - could it be confused that it is
in some way official, yes, possibly. Although the end-user would have to go out of their way
to find it and install it so the risk was low, and I felt that the benefit to the community
of being able to test and install this easily was worth the small risk of confusion.

Possibly something I should have asked (voted on?) first, or before we do this in the future.

-ash

> On 12 Nov 2018, at 20:32, Hitesh Shah <hitesh@apache.org> wrote:
> 
> Hello Ash
> 
> For someone who is not familiar with the beta notation or folks who do not
> check whether it is signed or not, could this be confused as an apache
> release? Also, is there a plan to clean up/delete this version within a
> finite time window once the official release voting is kicked off?
> 
> thanks
> Hitesh
> 
> 
> On Fri, Nov 9, 2018 at 5:52 AM Naik Kaxil <k.naik@reply.com> wrote:
> 
>> Good work Ash. Appreciate the clean and categorised Change Log.
>> 
>> Regards,
>> Kaxil
>> 
>> ´╗┐On 09/11/2018, 13:45, "Ash Berlin-Taylor" <ash@apache.org> wrote:
>> 
>>    Hi Everyone,
>> 
>>    I've just released a beta version of 1.10.1! Please could you test
>> this and report back any problems you notice, and also report back if you
>> tried it and it works fine. As this is the first time I've released Airflow
>> there is possible that there are packaging mistakes too. I'm not calling
>> for a vote just yet, but I will give this a few days until I start making
>> release candidates and calling for a formal vote, probably on Monday or
>> Tuesday.
>> 
>>    In order to distinguish it from an actual (apache) release it is:
>> 
>>    1. Marked as beta (python package managers do not install beta
>> versions by default - PEP 440)
>>    2. It is not signed
>>    3. It is not at an official apache distribution location
>> 
>>    It can be installed with SLUGIFY_USES_TEXT_UNIDECODE=yes pip install
>> 'apache-airflow==1.10.1b1'
>> 
>>    (Don't worry, without asking for `--pre` or specifying the version
>> `pip install apache-airflow` will still get 1.10.0)
>> 
>>    Thanks,
>>    Ash
>> 
>> 
>> 
>> 
>> 
>>    Included below is the changelog of this release:
>> 
>>    New features:
>> 
>>    [AIRFLOW-2524] Airflow integration with AWS Sagemaker
>>    [AIRFLOW-2657] Add ability to delete DAG from web ui
>>    [AIRFLOW-2780] Adds IMAP Hook to interact with a mail server
>>    [AIRFLOW-2794] Add delete support for Azure blob
>>    [AIRFLOW-2912] Add operators for Google Cloud Functions
>>    [AIRFLOW-2974] Add Start/Restart/Terminate methods Databricks Hook
>>    [AIRFLOW-2989] No Parameter to change bootDiskType for
>> DataprocClusterCreateOperator
>>    [AIRFLOW-3078] Basic operators for Google Compute Engine
>>    [AIRFLOW-3147] Update Flask-AppBuilder version
>>    [AIRFLOW-3231] Basic operators for Google Cloud SQL (deploy / patch /
>> delete)
>>    [AIRFLOW-3276] Google Cloud SQL database create / patch / delete
>> operators
>> 
>>    Improvements:
>> 
>>    [AIRFLOW-393] Add progress callbacks for FTP downloads
>>    [AIRFLOW-520] Show Airflow version on web page
>>    [AIRFLOW-843] Excpetions now available in context durint
>> on_failure_callback
>>    [AIRFLOW-2476] Update tabulate dependency to v0.8.2
>>    [AIRFLOW-2592] Bump Bleach dependency
>>    [AIRFLOW-2622] Add "confirm=False" option to SFTPOperator
>>    [AIRFLOW-2662] support affinity & nodeSelector policies for kubernetes
>> executor/operator
>>    [AIRFLOW-2709] Improve error handling in Databricks hook
>>    [AIRFLOW-2763] No precheck mechanism in place during worker
>> initialisation for the connection to metadata database
>>    [AIRFLOW-2789] Add ability to create single node cluster to
>> DataprocClusterCreateOperator
>>    [AIRFLOW-2797] Add ability to create Google Dataproc cluster with
>> custom image
>>    [AIRFLOW-2854] kubernetes_pod_operator add more configuration items
>>    [AIRFLOW-2855] Need to Check Validity of Cron Expression When Process
>> DAG File/Zip File
>>    [AIRFLOW-2904] Clean an unnecessary line in
>> airflow/executors/celery_executor.py
>>    [AIRFLOW-2921] A trivial incorrectness in CeleryExecutor()
>>    [AIRFLOW-2922] Potential deal-lock bug in CeleryExecutor()
>>    [AIRFLOW-2932] GoogleCloudStorageHook - allow compression of file
>>    [AIRFLOW-2949] Syntax Highlight for Single Quote
>>    [AIRFLOW-2951] dag_run end_date Null after a dag is finished
>>    [AIRFLOW-2956] Kubernetes tolerations for pod operator
>>    [AIRFLOW-2997] Support for clustered tables in Bigquery hooks/operators
>>    [AIRFLOW-3006] Fix rrror when schedule_interval="None"
>>    [AIRFLOW-3008] Move Kubernetes related example DAGs to
>> contribe/example_dags
>>    [AIRFLOW-3025] Allow to specify dns and dns-search parameters for
>> DockerOperator
>>    [AIRFLOW-3067] (www_rbac) Flask flash messages are not displayed
>> properly (no background color)
>>    [AIRFLOW-3069] Decode output of S3 file transform operator
>>    [AIRFLOW-3090] INFO logs are too verbose
>>    [AIRFLOW-3103] Update Flask-Login
>>    [AIRFLOW-3112] Align SFTP hook with SSH hook
>>    [AIRFLOW-3119] Enable loglevel on celery worker and inherit from
>> airflow.cfg
>>    [AIRFLOW-3137] Make ProxyFix middleware optional
>>    [AIRFLOW-3173] Add _cmd options for more password config options
>>    [AIRFLOW-3177] Change scheduler_heartbeat metric from gauge to counter
>>    [AIRFLOW-3195] Druid Hook: Log ingestion spec and task id
>>    [AIRFLOW-3197] EMR Hook is missing some parameters to valid on the AWS
>> API
>>    [AIRFLOW-3232] Make documentation for GCF Functions operator more
>> readable
>>    [AIRFLOW-3262] Can't get log containing Response when using
>> SimpleHttpOperator
>>    [AIRFLOW-3265] Add support for "unix_socket" in connection extra for
>> Mysql Hook
>> 
>>    Doc-only changes:
>> 
>>    [AIRFLOW-1441] Tutorial Inconsistencies Between Example Pipeline
>> Definition and Recap
>>    [AIRFLOW-2682] Add how-to guide(s) for how to use basic operators like
>> BashOperator and PythonOperator
>>    [AIRFLOW-3104] .airflowignore feature is not mentioned at all in
>> documentation
>>    [AIRFLOW-3237] Refactor example DAGs
>>    [AIRFLOW-3187] Update airflow.gif file with a slower version
>>    [AIRFLOW-3159] Update Airflow documentation on GCP Logging
>>    [AIRFLOW-3030] Command Line docs incorrect subdir
>>    [AIRFLOW-2990] Docstrings for Hooks/Operators are in incorrect format
>>    [AIRFLOW-3127] Celery SSL Documentation is out-dated
>> 
>>    Bug fixes:
>> 
>>    [AIRFLOW-839] docker_operator.py attempts to log status key without
>> first checking existence
>>    [AIRFLOW-1104] Concurrency check in scheduler should count queued
>> tasks as well as running
>>    [AIRFLOW-1163] Add support for x-forwarded-* headers to support access
>> behind AWS ELB
>>    [AIRFLOW-1195] Cleared tasks in SubDagOperator do not trigger Parent
>> dag_runs
>>    [AIRFLOW-1508] Skipped state not part of State.task_states
>>    [AIRFLOW-1762] Use key_file in SSHHook.create_tunnel()
>>    [AIRFLOW-1837] Differing start_dates on tasks not respected by
>> scheduler.
>>    [AIRFLOW-1874] Support standard SQL in Check, ValueCheck and
>> IntervalCheck BigQuery operators
>>    [AIRFLOW-1917] print() from python operators end up with extra new line
>>    [AIRFLOW-1970] Database cannot be initialized if an invalid fernet key
>> is provided
>>    [AIRFLOW-2145] Deadlock after clearing a running task
>>    [AIRFLOW-2216] Cannot specify a profile for AWS Hook to load with s3
>> config file
>>    [AIRFLOW-2574] initdb fails when mysql password contains percent sign
>>    [AIRFLOW-2707] Error accessing log files from web UI
>>    [AIRFLOW-2716] Replace new Python 3.7 keywords
>>    [AIRFLOW-2744] RBAC app doesn't integrate plugins (blueprints etc)
>>    [AIRFLOW-2772] BigQuery hook does not allow specifying both the
>> partition field name and table name at the same time
>>    [AIRFLOW-2778] Bad Import in collect_dag in DagBag
>>    [AIRFLOW-2786] Variables view fails to render if a variable has an
>> empty key
>>    [AIRFLOW-2799] Filtering UI objects by datetime is broken
>>    [AIRFLOW-2800] Remove airflow/ low-hanging linting errors
>>    [AIRFLOW-2825] S3ToHiveTransfer operator may not may able to handle
>> GZIP file with uppercase ext in S3
>>    [AIRFLOW-2848] dag_id is missing in metadata table "job" for
>> LocalTaskJob
>>    [AIRFLOW-2860] DruidHook: time variable is not updated correctly when
>> checking for timeout
>>    [AIRFLOW-2865] Race condition between on_success_callback and
>> LocalTaskJob's cleanup
>>    [AIRFLOW-2893] Stuck dataflow job due to jobName mismatch.
>>    [AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs
>>    [AIRFLOW-2900] Code not visible for Packaged DAGs
>>    [AIRFLOW-2905] Switch to regional dataflow job service.
>>    [AIRFLOW-2907] Sendgrid - Attachments - ERROR - Object of type 'bytes'
>> is not JSON serializable
>>    [AIRFLOW-2938] Invalid 'extra' field in connection can raise an
>> AttributeError when attempting to edit
>>    [AIRFLOW-2979] Deprecated Celery Option not in Options list
>>    [AIRFLOW-2981] TypeError in dataflow operators when using GCS jar or
>> py_file
>>    [AIRFLOW-2984] Cannot convert naive_datetime when task has a naive
>> start_date/end_date
>>    [AIRFLOW-2994] flatten_results in BigQueryOperator/BigQueryHook should
>> default to None
>>    [AIRFLOW-3002] ValueError in dataflow operators when using GCS jar or
>> py_file
>>    [AIRFLOW-3012] Email on sla miss is send only to first address on the
>> list
>>    [AIRFLOW-3046] ECS Operator mistakenly reports success when task is
>> killed due to EC2 host termination
>>    [AIRFLOW-3064] No output from `airflow test` due to default logging
>> config
>>    [AIRFLOW-3072] Only admin can view logs in RBAC UI
>>    [AIRFLOW-3079] Improve initdb to support MSSQL Server
>>    [AIRFLOW-3089] Google auth doesn't work under http
>>    [AIRFLOW-3099] Errors raised when some blocs are missing in airflow.cfg
>>    [AIRFLOW-3109] Default user permission should contain 'can_clear'
>>    [AIRFLOW-3111] Confusing comments and instructions for log templates
>> in UPDATING.md and default_airflow.cfg
>>    [AIRFLOW-3124] Broken webserver debug mode (RBAC)
>>    [AIRFLOW-3136] Scheduler Failing the Task retries run while processing
>> Executor Events
>>    [AIRFLOW-3138] Migration cc1e65623dc7 creates issues with postgres
>>    [AIRFLOW-3161] Log Url link does not link to task instance logs in
>> RBAC UI
>>    [AIRFLOW-3162] HttpHook fails to parse URL when port is specified
>>    [AIRFLOW-3183] Potential Bug in
>> utils/dag_processing/DagFileProcessorManager.max_runs_reached()
>>    [AIRFLOW-3203] Bugs in DockerOperator & Some operator test scripts
>> were named incorrectly
>>    [AIRFLOW-3238] Dags, removed from the filesystem, are not deactivated
>> on initdb
>>    [AIRFLOW-3268] Cannot pass SSL dictionary to mysql connection via URL
>>    [AIRFLOW-3277] Invalid timezone transition handling for cron schedules
>>    [AIRFLOW-3295] Require encryption in DaskExecutor when certificates
>> are configured.
>>    [AIRFLOW-3297] EmrStepSensor marks cancelled step as successful
>> 
>> 
>> 
>> 
>> 
>> Kaxil Naik
>> 
>> Data Reply
>> Nova South
>> 160 Victoria Street, Westminster
>> London SW1E 5LB - UK
>> phone: +44 (0)20 7730 6000
>> k.naik@reply.com
>> www.reply.com
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message