airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ash Berlin-Taylor <...@apache.org>
Subject Re: API Reference - current confusion and improvement plan
Date Tue, 05 Feb 2019 16:18:29 GMT
> We want to rewrite the `integration.rst` file so that it does not contain
> duplicates from `code.rst ' (API Reference). In the next step, introduce
> the reference API generation based on the source code that will replace the
> `code.rst` file.

:100: Yes please!


Given a number of integrations are across multiple files (n operators, and m hooks) my first
thought is that something in integration.rst, or a file elsewhere in the docs/ tree is the
place to put this.

On epydoc vs a sphinx extension I lean very heavily towards the sphinx extension, as we are
already using much of sphinx.

Can you create a _small_ example of what you'd propse for no.4 (I don't want you to do a lot
of work that might be wasted) 

-ash


> On 5 Feb 2019, at 15:55, Kamil Breguła <kamil.bregula@polidea.com> wrote:
> 
> Hello community,
> 
> While working on the documentation for the GCP operators, my team at
> Polidea encountered some confusion related to the structure of the
> documentation.
> 
> Short story:
> 
> We want to rewrite the `integration.rst` file so that it does not contain
> duplicates from `code.rst ' (API Reference). In the next step, introduce
> the reference API generation based on the source code that will replace the
> `code.rst` file.
> 
> Long story:
> 
> Currently, the documentation contains two places where the description of
> classes related to operators is included. They are `code.rst` and
> `integration.rst` files.
> 
> The `integration.rst` file contains information about integration, in
> particular for Azure: Microsoft Azure, AWS: Amazon Web Services,
> Databricks, GCP: Google Cloud Platform, Qubole. Other integrations,
> however, do not have descriptions.
> 
> The `code.rst` file contains “API Reference” which contains information
> about *all* classes including those included in the file `integration.rst`.
> 
> Such duplication, however, is problematic for several reasons:
> 
>   1.
> 
>   Users may feel lost and may not know which section they should look into.
>   2.
> 
>   Changes must be made in many places which leads to desynchronization.
>   Most often, changes are made only in the source code, so they do not appear
>   in the generated documentation.
>   3.
> 
>   Linking to classes using the `class` directive for Sphinx is
>   inconclusive - if the code is embedded both in `integration.rst` and
>   `code.rst` using the `autoclass` directive, we’re not sure where the user
>   will be navigated.
> 
> 
> There are several solutions::
> 
>   1.
> 
>   Leave it as is. Then we need to agree on which `autoclass` directive
>   should have the `no-index` flags.
>   2.
> 
>   Delete duplicates from the `code.rst` file and add a note about the
>   `integration.rst` file in the `code.rst` file.
>   3.
> 
>   Delete duplicates from the `integration.rst` file and add a note about
>   the `code.rst` file in the `integration.rst` file.
>   4.
> 
>   Delete information from both files and generate the API documentation
>   always based only on the source code. This solution means that we would
>   have to write less documentation.
>   There are ready tools that we can use:
>   1.
> 
>      epydoc - http://epydoc.sourceforge.net/ ;
>      2.
> 
>      autoapi extension for Sphinx - https://github.com/rtfd/sphinx-autoapi
>      ;
>      3.
> 
>      other - https://wiki.python.org/moin/DocumentationTools
> 
> 
> The first, second, third solution does not solve all problems. In
> particular, we still need to complete the `code.rst` and `integration.rst`
> files. The fourth solution solves all problems, but is the most complex. It
> is worth noting that mixing solutions is possible. For example, we can
> delete information from the file `integration.rst` as short term solution
> and start working on creating another form of documentation as a long term
> solution. This is the best option in our opinion.
> 
> I’ve recently done a few activities related to this topic.
> 
> First, I added the noindex flag to autoclass directives for all operators
> in `integration.rst` file. In rare cases (If any), this caused links that
> were previously directed to the file `integration.rst` to be redirected to
> the `code.rst` file. Elements had to be linked using `:class:` instead of a
> section link. Each operator is included in the new section in this file.
> 
> PR: https://github.com/apache/airflow/pull/4585
> <https://github.com/apache/airflow/pull/4585/files>
> 
> Second, I completed the `code.rst` file with the missing classes.
> 
> PR: https://github.com/apache/airflow/pull/4644
> 
> I would like to ask which solution is the best in your opinion? What steps
> should we take to make the documentation more enjoyable?
> 
> Greetings
> 
> Kamil Breguła


Mime
View raw message