New release v2.10.0

* Fixes due to branch being renamed (#71115)
The ansible collection repository correctly renamed their default branch from `master` to `main`, which has caused a number for broken urls. This PR fixes those urls.

(cherry picked from commit fb9c9570d5)

* Docs:  Fix typo (#71119)

(cherry picked from commit cb9336ab6d)

* remove network for 2.10 base porting guide (#71158)

(cherry picked from commit 56748a8060)

* Updating Getting Started with Resources section #68962 (#71102)

* Updating Getting Started with Resources section #68962
* Add links, including Workshops URL #68962

(cherry picked from commit 5f8b45a70e)

* start of 'data manipulation' examples (#46979)

Co-authored-by: Klaus Frank <agowa338@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit f46b124d65)

* toml: Clarify about inventory examples (#71180)

Add a note in toml inventory plugin about three different
inventory examples. Fixes: #67003

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit edac065bd2)

* filters: minor doc fix (#71178)

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 0a7ab396c7)

* docs: 'ansible_play_hosts' lists active hosts, not limited by serial (#71116)

ansible_play_batch lists the currently targeted host(s) in the serial/batch, while
ansible_play_hosts lists all the hosts which will be targeted by the play.

(cherry picked from commit e72e12aa27)

* Fix references to Ansible Collections Overview (#71227)

(cherry picked from commit 19589db10c)

* add another resource module example (#71162)

* Update docs/docsite/rst/network/user_guide/network_resource_modules.rst
Co-authored-by: Nilashish Chakraborty <nilashishchakraborty8@gmail.com>

(cherry picked from commit f4388de14d)

* Adds fest link (#71241)

(cherry picked from commit ae3b8eec12)

* Update release page for ansible and ansible-base (#71229)

* [docs] 2.7 is EOL, add 2.10 which is almost out
- Remove 2.7 support from the maintenance schedule
- Add 2.10 which is in RC and will be out soon enough.
Signed-off-by: Rick Elrod <rick@elrod.me>

* Update docs/docsite/rst/reference_appendices/release_and_maintenance.rst, fix table and separate ansible-base from ansible, fix rstcheck errors, clean up sections, explain the two packages
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Rick Elrod <rick@elrod.me>

(cherry picked from commit 553ccedcd3)

Co-authored-by: Daniel Finneran <dan@thebsdbox.co.uk>
Co-authored-by: Liviu Chircu <liviu@opensips.org>
Co-authored-by: kshitijcode <ikshitijsharma@gmail.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Håkon Solbjørg <hakon@solbj.org>
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>

.github/ISSUE_TEMPLATE/config.yml

@@ -20,7 +20,7 @@ contact_links:
   about: Please ask and answer usage questions here
 - name: ⚡ Working groups
   url: https://github.com/ansible/community/wiki
-  about: Interedted in improving a specific area? Become a part of a working group!
+  about: Interested in improving a specific area? Become a part of a working group!
 - name: 💼 For Enterprise
   url: https://www.ansible.com/products/engine?utm_medium=github&utm_source=issue_template_chooser
-  about: Red Hat offers supported builds of Ansible Engine
+  about: Red Hat offers support for the Ansible Automation Platform

.gitignore

@@ -37,6 +37,8 @@ docs/docsite/rst/cli/ansible.rst
 docs/docsite/rst/dev_guide/collections_galaxy_meta.rst
 docs/docsite/rst/dev_guide/testing/sanity/index.rst.new
 docs/docsite/rst/modules/*.rst
+docs/docsite/rst/collections/*.rst
+docs/docsite/rst/collections/*/*.rst
 docs/docsite/rst/playbooks_directives.rst
 docs/docsite/rst/plugins_by_category.rst
 docs/docsite/rst/plugins/*/*.rst

MANIFEST.in

@@ -32,5 +32,6 @@ recursive-include test/units *
 include Makefile
 include MANIFEST.in
 include changelogs/CHANGELOG*.rst
+include changelogs/changelog.yaml
 recursive-include hacking/build_library *.py
 include hacking/build-ansible.py

Makefile

@@ -275,7 +275,7 @@ linkcheckdocs:
 .PHONY: generate_rst
 generate_rst: lib/ansible/cli/*.py
 	mkdir -p ./docs/man/man1/ ; \
-	PYTHONPATH=./lib $(GENERATE_CLI) --template-file=docs/templates/man.j2 --output-dir=docs/man/man1/ --output-format man lib/ansible/cli/*.py
+	$(GENERATE_CLI) --template-file=docs/templates/man.j2 --output-dir=docs/man/man1/ --output-format man lib/ansible/cli/*.py
 
 
 docs: generate_rst

changelogs/CHANGELOG-v2.10.rst

@@ -0,0 +1,673 @@
+=======================================================
+Ansible Base 2.10 "When the Levee Breaks" Release Notes
+=======================================================
+
+.. contents:: Topics
+
+
+v2.10.0
+=======
+
+Release Summary
+---------------
+
+| Release Date: 2020-08-13
+| `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__
+
+
+Major Changes
+-------------
+
+- Both ansible-doc and ansible-console's help command will error for modules and plugins whose return documentation cannot be parsed as YAML. All modules and plugins passing ``ansible-test sanity --test yamllint`` will not be affected by this.
+- Collections may declare a list of supported/tested Ansible versions for the collection. A warning is issued if a collection does not support the Ansible version that loads it (can also be configured as silent or a fatal error). Collections that do not declare supported Ansible versions do not issue a warning/error.
+- Plugin routing allows collections to declare deprecation, redirection targets, and removals for all plugin types.
+- Plugins that import module_utils and other ansible namespaces that have moved to collections should continue to work unmodified.
+- Routing data built into Ansible 2.10 ensures that 2.9 content should work unmodified on 2.10. Formerly included modules and plugins that were moved to collections are still accessible by their original unqualified names, so long as their destination collections are installed.
+- When deprecations are done in code, they to specify a ``collection_name`` so that deprecation warnings can mention which collection - or ansible-base - is deprecating a feature. This affects all ``Display.deprecated()`` or ``AnsibleModule.deprecate()`` or ``Ansible.Basic.Deprecate()`` calls, and ``removed_in_version``/``removed_at_date`` or ``deprecated_aliases`` in module argument specs.
+- ansible-test now uses a different ``default`` test container for Ansible Collections
+
+Minor Changes
+-------------
+
+- 'Edit on GitHub' link for plugin, cli documentation fixed to navigate to correct plugin, cli source.
+- Add 'auth_url' field to galaxy server config stanzas in ansible.cfg The url should point to the token_endpoint of a Keycloak server.
+- Add --ask-vault-password and --vault-pass-file options to ansible cli commands
+- Add ``--pre`` flag to ``ansible-galaxy collection install`` to allow pulling in the most recent pre-release version of a collection (https://github.com/ansible/ansible/issues/64905)
+- Add a global toggle to control when vars plugins are executed (per task by default for backward compatibility or after importing inventory).
+- Add a new config parameter, WIN_ASYNC_STARTUP_TIMEOUT, which allows configuration of the named pipe connection timeout under Windows when launching async tasks.
+- Add a per-plugin stage option to override the global toggle to control the execution of individual vars plugins (per task, after inventory, or both).
+- Add an additional check for importing journal from systemd-python module (https://github.com/ansible/ansible/issues/60595).
+- Add an example for using var in with_sequence (https://github.com/ansible/ansible/issues/68836).
+- Add new magic variable ``ansible_collection`` that contains the collection name
+- Add new magic variable ``ansible_role_name`` that contains the FQCN of the role
+- Add standard Python 2/3 compatibility boilerplate to setup script, module_utils and docs_fragments which were missing them.
+- Added PopOS as a part of Debian OS distribution family (https://github.com/ansible/ansible/issues/69286).
+- Added hostname support for PopOS in hostname module.
+- Added openEuler OS in RedHat OS Family.
+- Added the ability to set ``DEFAULT_NO_TARGET_SYSLOG`` through the ``ansible_no_target_syslog`` variable on a task
+- Ansible CLI fails with warning if extra_vars parameter is used with filename without @ sign (https://github.com/ansible/ansible/issues/51857).
+- Ansible modules created with ``add_file_common_args=True`` added a number of undocumented arguments which were mostly there to ease implementing certain action plugins. The undocumented arguments ``src``, ``follow``, ``force``, ``content``, ``backup``, ``remote_src``, ``regexp``, ``delimiter``, and ``directory_mode`` are now no longer added. Modules relying on these options to be added need to specify them by themselves. Also, action plugins relying on these extra elements in ``FILE_COMMON_ARGUMENTS`` need to be adjusted.
+- Ansible now allows deprecation by date instead of deprecation by version. This is possible for plugins and modules (``meta/runtime.yml`` and ``deprecated.removed_at_date`` in ``DOCUMENTATION``, instead of ``deprecated.removed_in``), for plugin options (``deprecated.date`` instead of ``deprecated.version`` in ``DOCUMENTATION``), for module options (``removed_at_date`` instead of ``removed_in_version`` in argument spec), and for module option aliases (``deprecated_aliases.date`` instead of ``deprecated_aliases.version`` in argument spec).
+- Ansible should fail with error when non-existing limit file is provided in command line.
+- Ansible.Basic - Added the ability to specify multiple fragments to load in a generic way for modules that use a module_util with fragment options
+- Ansible.Basic.cs - Added support for ``deprecated_aliases`` to deprecated aliases in a standard way
+- Ansible.ModuleUtils.WebRequest - Move username and password aliases out of util to avoid option name collision
+- Change order of arguments in ansible cli to use --ask-vault-password and --vault-password-file by default
+- CollectionRequirement - Add a metadata property to update and retrieve the _metadata attribute.
+- Command module: Removed suggestions to use modules which have moved to collections and out of ansible-base
+- Enable Ansible Collections loader to discover and import collections from ``site-packages`` dir and ``PYTHONPATH``-added locations.
+- Enable testing the AIX platform as a remote OS in ansible-test
+- Flatten the directory hierarchy of modules
+- Ignore plesk-release file while parsing distribution release (https://github.com/ansible/ansible/issues/64101).
+- Openstack inventory script is migrated to ansible-openstack-collection, adjusted the link in documentation accordingly.
+- Openstack inventory script is moved to openstack.cloud from community.general.
+- PowerShell Add-Type - Add an easier way to reference extra types when compiling C# code on PowerShell Core
+- PowerShell Add-Type - Added the ``X86`` and ``AMD64`` preprocessor symbols for conditional compiling
+- Prevent losing useful error information by including both the loop and the conditional error messages (https://github.com/ansible/ansible/issues/66529)
+- Provides additional information about collection namespace name restrictions (https://github.com/ansible/ansible/issues/65151).
+- Raise error when no task file is provided to import_tasks (https://github.com/ansible/ansible/issues/54095).
+- Refactor test_distribution_version testcases.
+- Remove the deprecation message for the ``TRANSFORM_INVALID_GROUP_CHARS`` setting. (https://github.com/ansible/ansible/issues/61889)
+- Removed extras_require support from setup.py (and [azure] extra). Requirements will float with the collections, so it's not appropriate for ansible-base to host requirements for them any longer.
+- Simplify dict2items filter example in loop documentation (https://github.com/ansible/ansible/issues/65505).
+- Templating - Add globals to the jinja2 environment at ``Templar`` instantiation, instead of customizing the template object. Only customize the template object, to disable lookups. (https://github.com/ansible/ansible/pull/69278)
+- Templating - Add support to auto unroll generators produced by jinja2 filters, to prevent the need of explicit use of ``|list`` (https://github.com/ansible/ansible/pull/68014)
+- The plugin loader now keeps track of the collection where a plugin was resolved to, in particular whether the plugin was loaded from ansible-base's internal paths (``ansible.builtin``) or from user-supplied paths (no collection name).
+- The results queue and counter for results are now split for standard / handler results. This allows the governing strategy to be truly independent from the handler strategy, which basically follows the linear methodology.
+- Update required library message with correct grammer in basic.py.
+- Updated inventory script location for EC2, Openstack, and Cobbler after collection (https://github.com/ansible/ansible/issues/68897).
+- Updated inventory script location for infoblox, ec2 and other after collection migration (https://github.com/ansible/ansible/issues/69139).
+- Updates ``ansible_role_names``, ``ansible_play_role_names``, and ``ansible_dependent_role_names`` to include the FQCN
+- Use OrderedDict by default when importing mappings from YAML.
+- Windows - Add a check for the minimum PowerShell version so we can create a friendly error message on older hosts
+- Windows - add deprecation notice in the Windows setup module when running on Server 2008, 2008 R2, and Windows 7
+- `AnsibleModule.fail_json()` has always required that a message be passed in which informs the end user why the module failed.  In the past this message had to be passed as the `msg` keyword argument but it can now be passed as the first positional argument instead.
+- ``AnsibleModule.load_file_common_arguments`` now allows to simply override ``path``.
+- add mechanism for storing warnings and deprecations globally and not attached to an ``AnsibleModule`` object (https://github.com/ansible/ansible/pull/58993)
+- added more ways to configure new uri options in 2.10.
+- ansible-doc - improve suboptions formatting (https://github.com/ansible/ansible/pull/69795).
+- ansible-doc - now indicates if an option is added by a doc fragment from another collection by prepending the collection name, or ``ansible.builtin`` for ansible-base, to the version number.
+- ansible-doc - return values will be properly formatted (https://github.com/ansible/ansible/pull/69796).
+- ansible-galaxy - Add ``download`` option for ``ansible-galaxy collection`` to download collections and their dependencies for an offline install
+- ansible-galaxy - Add a `verify` subcommand to `ansible-galaxy collection`. The collection found on the galaxy server is downloaded to a tempfile to compare the checksums of the files listed in the MANIFEST.json and the FILES.json with the contents of the installed collection.
+- ansible-galaxy - Add installation successful message
+- ansible-galaxy - Added the ability to display the progress wheel through the C.GALAXY_DISPLAY_PROGRESS config option. Also this now defaults to displaying the progress wheel if stdout has a tty.
+- ansible-galaxy - Added the ability to ignore further files and folders using a pattern with the ``build_ignore`` key in a collection's ``galaxy.yml`` (https://github.com/ansible/ansible/issues/59228).
+- ansible-galaxy - Allow installing collections from git repositories.
+- ansible-galaxy - Always ignore the ``tests/output`` directory when building a collection as it is used by ``ansible-test`` for test output (https://github.com/ansible/ansible/issues/59228).
+- ansible-galaxy - Change the output verbosity level of the download message from 3 to 0 (https://github.com/ansible/ansible/issues/70010)
+- ansible-galaxy - Display message if both collections and roles are specified in a requirements file but can't be installed together.
+- ansible-galaxy - Install both collections and roles with ``ansible-galaxy install -r requirements.yml`` in certain scenarios.
+- ansible-galaxy - Requirement entries for collections now support a 'type' key to indicate whether the collection is a galaxy artifact, file, url, or git repo.
+- ansible-galaxy - Support both 'galaxy.yml' and 'galaxy.yaml' files for collections.
+- ansible-galaxy - add ``--token`` argument which is the same as ``--api-key`` (https://github.com/ansible/ansible/issues/65955)
+- ansible-galaxy - add ``collection list`` command for listing installed collections (https://github.com/ansible/ansible/pull/65022)
+- ansible-galaxy - add ``validate_collection_path()`` utility function ()
+- ansible-galaxy - add collections path argument
+- ansible-galaxy - allow role to define dependency requirements that will be only installed by defining them in ``meta/requirements.yml`` (https://github.com/ansible/proposals/issues/57)
+- ansible-test - --docker flag now has an associated --docker-terminate flag which controls if and when the docker container is removed following tests
+- ansible-test - Add a test to prevent ``state=get``
+- ansible-test - Add a test to prevent ``state=list`` and ``state=info``
+- ansible-test - Add a verbosity option for displaying warnings.
+- ansible-test - Add support for Python 3.9.
+- ansible-test - Added CI provider support for Azure Pipelines.
+- ansible-test - Added a ``ansible-test coverage analyze targets filter`` command to filter aggregated coverage reports by path and/or target name.
+- ansible-test - Added a ``ansible-test coverage analyze targets`` command to analyze integration test code coverage by test target.
+- ansible-test - Added support for Ansible Core CI request signing for Shippable.
+- ansible-test - Added support for testing on Fedora 32.
+- ansible-test - General code cleanup.
+- ansible-test - Now includes testing support for RHEL 8.2
+- ansible-test - Provisioning of RHEL instances now includes installation of pinned versions of ``packaging`` and ``pyparsing`` to match the downstream vendored versions.
+- ansible-test - Refactor code to consolidate filesystem access and improve handling of encoding.
+- ansible-test - Refactored CI related logic into a basic provider abstraction.
+- ansible-test - Remove obsolete support for provisioning remote vCenter instances. The supporting services are no longer available.
+- ansible-test - Report the correct line number in the ``yamllint`` sanity test when reporting ``libyaml`` parse errors in module documentation.
+- ansible-test - Support writing compact JSON files instead of formatting and indenting the output.
+- ansible-test - Update Ubuntu 18.04 test container to version 1.13 which includes ``venv``
+- ansible-test - Update ``default-test-container`` to version 1.11, which includes Python 3.9.0a4.
+- ansible-test - Updated the default test containers to include Python 3.9.0b3.
+- ansible-test - Upgrade OpenSUSE containers to use Leap 15.1.
+- ansible-test - Upgrade distro test containers from 1.16.0 to 1.17.0
+- ansible-test - Upgrade from ansible-base-test-container 1.1 to 2.2
+- ansible-test - Upgrade from default-test-container 2.1 to 2.2
+- ansible-test - ``mutually_exclusive``, ``required_if``, ``required_by``, ``required_together`` and ``required_one_of`` in modules are now validated.
+- ansible-test - ``validate-modules`` now also accepts an ISO 8601 formatted date as ``deprecated.removed_at_date``, instead of requiring a version number in ``deprecated.removed_in``.
+- ansible-test - ``validate-modules`` now makes sure that module documentation deprecation removal version and/or date matches with removal version and/or date in meta/runtime.yml.
+- ansible-test - ``validate-modules`` now validates all version numbers in documentation and argument spec. Version numbers for collections are checked for being valid semantic versioning version number strings.
+- ansible-test - add ``validate-modules`` tests for ``removed_in_version`` and ``deprecated_aliases`` (https://github.com/ansible/ansible/pull/66920/).
+- ansible-test - add check for ``print()`` calls in modules and module_utils.
+- ansible-test - added a ``--no-pip-check`` option
+- ansible-test - added a ``--venv-system-site-packages`` option for use with the ``--venv`` option
+- ansible-test - added new ``changelog`` test, which runs if a `antsibull-changelog <https://pypi.org/project/antsibull-changelog/>`_ configuration or files in ``changelogs/fragments/`` are found (https://github.com/ansible/ansible/pull/69313).
+- ansible-test - allow delegation config to specify equivalents to the ``--no-pip-check``, ``--disable-httptester`` and `--no-temp-unicode`` options
+- ansible-test - allow sanity tests to check for optional errors by specifying ``--enable-optional-errors`` (https://github.com/ansible/ansible/pull/66920/).
+- ansible-test - also run the ``ansible-doc`` sanity test with ``--json`` to ensure that the documentation does not contain something that cannot be exported as JSON (https://github.com/ansible/ansible/issues/69238).
+- ansible-test - enable deprecated version testing for modules and ``module.deprecate()`` calls (https://github.com/ansible/ansible/pull/66920/).
+- ansible-test - extend alias validation.
+- ansible-test - fixed ``units`` command with ``--docker`` to (mostly) work under podman
+- ansible-test - improve module validation so that ``default``, ``sample`` and ``example`` contain JSON values and not arbitrary YAML values, like ``datetime`` objects or dictionaries with non-string keys.
+- ansible-test - module validation will now consider arguments added by ``add_file_common_arguments=True`` correctly.
+- ansible-test - switch from testing RHEL 8.0 and RHEL 8.1 Beta to RHEL 8.1
+- ansible-test - the argument spec of modules is now validated by a YAML schema.
+- ansible-test - the module validation code now checks whether ``elements`` documentation for options matches the argument_spec.
+- ansible-test - the module validation code now checks whether ``elements`` is defined when ``type=list``
+- ansible-test - the module validation code now checks whether ``requirement`` for options is documented correctly.
+- ansible-test add pyparsing constraint for Python 2.x to avoid compatibility issues with the upcoming pyparsing 3 release
+- ansible-test defaults to redacting sensitive values (disable with the ``--no-redact`` option)
+- ansible-test has been updated to use ``default-test-container:1.13`` which includes fewer Python requirements now that most modules and tests have been migrated to collections.
+- ansible-test no longer detects ``git`` submodule directories as files.
+- ansible-test no longer provides a ``--tox`` option. Use the ``--venv`` option instead. This only affects testing the Ansible source. The feature was never available for Ansible Collections or when running from an Ansible install.
+- ansible-test no longer tries to install sanity test dependencies on unsupported Python versions
+- ansible-test now checks for the minimum and maximum supported versions when importing ``coverage``
+- ansible-test now filters out unnecessary warnings and messages from pip when installing its own requirements
+- ansible-test now has a ``--list-files`` option to list files using the ``env`` command.
+- ansible-test now includes the ``pylint`` plugin ``mccabe`` in optional sanity tests enabled with ``--enable-optional-errors``
+- ansible-test now places the ansible source and collections content in separate directories when using the ``--docker`` or ``--remote`` options.
+- ansible-test now provides a more helpful error when loading coverage files created by ``coverage`` version 5 or later
+- ansible-test now supports provisioning of network resources when testing network collections
+- ansible-test now supports skip aliases in the format ``skip/{arch}/{platform}`` and ``skip/{arch}/{platform}/{version}`` where ``arch`` can be ``power``. These aliases are only effective for the ``--remote`` option.
+- ansible-test now supports skip aliases in the format ``skip/{platform}/{version}`` for the ``--remote`` option. This is preferred over the older ``skip/{platform}{version}`` format which included no ``/`` between the platform and version.
+- ansible-test now supports testing against RHEL 7.8 when using the ``--remote`` option.
+- ansible-test now supports the ``--remote power/centos/7`` platform option.
+- ansible-test now validates the schema of ansible_builtin_runtime.yml and a collections meta/runtime.yml file.
+- ansible-test provides clearer error messages when failing to detect the provider to use with the ``--remote`` option.
+- ansible-test provisioning of network devices for ``network-integration`` has been updated to use collections.
+- ansible_native_concat() - use ``to_text`` function rather than Jinja2's ``text_type`` which has been removed in Jinja2 master branch.
+- apt - Implemented an exponential backoff behaviour when retrying to update the cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
+- apt_repository - Implemented an exponential backoff behaviour when retrying to update the apt cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
+- blockinfile - Update module documentation to clarify insertbefore/insertafter usage.
+- callbacks - Allow modules to return `None` as before/after entries for diff. This should make it easier for modules to report the "not existing" state of the entity they touched.
+- combine filter - now accept a ``list_merge`` argument which modifies its behaviour when the hashes to merge contain arrays/lists.
+- conditionals - change the default of CONDITIONAL_BARE_VARS to False (https://github.com/ansible/ansible/issues/70682).
+- config - accept singular version of ``collections_path`` ini setting and ``ANSIBLE_COLLECTIONS_PATH`` environment variable setting
+- core filters - Adding ``path_join`` filter to the core filters list
+- debconf - add a note about no_log=True since module might expose sensitive information to logs (https://github.com/ansible/ansible/issues/32386).
+- default_callback - moving 'check_mode_markers' documentation in default_callback doc_fragment (https://github.com/ansible-collections/community.general/issues/565).
+- distro - Update bundled version of distro from 1.4.0 to 1.5.0
+- dnf - Properly handle idempotent transactions with package name wildcard globs (https://github.com/ansible/ansible/issues/62809)
+- dnf - Properly handle module AppStreams that don't define stream (https://github.com/ansible/ansible/issues/63683)
+- dnf param to pass allowerasing
+- downstream packagers may install packages under ansible._vendor, which will be added to head of sys.path at ansible package load
+- file - specifying ``src`` without ``state`` is now an error
+- get_bin_path() - change the interface to always raise ``ValueError`` if the command is not found (https://github.com/ansible/ansible/pull/56813)
+- get_url - Remove deprecated string format support for the headers option (https://github.com/ansible/ansible/issues/61891)
+- git - added an ``archive_prefix`` option to set a prefix to add to each file path in archive
+- host_group_vars plugin - Require whitelisting and whitelist by default.
+- new magic variable - ``ansible_config_file`` - full path of used Ansible config file
+- package_facts.py - Add support for Pacman package manager.
+- pipe lookup - update docs for Popen with shell=True usages (https://github.com/ansible/ansible/issues/70159).
+- plugin loader - Add MODULE_IGNORE_EXTS config option to skip over certain extensions when looking for script and binary modules.
+- powershell (shell plugin) - Fix `join_path` to support UNC paths (https://github.com/ansible/ansible/issues/66341)
+- regexp_replace filter - add multiline support for regex_replace filter (https://github.com/ansible/ansible/issues/61985)
+- rename ``_find_existing_collections()`` to ``find_existing_collections()`` to reflect its use across multiple files
+- reorganized code for the ``ansible-test coverage`` command for easier maintenance and feature additions
+- service_facts - Added undocumented 'indirect' and 'static' as service status (https://github.com/ansible/ansible/issues/69752).
+- ssh - connection plugin now supports a new variable ``sshpass_prompt`` which gets passed to ``sshpass`` allowing the user to set a custom substring to search for a password prompt (requires sshpass 1.06+)
+- systemd - default scope is now explicitly "system"
+- tests - Add new ``truthy`` and ``falsy`` jinja2 tests to evaluate the truthiness or falsiness of a value
+- to_nice_json filter - Removed now-useless exception handler
+- to_uuid - add a named parameter to let the user optionally set a custom namespace
+- update ansible-test default-test-container from version 1.13 to 1.14, which includes an update from Python 3.9.0a6 to Python 3.9.0b1
+- update ansible-test default-test-container from version 1.9.1 to 1.9.2
+- update ansible-test default-test-container from version 1.9.2 to 1.9.3
+- update ansible-test default-test-container from version 1.9.3 to 1.10.1
+- update ansible-test images to 1.16.0, which includes system updates and pins CentOS versions
+- uri/galaxy - Add new ``prepare_multipart`` helper function for creating a ``multipart/form-data`` body (https://github.com/ansible/ansible/pull/69376)
+- url_lookup_plugin - add parameters to match what is available in ``module_utils/urls.py``
+- user - allow groups, append parameters with local
+- user - usage of ``append: True`` without setting a list of groups. This is currently a no-op with a warning, and will change to an error in 2.14. (https://github.com/ansible/ansible/pull/65795)
+- validate-modules checks for deprecated in collections against meta/runtime.yml
+- validation - Sort missing parameters in exception message thrown by check_required_arguments
+- vars plugins - Support vars plugins in collections by adding the ability to whitelist plugins.
+- vars_prompt - throw error when encountering unsupported key
+- win_package - Added proxy support for retrieving packages from a URL - https://github.com/ansible/ansible/issues/43818
+- win_package - Added support for ``.appx``, ``.msix``, ``.appxbundle``, and ``.msixbundle`` package - https://github.com/ansible/ansible/issues/50765
+- win_package - Added support for ``.msp`` packages - https://github.com/ansible/ansible/issues/22789
+- win_package - Added support for specifying the HTTP method when getting files from a URL - https://github.com/ansible/ansible/issues/35377
+- win_package - Read uninstall strings from the ``QuietUninstallString`` if present to better support argumentless uninstalls of registry based packages.
+- win_package - Scan packages in the current user's registry hive - https://github.com/ansible/ansible/issues/45950
+- windows collections - Support relative module util imports in PowerShell modules and module_utils
+
+Deprecated Features
+-------------------
+
+- Using the DefaultCallback without the correspodning doc_fragment or copying the documentation.
+- hash_behaviour - Deprecate ``hash_behaviour`` for future removal.
+- script inventory plugin - The 'cache' option is deprecated and will be removed in 2.12. Its use has been removed from the plugin since it has never had any effect.
+
+Removed Features (previously deprecated)
+----------------------------------------
+
+- core - remove support for ``check_invalid_arguments`` in ``AnsibleModule``, ``AzureModule`` and ``UTMModule``.
+
+Security Fixes
+--------------
+
+- **security issue** - Convert CLI provided passwords to text initially, to prevent unsafe context being lost when converting from bytes->text during post processing of PlayContext. This prevents CLI provided passwords from being incorrectly templated (CVE-2019-14856)
+- **security issue** - Redact cloud plugin secrets in ansible-test when running integration tests using cloud plugins. Only present in 2.9.0b1.
+- **security issue** - TaskExecutor - Ensure we don't erase unsafe context in TaskExecutor.run on bytes. Only present in 2.9.0beta1 (https://github.com/ansible/ansible/issues/62237)
+- **security issue** - The ``subversion`` module provided the password via the svn command line option ``--password`` and can be retrieved from the host's /proc/<pid>/cmdline file. Update the module to use the secure ``--password-from-stdin`` option instead, and add a warning in the module and in the documentation if svn version is too old to support it. (CVE-2020-1739)
+- **security issue** - Update ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` to maintain unsafe context by overriding ``.encode`` and ``.decode``. This prevents future issues with ``to_text``, ``to_bytes``, or ``to_native`` removing the unsafe wrapper when converting between string types (CVE-2019-14856)
+- **security issue** - properly hide parameters marked with ``no_log`` in suboptions when invalid parameters are passed to the module (CVE-2019-14858)
+- **security issue** win_unzip - normalize paths in archive to ensure extracted files do not escape from the target directory (CVE-2020-1737)
+- **security_issue** - create temporary vault file with strict permissions when editing and prevent race condition (CVE-2020-1740)
+- Ensure we get an error when creating a remote tmp if it already exists. CVE-2020-1733
+- In fetch action, avoid using slurp return to set up dest, also ensure no dir traversal CVE-2020-1735.
+- Sanitize no_log values from any response keys that might be returned from the uri module (CVE-2020-14330).
+- ansible-galaxy - Error when install finds a tar with a file that will be extracted outside the collection install directory - CVE-2020-10691
+
+Bugfixes
+--------
+
+- **security issue** atomic_move - change default permissions when creating temporary files so they are not world readable (https://github.com/ansible/ansible/issues/67794) (CVE-2020-1736)
+- ActionBase - Add new ``cleanup`` method that is explicitly run by the ``TaskExecutor`` to ensure that the shell plugins ``tmpdir`` is always removed. This change means that individual action plugins need not be responsible for removing the temporary directory, which ensures that we don't have code paths that accidentally leave behind the temporary directory.
+- Add example setting for ``collections_paths`` parameter to ``examples/ansible.cfg``
+- Add missing gcp modules to gcp module defaults group
+- Added support for Flatcar Container Linux in distribution and hostname modules. (https://github.com/ansible/ansible/pull/69627)
+- Added support for OSMC distro in hostname module (https://github.com/ansible/ansible/issues/66189).
+- Address compat with rpmfluff-0.6 for integration tests
+- Address the deprecation of the use of stdlib distutils in packaging. It's a short-term hotfix for the problem (https://github.com/ansible/ansible/issues/70456, https://github.com/pypa/setuptools/issues/2230, https://github.com/pypa/setuptools/commit/bd110264)
+- Allow TypeErrors on Undefined variables in filters to be handled or deferred when processing for loops.
+- Allow tasks to notify a fqcn handler name (https://github.com/ansible/ansible/issues/68181)
+- An invalid value is hard to track down if you don't know where it came from, return field name instead.
+- Ansible output now uses stdout to determine column width instead of stdin
+- Ansible.Basic - Fix issue when setting a ``no_log`` parameter to an empty string - https://github.com/ansible/ansible/issues/62613
+- Ansible.ModuleUtils.WebRequest - actually set no proxy when ``use_proxy: no`` is set on a Windows module - https://github.com/ansible/ansible/issues/68528
+- AnsibleDumper - Add a representer for AnsibleUnsafeBytes (https://github.com/ansible/ansible/issues/62562).
+- AnsibleModule.run_command() - set ``close_fds`` to ``False`` on Python 2 if ``pass_fds`` are passed to ``run_command()``. Since ``subprocess.Popen()`` on Python 2 does not have the ``pass_fds`` option, there is no way to exclude a specific list of file descriptors from being closed.
+- Avoid bare select() for running commands to avoid too large file descriptor numbers failing tasks
+- Avoid running subfunctions that are passed to show_vars function when it will be a noop.
+- By passing the module_tmpdir as a parameter in the write_ssh_wrapper function instead of initalizing module_tmpdir via get_module_path()
+- CLI - the `ANSIBLE_PLAYBOOK_DIR` envvar or `playbook_dir` config can now substitute for the --playbook-dir arg on CLIs that support it (https://github.com/ansible/ansible/issues/59464)
+- Check NoneType for raw_params before proceeding in include_vars (https://github.com/ansible/ansible/issues/64939).
+- Collections - Allow a collection role to call a stand alone role, without needing to explicitly add ``ansible.legacy`` to the collection search order within the collection role. (https://github.com/ansible/ansible/issues/69101)
+- Correctly process raw_params in add_hosts.
+- Create an ``import_module`` compat util, for use across the codebase, to allow collection loading to work properly on Python26
+- DUPLICATE_YAML_DICT_KEY - Fix error output when configuration option DUPLICATE_YAML_DICT_KEY is set to error (https://github.com/ansible/ansible/issues/65366)
+- Do not keep empty blocks in PlayIterator after skipping tasks with tags.
+- Ensure DataLoader temp files are removed at appropriate times and that we observe the LOCAL_TMP setting.
+- Ensure that ``--version`` works with non-ascii ansible project paths (https://github.com/ansible/ansible/issues/66617)
+- Ensure that keywords defined as booleans are correctly interpreting their input, before patch any random string would be interpreted as False
+- Ensure we don't allow ansible_facts subkey of ansible_facts to override top level, also fix 'deprefixing' to prevent key transforms.
+- Fact Delegation - Add ability to indicate which facts must always be delegated. Primarily for ``discovered_interpreter_python`` right now, but extensible later. (https://github.com/ansible/ansible/issues/61002)
+- Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)
+- Fix a bug when a host was not removed from a play after ``meta: end_host`` and as a result the host was still present in ``ansible_play_hosts`` and ``ansible_play_batch`` variables.
+- Fix an issue with the ``fileglob`` plugin where passing a subdirectory of non-existent directory would cause it to fail - https://github.com/ansible/ansible/issues/69450
+- Fix case sensitivity for ``lookup()`` (https://github.com/ansible/ansible/issues/66464)
+- Fix collection install error that happened if a dependency specified dependencies to be null (https://github.com/ansible/ansible/issues/67574).
+- Fix https://github.com/ansible/galaxy-dev/issues/96 Add support for automation-hub authentication to ansible-galaxy
+- Fix incorrect "Could not match supplied host pattern" warning (https://github.com/ansible/ansible/issues/66764)
+- Fix issue git module cannot use custom `key_file` or `ssh_opts` as non-root user on system with noexec `/tmp` (https://github.com/ansible/ansible/issues/30064).
+- Fix issue git module ignores remote_tmp (https://github.com/ansible/ansible/issues/33947).
+- Fix issue where the collection loader tracebacks if ``collections_paths = ./`` is set in the config
+- Fix issue with callbacks ``set_options`` method that was not called with collections
+- Fix label lookup in the default callback for includes (https://github.com/ansible/ansible/issues/65904)
+- Fix regression when ``ansible_failed_task`` and ``ansible_failed_result`` are not defined in the rescue block (https://github.com/ansible/ansible/issues/64789)
+- Fix string parsing of inline vault strings for plugin config variable sources
+- Fix traceback when printing ``HostVars`` on native Jinja2 (https://github.com/ansible/ansible/issues/65365)
+- Fix warning for default permission change when no mode is specified. Follow up to https://github.com/ansible/ansible/issues/67794. (CVE-2020-1736)
+- Fixed a bug with the copy action plugin where mode=preserve was being passed on symlink files and causing a traceback (https://github.com/ansible/ansible/issues/68471).
+- Fixed the equality check for IncludedFiles to ensure they are not accidently merged when process_include_results runs.
+- Fixes ansible-test traceback when plugin author is not a string or a list of strings (https://github.com/ansible/ansible/pull/70507)
+- Fixes in network action plugins load from collections using module prefix (https://github.com/ansible/ansible/issues/65071)
+- Force collection names to be static so that a warning is generated because templating currently does not work (see https://github.com/ansible/ansible/issues/68704).
+- Handle empty extra vars in ansible cli (https://github.com/ansible/ansible/issues/61497).
+- Handle empty roles and empty collections in requirements.yml in ansible-galaxy install command (https://github.com/ansible/ansible/issues/68186).
+- Handle exception encountered while parsing the argument description in module when invoked via ansible-doc command (https://github.com/ansible/ansible/issues/60587).
+- Handle exception when /etc/shadow file is missing or not found, while operating user operation in user module (https://github.com/ansible/ansible/issues/63490).
+- HostVarsVars - Template the __repr__ value (https://github.com/ansible/ansible/issues/64128).
+- JSON Encoder - Ensure we treat single vault encrypted values as strings (https://github.com/ansible/ansible/issues/70784)
+- Make netconf plugin configurable to set ncclient device handler name in netconf plugin (https://github.com/ansible/ansible/pull/65718)
+- Make sure if a collection is supplied as a string that we transform it into a list.
+- Misc typo fixes in various documentation pages.
+- Module arguments in suboptions which were marked as deprecated with ``removed_in_version`` did not result in a warning.
+- On HTTP status code 304, return status_code
+- Plugin Metadata is supposed to have default values.  When the metadata was missing entirely, we were properly setting the defaults.  Fixed the metadata parsing so that the defaults are also set when we were missing just a few fields.
+- Prevent a race condition when running handlers using a combination of the free strategy and include_role.
+- Prevent rewriting nested Block's data in filter_tagged_tasks
+- Prevent templating unused variables for {% include %} (https://github.com/ansible/ansible/issues/68699)
+- Properly handle unicode in ``safe_eval``. (https://github.com/ansible/ansible/issues/66943)
+- Python module_utils finder - refactor logic to eliminate many corner cases, remove recursion, fix base module_utils redirections
+- Remove a temp directory created by wait_for_connection action plugin (https://github.com/ansible/ansible/issues/62407).
+- Remove the unnecessary warning about aptitude not being installed (https://github.com/ansible/ansible/issues/56832).
+- Remove unused Python imports in ``ansible-inventory``.
+- Restore the ability for changed_when/failed_when to function with group_by (#70844).
+- Role Installation - Ensure that a role containing files with non-ascii characters can be installed (https://github.com/ansible/ansible/issues/69133)
+- RoleRequirement - include stderr in the error message if a scm command fails (https://github.com/ansible/ansible/issues/41336)
+- SSH plugin - Improve error message when ssh client is not found on the host
+- Skipping of become for ``network_cli`` connections now works when ``network_cli`` is sourced from a collection.
+- Stop adding the connection variables to the output results
+- Strictly check string datatype for 'tasks_from', 'vars_from', 'defaults_from', and 'handlers_from' in include_role (https://github.com/ansible/ansible/issues/68515).
+- Strip no log values from module response keys (https://github.com/ansible/ansible/issues/68400)
+- TaskExecutor - Handle unexpected errors as failed while post validating loops (https://github.com/ansible/ansible/issues/70050).
+- TaskQueueManager - Explicitly set the mutliprocessing start method to ``fork`` to avoid issues with the default on macOS now being ``spawn``.
+- Template connection variables before using them (https://github.com/ansible/ansible/issues/70598).
+- Templating - Ansible was caching results of Jinja2 expressions in some cases where these expressions could have dynamic results, like password generation (https://github.com/ansible/ansible/issues/34144).
+- Terminal plugins - add "\e[m" to the list of ANSI sequences stripped from device output
+- The `ansible_become` value was not being treated as a boolean value when set in an INI format inventory file (fixes bug https://github.com/ansible/ansible/issues/70476).
+- The ansible-galaxy publish command was using an incorrect URL for v3 servers. The configuration for v3 servers includes part of the path fragment that was added in the new test.
+- The machine-readable changelog ``changelogs/changelog.yaml`` is now contained in the release.
+- Update ActionBase._low_level_execute_command to honor executable (https://github.com/ansible/ansible/issues/68054)
+- Update the warning message for ``CONDITIONAL_BARE_VARS`` to list the original conditional not the value of the original conditional (https://github.com/ansible/ansible/issues/67735)
+- Use ``sys.exit`` instead of ``exit`` in ``ansible-inventory``.
+- Use fqcr from command module invocation using shell module. Fixes https://github.com/ansible/ansible/issues/69788
+- Use hostnamectl command to get current hostname for host while using systemd strategy (https://github.com/ansible/ansible/issues/59438).
+- Using --start-at-task would fail when it attempted to skip over tasks with no name.
+- Validate include args in handlers.
+- Vault - Allow single vault encrypted values to be used directly as module parameters. (https://github.com/ansible/ansible/issues/68275)
+- Vault - Make the single vaulted value ``AnsibleVaultEncryptedUnicode`` class work more like a string by replicating the behavior of ``collections.UserString`` from Python. These changes don't allow it to be considered a string, but most common python string actions will now work as expected. (https://github.com/ansible/ansible/pull/67823)
+- ``AnsibleUnsafe``/``AnsibleContext``/``Templar`` - Do not treat ``AnsibleUndefined`` as being "unsafe" (https://github.com/ansible/ansible/issues/65198)
+- account for empty strings in when splitting the host pattern (https://github.com/ansible/ansible/issues/61964)
+- action plugins - change all action/module delegations to use FQ names while allowing overrides (https://github.com/ansible/ansible/issues/69788)
+- add constraints file for ``anisble_runner`` test since an update to ``psutil`` is now causing test failures
+- add magic/connection vars updates from delegated host info.
+- add parameter name to warning message when values are converted to strings (https://github.com/ansible/ansible/pull/57145)
+- add_host action now correctly shows idempotency/changed status
+- added 'unimplemented' prefix to file based caching
+- added new option for default callback to compat variable to avoid old 3rd party plugins from erroring out.
+- adhoc CLI - when playbook-dir is specified and inside a collection, use default collection logic to resolve modules/actions
+- allow external collections to be created in the 'ansible' collection namespace (https://github.com/ansible/ansible/issues/59988)
+- also strip spaces around config values in pathlist as we do in list types
+- ansiballz - remove '' and '.' from sys.path to fix a permissions issue on OpenBSD with pipelining (#69320)
+- ansible command now correctly sends v2_playbook_on_start to callbacks
+- ansible-connection persists even after playbook run is completed (https://github.com/ansible/ansible/pull/61591)
+- ansible-doc - Allow and give precedence to `removed_at_date` for deprecated modules.
+- ansible-doc - collection name for plugin top-level deprecation was not inserted when deprecating by version (https://github.com/ansible/ansible/pull/70344).
+- ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046).
+- ansible-doc - improve man page formatting to avoid problems when YAML anchors are used (https://github.com/ansible/ansible/pull/70045).
+- ansible-doc - include the collection name in the text output (https://github.com/ansible/ansible/pull/70401).
+- ansible-doc now properly handles removed modules/plugins
+- ansible-galaxy - Default collection install path to first path in COLLECTIONS_PATHS (https://github.com/ansible/ansible/pull/62870)
+- ansible-galaxy - Display proper error when invalid token is used for Galaxy servers
+- ansible-galaxy - Ensure we preserve the new URL when appending ``/api`` for the case where the GET succeeds on galaxy.ansible.com
+- ansible-galaxy - Expand the ``User-Agent`` to include more information and add it to more calls to Galaxy endpoints.
+- ansible-galaxy - Fix ``collection install`` when installing from a URL or a file - https://github.com/ansible/ansible/issues/65109
+- ansible-galaxy - Fix ``multipart/form-data`` body to include extra CRLF (https://github.com/ansible/ansible/pull/67942)
+- ansible-galaxy - Fix issue when compared installed dependencies with a collection having no ``MANIFEST.json`` or an empty version string in the json
+- ansible-galaxy - Fix pagination issue when retrieving role versions for install - https://github.com/ansible/ansible/issues/64355
+- ansible-galaxy - Fix up pagination searcher for collection versions on Automation Hub
+- ansible-galaxy - Fix url building to not truncate the URL (https://github.com/ansible/ansible/issues/61624)
+- ansible-galaxy - Handle the different task resource urls in API responses from publishing collection artifacts to galaxy servers using v2 and v3 APIs.
+- ansible-galaxy - Preserve symlinks when building and installing a collection
+- ansible-galaxy - Remove uneeded verbose messages when accessing local token file
+- ansible-galaxy - Return the HTTP code reason if no error msg was returned by the server - https://github.com/ansible/ansible/issues/64850
+- ansible-galaxy - Send SHA256 hashes when publishing a collection
+- ansible-galaxy - Set ``User-Agent`` to Ansible version when interacting with Galaxy or Automation Hub
+- ansible-galaxy - Treat the ``GALAXY_SERVER_LIST`` config entry that is defined but with no values as an empty list
+- ansible-galaxy - Utilize ``Templar`` for templating skeleton files, so that they have access to Ansible filters/tests/lookups (https://github.com/ansible/ansible/issues/69104)
+- ansible-galaxy - fix a bug where listing a specific role if it was not in the first path failed to find the role
+- ansible-galaxy - fix regression that prenented roles from being listed
+- ansible-galaxy - hide warning during collection installation if other installed collections do not contain a ``MANIFEST.json`` (https://github.com/ansible/ansible/issues/67490)
+- ansible-galaxy - properly list roles when the role name also happens to be in the role path (https://github.com/ansible/ansible/issues/67365)
+- ansible-galaxy - properly show the role description when running offline (https://github.com/ansible/ansible/issues/60167)
+- ansible-galaxy cli - fixed ``--version`` argument
+- ansible-galaxy collection - Preserve executable bit on build and preserve mode on install from what tar member is set to - https://github.com/ansible/ansible/issues/68415
+- ansible-galaxy collection download - fix downloading tar.gz files and collections in git repositories (https://github.com/ansible/ansible/issues/70429)
+- ansible-galaxy collection install - fix fallback mechanism if the AH server did not have the collection requested - https://github.com/ansible/ansible/issues/70940
+- ansible-galaxy role - Fix issue where ``--server`` was not being used for certain ``ansible-galaxy role`` actions - https://github.com/ansible/ansible/issues/61609
+- ansible-galaxy- On giving an invalid subcommand to ansible-galaxy, the help would be shown only for role subcommand (collection subcommand help is not shown). With this change, the entire help for ansible-galaxy (same as ansible-galaxy --help) is displayed along with the help for role subcommand. (https://github.com/ansible/ansible/issues/69009)
+- ansible-inventory - Fix long standing bug not loading vars plugins for group vars relative to the playbook dir when the '--playbook-dir' and '--export' flags are used together.
+- ansible-inventory - Fix regression loading vars plugins. (https://github.com/ansible/ansible/issues/65064)
+- ansible-inventory - Properly hide arguments that should not be shown (https://github.com/ansible/ansible/issues/61604)
+- ansible-inventory - Restore functionality to allow ``--graph`` to be limited by a host pattern
+- ansible-test - Add ``pytest < 6.0.0`` constraint for managed installations on Python 3.x to avoid issues with relative imports.
+- ansible-test - Change detection now properly resolves relative imports instead of treating them as absolute imports.
+- ansible-test - Code cleanup.
+- ansible-test - Disabled the ``duplicate-code`` and ``cyclic-import`` checks for the ``pylint`` sanity test due to inconsistent results.
+- ansible-test - Do not try to validate PowerShell modules ``setup.ps1``, ``slurp.ps1``, and ``async_status.ps1``
+- ansible-test - Do not warn on missing PowerShell or C# util that are in other collections
+- ansible-test - Fix PowerShell module util analysis to properly detect the names of a util when running in a collection
+- ansible-test - Fix regression introduced in https://github.com/ansible/ansible/pull/67063 which caused module_utils analysis to fail on Python 2.x.
+- ansible-test - Fix traceback in validate-modules test when argument_spec is None.
+- ansible-test - Make sure import sanity test virtual environments also remove ``pkg-resources`` if it is not removed by uninstalling ``setuptools``.
+- ansible-test - Remove out-of-date constraint on installing paramiko versions 2.5.0 or later in tests.
+- ansible-test - The ``ansible-doc`` sanity test now works for ``netconf`` plugins.
+- ansible-test - The ``import`` sanity test now correctly blocks access to python modules, not just packages, in the ``ansible`` package.
+- ansible-test - The ``import`` sanity test now correctly provides an empty ``ansible`` package.
+- ansible-test - The shebang sanity test now correctly identifies modules in subdirectories in collections.
+- ansible-test - Updated Python constraints for installing ``coverage`` to resolve issues on multiple Python versions when using the ``--coverage`` option.
+- ansible-test - Updated requirements to limit ``boto3`` and ``botocore`` versions on Python 2.6 to supported versions.
+- ansible-test - Use ``sys.exit`` instead of ``exit``.
+- ansible-test - Use ``virtualenv`` versions before 20 on provisioned macOS instances to remain compatible with an older pip install.
+- ansible-test - avoid use of deprecated junit_xml method
+- ansible-test - bump version of ACME test container. The new version includes updated dependencies.
+- ansible-test - during module validation, handle add_file_common_args only for top-level arguments.
+- ansible-test - during module validation, improve alias handling.
+- ansible-test - for local change detection, allow to specify branch to compare to with ``--base-branch`` for all types of tests (https://github.com/ansible/ansible/pull/69508).
+- ansible-test - improve ``deprecate()`` call checker.
+- ansible-test - integration and unit test change detection now works for filter, lookup and test plugins
+- ansible-test can now install argparse with ``--requirements`` or delegation when the pip version in use is older than version 7.1
+- ansible-test change detection - Run only sanity tests on ``docs/`` and ``changelogs/`` in collections, to avoid triggering full CI runs of integration and unit tests when files in these directories change.
+- ansible-test coverage - Fix the ``--all`` argument when generating coverage reports - https://github.com/ansible/ansible/issues/62096
+- ansible-test import sanity test now consistently reports errors against the file being tested.
+- ansible-test import sanity test now consistently reports warnings as errors.
+- ansible-test import sanity test now properly handles relative imports.
+- ansible-test import sanity test now properly invokes Ansible modules as scripts.
+- ansible-test is now able to find its ``egg-info`` directory when it contains the Ansible version number
+- ansible-test no longer errors reporting coverage when no Python coverage exists. This fixes issues reporting on PowerShell only coverage from collections.
+- ansible-test no longer fails when downloading test results for a collection without a ``tests`` directory when using the ``--docker`` option.
+- ansible-test no longer optimizes setting ``PATH`` by prepending the directory containing the selected Python interpreter when it is named ``python``. This avoids unintentionally making other programs available on ``PATH``, including an already installed version of Ansible.
+- ansible-test no longer tracebacks during change analysis due to processing an empty python file
+- ansible-test no longer tries to install ``coverage`` 5.0+ since those versions are unsupported
+- ansible-test no longer tries to install ``setuptools`` 45+ on Python 2.x since those versions are unsupported
+- ansible-test now always uses the ``--python`` option for ``virtualenv`` to select the correct interpreter when creating environments with the ``--venv`` option
+- ansible-test now correctly collects code coverage on the last task in a play. This should resolve issues with missing code coverage, empty coverage files and corrupted coverage files resulting from early worker termination.
+- ansible-test now correctly enumerates submodules when a collection resides below the repository root
+- ansible-test now correctly excludes the test results temporary directory when copying files from the remote test system to the local system
+- ansible-test now correctly includes inventory files ignored by git when running tests with the ``--docker`` option
+- ansible-test now correctly installs the requirements specified by the collection's unit and integration tests instead of the requirements specified for Ansible's own unit and integration tests
+- ansible-test now correctly recognizes imports in collections when using the ``--changed`` option.
+- ansible-test now correctly rewrites coverage paths for PowerShell files when testing collections
+- ansible-test now creates its integration test temporary directory within the collection so ansible-playbook can properly detect the default collection
+- ansible-test now enables color ``ls`` on a remote host only if the host supports the feature
+- ansible-test now ignores empty ``*.py`` files when analyzing module_utils imports for change detection
+- ansible-test now ignores version control within subdirectories of collections. Previously this condition was an error.
+- ansible-test now ignores warnings when comparing pip versions before and after integration tests run
+- ansible-test now installs sanity test requirements specific to each test instead of installing requirements for all sanity tests
+- ansible-test now installs the correct version of ``cryptography`` with ``--requirements`` or delegation when setuptools is older than version 18.5
+- ansible-test now limits Jinja2 installs to version 2.10 and earlier on Python 2.6
+- ansible-test now limits ``pathspec`` to versions prior to 0.6.0 on Python 2.6 to avoid installation errors
+- ansible-test now limits installation of ``hcloud`` to Python 2.7 and 3.5 - 3.8 since other versions are unsupported
+- ansible-test now limits the version of ``setuptools`` on Python 2.6 to versions older than 37
+- ansible-test now loads the collection loader plugin early enough for ansible_collections imports to work in unit test conftest.py modules
+- ansible-test now preserves existing SSH authorized keys when provisioning a remote host
+- ansible-test now properly activates the vcenter plugin for vcenter tests when docker is available
+- ansible-test now properly activates virtual environments created using the --venv option
+- ansible-test now properly creates a virtual environment using ``venv`` when running in a ``virtualenv`` created virtual environment
+- ansible-test now properly excludes the ``tests/output/`` directory from code coverage
+- ansible-test now properly handles creation of Python execv wrappers when the selected interpreter is a script
+- ansible-test now properly handles enumeration of git submodules. Enumeration is now done with ``git submodule status --recursive`` without specifying ``.`` for the path, since that could cause the command to fail. Instead, relative paths outside the current directory are filtered out of the results. Errors from ``git`` commands will now once again be reported as errors instead of warnings.
+- ansible-test now properly handles warnings for removed modules/plugins
+- ansible-test now properly ignores the ``tests/output//`` directory when not using git
+- ansible-test now properly installs requirements for multiple Python versions when running sanity tests
+- ansible-test now properly recognizes modules and module_utils in collections when using the ``blacklist`` plugin for the ``pylint`` sanity test
+- ansible-test now properly registers its own code in a virtual environment when running from an install
+- ansible-test now properly reports import errors for collections when running the import sanity test
+- ansible-test now properly searches for ``pythonX.Y`` instead of ``python`` when looking for the real python that created a ``virtualenv``
+- ansible-test now properly sets PYTHONPATH for tests when running from an Ansible installation
+- ansible-test now properly sets ``ANSIBLE_PLAYBOOK_DIR`` for integration tests so unqualified collection references work for adhoc ``ansible`` usage
+- ansible-test now properly uses a fresh copy of environment variables for each command invocation to avoid mixing vars between commands
+- ansible-test now shows sanity test doc links when installed (previously the links were only visible when running from source)
+- ansible-test now shows the correct source path instead of ``%s`` for collection role based test targets when the ``-v`` option is used
+- ansible-test now supports submodules using older ``git`` versions which require querying status from the top level directory of the repo.
+- ansible-test now updates SSH keys it generates with newer versions of ssh-keygen to function with Paramiko
+- ansible-test now upgrades ``pip`` with `--requirements`` or delegation as needed when the pip version in use is older than version 7.1
+- ansible-test now uses GNU tar format instead of the Python default when creating payloads for remote systems
+- ansible-test now uses ``pycodestyle`` frozen at version 2.6.0 for consistent test results.
+- ansible-test now uses modules from the ``ansible.windows`` collection for setup and teardown of ``windows-integration`` tests and code coverage
+- ansible-test once again properly collects code coverage for ``ansible-connection``
+- ansible-test validate-modules - Fix arg spec collector for PowerShell to find utils in both a collection and base.
+- ansible-test validate-modules - ``version_added`` on module level was not validated for modules in collections (https://github.com/ansible/ansible/pull/70869).
+- ansible-test validate-modules - return correct error codes ``option-invalid-version-added`` resp. ``return-invalid-version-added`` instead of the wrong error ``deprecation-either-date-or-version`` when an invalid value of ``version_added`` is specified for an option or a return value (https://github.com/ansible/ansible/pull/70869).
+- ansible-test validate-modules sanity test code ``missing-module-utils-import-c#-requirements`` is now ``missing-module-utils-import-csharp-requirements`` (fixes ignore bug).
+- ansible-test validate-modules sanity test code ``multiple-c#-utils-per-requires`` is now ``multiple-csharp-utils-per-requires`` (fixes ignore bug).
+- ansible-test validate-modules sanity test now checks for AnsibleModule initialization instead of module_utils imports, which did not work in many cases.
+- ansible-test validate-modules sanity test now properly handles collections imports using the Ansible collection loader.
+- ansible-test validate-modules sanity test now properly handles relative imports.
+- ansible-test validate-modules sanity test now properly handles sys.exit in modules.
+- ansible-test validate-modules sanity test now properly invokes Ansible modules as scripts.
+- ansible-test windows coverage - Ensure coverage reports are UTF-8 encoded without a BOM
+- ansible-test windows coverage - Output temp files as UTF-8 with BOM to standardise against non coverage runs
+- ansible-vault - Fix ``encrypt_string`` output in a tty when using ``--sdtin-name`` option (https://github.com/ansible/ansible/issues/65121)
+- ansible-vault create - Fix exception on no arguments given
+- api - time.clock is removed in Python 3.8, add backward compatible code (https://github.com/ansible/ansible/issues/70649).
+- apt - Fixed the issue the cache being updated while auto-installing its dependencies even when ``update_cache`` is set to false.
+- apt - include exception message from apt python library in error output
+- assemble - fix decrypt argument in the module (https://github.com/ansible/ansible/issues/65450).
+- assemble module - fix documentation - the remote_src property specified a default value of no but it's actually yes.
+- avoid fatal traceback when a bad FQCN for a callback is supplied in the whitelist (#69401).
+- basic - use PollSelector implementation when DefaultSelector fails (https://github.com/ansible/ansible/issues/70238).
+- become - Fix various plugins that still used play_context to get the become password instead of through the plugin - https://github.com/ansible/ansible/issues/62367
+- blockinfile - fix regression that results in incorrect block in file when the block to be inserted does not end in a line separator (https://github.com/ansible/ansible/pull/69734)
+- blockinfile - preserve line endings on update (https://github.com/ansible/ansible/issues/64966)
+- clean_facts - use correct variable to avoid unnecessary handling of ``AttributeError``
+- code - removes some Python compatibility code for dealing with socket timeouts in ``wait_for``
+- collection loader - ensure Jinja function cache is fully-populated before lookup
+- collection loader - fixed relative imports on Python 2.7, ensure pluginloader caches use full name to prevent names from being clobbered (https://github.com/ansible/ansible/pull/60317)
+- collection metadata - ensure collection loader uses libyaml/CSafeLoader to parse collection metadata if available
+- collection_loader - sort Windows modules below other plugin types so the correct builtin plugin inside a role is selected (https://github.com/ansible/ansible/issues/65298)
+- collections - Handle errors better for filters and tests in collections, where a non-existent collection is specified, or importing the plugin results in an exception (https://github.com/ansible/ansible/issues/66721)
+- combine filter - ``[dict1, [dict2]] | combine`` now raise an error; previously ``combine`` had an undocumented behaviour where it was flattening the list before combining it (https://github.com/ansible/ansible/pull/57894#discussion_r339517518).
+- config - encoding failures on config values should be non-fatal (https://github.com/ansible/ansible/issues/63310)
+- copy - Fix copy modes when using remote_src=yes and src is a directory with trailing slash.
+- copy - Fixed copy module not working in case that remote_src is enabled and dest ends in a / (https://github.com/ansible/ansible/pull/47238)
+- copy - recursive copy with ``remote_src=yes`` now recurses beyond first level. (Fixes https://github.com/ansible/ansible/issues/58284)
+- core - remove unneeded Python version checks.
+- core - replace a compatibility import of pycompat24.literal_eval with ast.literal_eval.
+- core filters - fix ``extract()`` filter when key does not exist in container (https://github.com/ansible/ansible/issues/64957)
+- cron - encode and decode crontab files in UTF-8 explicitly to allow non-ascii chars in cron filepath and job (https://github.com/ansible/ansible/issues/69492)
+- cron and cronvar - use get_bin_path utility to locate the default crontab executable instead of the hardcoded /usr/bin/crontab. (https://github.com/ansible/ansible/pull/59765)
+- cron cronvar - only run ``get_bin_path()`` once
+- cronvar - use correct binary name (https://github.com/ansible/ansible/issues/63274)
+- deal with cases in which just a file is pased and not a path with directories, now fileglob correctly searches in 'files/' subdirs.
+- debug - fixed an issue introduced in Ansible 2.4 where a loop of debug tasks would lose the "changed" status on each item.
+- discovery will NOT update incorrect host anymore when in delegate_to task.
+- display - Improve method of removing extra new line after warnings so it does not break Tower/Runner (https://github.com/ansible/ansible/pull/68517)
+- display - remove extra new line after warnings (https://github.com/ansible/ansible/pull/65199)
+- display - remove leading space when displaying WARNING messages
+- display logging - Fix issue where 3rd party modules will print tracebacks when attempting to log information when ``ANSIBLE_LOG_PATH`` is set - https://github.com/ansible/ansible/issues/65249
+- display logging - Fixed up the logging formatter to use the proper prefixes for ``u=user`` and ``p=process``
+- display logging - Re-added the ``name`` attribute to the log formatter so that the source of the log can be seen
+- dnf - Fix idempotence of `state: installed` (https://github.com/ansible/ansible/issues/64963)
+- dnf - Unified error messages when trying to install a nonexistent package with newer dnf (4.2.18) vs older dnf (4.2.9)
+- dnf - Unified error messages when trying to remove a wildcard name that is not currently installed, with newer dnf (4.2.18) vs older dnf (4.2.9)
+- dnf - enable logging using setup_loggers() API in dnf-4.2.17-6 or later
+- dnf - remove custom ``fetch_rpm_from_url`` method in favor of more general ``ansible.module_utils.urls.fetch_file``.
+- dnf module - Ensure the modules exit_json['msg'] response is always string, not sometimes a tuple.
+- ensure delegated vars can resolve hostvars object and access vars from hostvars[inventory_hostname].
+- ensure we pass on interpreter discovery values to delegated host.
+- env lookup plugin - Fix handling of environment variables values containing utf-8 characters. (https://github.com/ansible/ansible/issues/65298)
+- fact gathering - Display warnings and deprecation messages that are created during the fact gathering phase
+- facts - account for Slackware OS with ``+`` in the name (https://github.com/ansible/ansible/issues/38760)
+- facts - fix detection of virtualization type when dmi product name is KVM Server
+- facts - fix incorrect UTC timestamp in ``iso8601_micro`` and ``iso8601``
+- facts - introduce fact "ansible_processor_nproc" which reflects the number of vcpus available to processes (falls back to the number of vcpus available to the scheduler)
+- file - Removed unreachable code in module
+- file - change ``_diff_peek`` in argument spec to be the correct type, which is ``bool`` (https://github.com/ansible/ansible/issues/59433)
+- file - return ``'state': 'absent'`` when a file does not exist (https://github.com/ansible/ansible/issues/66171)
+- find - clarify description of ``contains`` (https://github.com/ansible/ansible/issues/61983)
+- fix issue in which symlinked collection cannot be listed, though the docs/plugins can be loaded if referenced directly.
+- fix issue with inventory_hostname and delegated host vars mixing on connection settings.
+- fix wrong command line length calculation in ``ansible-console`` when long command inputted
+- for those running uids for invalid users (containers), fallback to uid=<uid> when logging fixes #68007
+- free strategy - Include failed hosts when filtering notified hosts for handlers. The strategy base should determine whether or not to run handlers on those hosts depending on whether forcing handlers is enabled (https://github.com/ansible/ansible/issues/65254).
+- galaxy - Fix an AttributeError on ansible-galaxy install with an empty requirements.yml (https://github.com/ansible/ansible/issues/66725).
+- get_url - Don't treat no checksum as a checksum match (https://github.com/ansible/ansible/issues/61978)
+- get_url pass incorrect If-Modified-Since header (https://github.com/ansible/ansible/issues/67417)
+- git - when force=True, apply --force flag to git fetches as well
+- group - The group module was not correctly detecting whether a local group is existing or not with local set to yes if the same group exists in a non local group repository e.g. LDAP. (https://github.com/ansible/ansible/issues/58619)
+- group_by now should correctly refect changed status.
+- hostname - Fixed an issue where the hostname on the cloudlinux 6 server could not be set.
+- hostname - make module work on Manjaro Linux (https://github.com/ansible/ansible/issues/61382)
+- hurd - Address FIXMEs. Extract functionality and exit early.
+- if the ``type`` for a module parameter in the argument spec is callable, do not pass ``kwargs`` to avoid errors (https://github.com/ansible/ansible/issues/70017)
+- include_vars - fix stack trace when passing ``dirs`` in an ad-hoc command (https://github.com/ansible/ansible/issues/62633)
+- interpreter discovery will now use correct vars (from delegated host) when in delegate_to task.
+- junit callback - avoid use of deprecated junit_xml method
+- lineinfile - add example of using alternative backrefs syntax (https://github.com/ansible/ansible/issues/42794)
+- lineinfile - don't attempt mkdirs when path doesn't contain directory path
+- lineinfile - fix bug that caused multiple line insertions (https://github.com/ansible/ansible/issues/58923).
+- lineinfile - fix not subscriptable error in exception handling around file creation
+- lineinfile - properly handle inserting a line when backrefs are enabled and the line already exists in the file (https://github.com/ansible/ansible/issues/63756)
+- lineinfile - use ``module.tmpdir`` to allow configuration of the remote temp directory (https://github.com/ansible/ansible/issues/68218)
+- lineinfile - use correct index value when inserting a line at the end of a file (https://github.com/ansible/ansible/issues/63684)
+- loops - Do not indiscriminately mark loop items as unsafe, only apply unsafe to ``with_`` style loops. The items from ``loop`` should not be explicitly wrapped in unsafe. The underlying templating mechanism should dictate this. (https://github.com/ansible/ansible/issues/64379)
+- make ``no_log=False`` on a module option silence the ``no_log`` warning (https://github.com/ansible/ansible/issues/49465 https://github.com/ansible/ansible/issues/64656)
+- match docs for ssh and ensure pipelining is configurable per connection plugin.
+- module executor - Address issue where changes to Ansiballz module code, change the behavior of module execution as it pertains to ``__file__`` and ``sys.modules`` (https://github.com/ansible/ansible/issues/64664)
+- module_defaults - support candidate action names for relocated content
+- module_defaults - support short group names for content relocated to collections
+- now correclty merge and not just overwrite facts when gathering using multiple modules.
+- objects - Remove FIXME comment because no fix is needed.
+- optimize 'smart' detection from being run over and over and preferably do it at config time.
+- package_facts - fix value of ``vital`` attribute which is returned when ``pkg`` manager is used
+- package_facts - use module warnings rather than a custom implementation for reporting warnings
+- packaging_yum - replace legacy file handling with a file manager.
+- paramiko - catch and handle exception to prevent stack trace when running in FIPS mode
+- paramiko_ssh - Removed redundant conditional statement in ``_parse_proxy_command`` that always evaluated to True.
+- paramiko_ssh - improve authentication error message so it is less confusing
+- paramiko_ssh - optimized file handling by using a context manager.
+- pause - handle exception when there is no stdout (https://github.com/ansible/ansible/pull/47851)
+- pip - The virtualenv_command option can now include arguments without requiring the full path to the binary. (https://github.com/ansible/ansible/issues/52275)
+- pip - check_mode with ``state: present`` now returns the correct state for pre-release versioned packages
+- playbooks - detect and propagate failures in ``always`` blocks after ``rescue`` (https://github.com/ansible/ansible/issues/70000)
+- plugins - Allow ensure_type to decrypt the value for string types (and implicit string types) when value is an inline vault.
+- psexec - Fix issue where the Kerberos package was not detected as being available.
+- psexec - Fix issue where the ``interactive`` option was not being passed down to the library.
+- reboot - Add support for the runit init system, used on Void Linux, that does not support the normal Linux syntax.
+- reboot, win_reboot - add ``boot_time_command`` parameter to override the default command used to determine whether or not a system was rebooted (https://github.com/ansible/ansible/issues/58868)
+- remove update/restore of vars from play_context as it is now redundant.
+- replace use of deprecated functions from ``ansible.module_utils.basic``.
+- reset logging level to INFO due to CVE-2019-14846.
+- roles - Ensure that ``allow_duplicates: true`` enables to run single role multiple times (https://github.com/ansible/ansible/issues/64902)
+- runas - Fix the ``runas`` ``become_pass`` variable fallback from ``ansible_runas_runas`` to ``ansible_runas_pass``
+- service_facts - Now correctly parses systemd list-unit-files for systemd >=245
+- setup - properly detect yum package manager for IBM i.
+- setup - service_mgr - detect systemd even if it isn't running, such as during a container build
+- shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).
+- shell cmd - Properly escape double quotes in the command argument
+- splunk httpapi plugin - switch from splunk.enterprise_security to splunk.es in runtime.yml to reflect upstream change of Collection Name
+- ssh connection plugin - use ``get_option()`` rather than ``_play_context`` to ensure ``ANSBILE_SSH_ARGS`` are applied properly (https://github.com/ansible/ansible/issues/70437)
+- synchronize - allow data to be passed between two managed nodes when using the docker connection plugin (https://github.com/ansible/ansible/pull/65698)
+- synchronize - fix password authentication on Python 2 (https://github.com/ansible/ansible/issues/56629)
+- sysctl - Remove FIXME comments to avoid confusion
+- systemd - don't require systemd to be running to enable/disable or mask/unmask units
+- systemd - the module should fail in check_mode when service not found on host (https://github.com/ansible/ansible/pull/68136).
+- sysvinit - Add missing parameter ``module`` in call to ``daemonize()``.
+- template lookup - ensure changes to the templar in the lookup, do not affect the templar context outside of the lookup (https://github.com/ansible/ansible/issues/60106)
+- template lookup - fix regression when templating hostvars (https://github.com/ansible/ansible/issues/63940)
+- the default parsing will now show existing JSON errors and not just YAML (last attempted), also we avoid YAML parsing when we know we only want JSON issue
+- throttle: the linear strategy didn't always stuck with the throttle limit
+- unarchive - Remove incorrect and unused function arguments.
+- unsafe_proxy - Ensure that data within a tuple is marked as unsafe (https://github.com/ansible/ansible/issues/65722)
+- update ``user`` module to support silencing ``no_log`` warnings in the future (see: https://github.com/ansible/ansible/pull/64733)
+- uri - Don't return the body even if it failed (https://github.com/ansible/ansible/issues/21003)
+- user - allow 13 asterisk characters in password field without warning
+- user - don't create home directory and missing parents when create_home == false (https://github.com/ansible/ansible/pull/70600).
+- user - fix comprasion on macOS so module does not improperly report a change (https://github.com/ansible/ansible/issues/62969)
+- user - fix stack trace on AIX when attempting to parse shadow file that does not exist (https://github.com/ansible/ansible/issues/62510)
+- user - on systems using busybox, honor the ``on_changed`` parameter to prevent unnecessary password changing (https://github.com/ansible/ansible/issues/65711)
+- user - update docs to reflect proper way to remove account from all groups
+- validate-modules - Fix hang when inspecting module with a delegate args spec type
+- virtual facts - detect generic container environment based on non-empty "container" env var
+- wait_for_connection - with pipelining enabled, interpreter discovery would fail if the first connection attempt was not successful
+- win setup - Fix redirection path for the windows setup module
+- win_exec_wrapper - Be more defensive when it comes to getting unhandled exceptions
+- win_package - Handle quoted and unquoted strings in the registry ``UninstallString`` value - https://github.com/ansible/ansible/issues/40973
+- win_uri win_get_url - Fix the behaviour of ``follow_redirects: safe`` to actual redirect on ``GET`` and ``HEAD`` requests - https://github.com/ansible/ansible/issues/65556
+- windows async - use full path when calling PowerShell to reduce reliance on environment vars being correct - https://github.com/ansible/ansible/issues/70655
+- windows environment - Support env vars that contain the unicode variant of single quotes - https://github.com/ansible-collections/ansible.windows/issues/45
+- winrm - preserve winrm forensic data on put_file failures
+- yum - fix bug that caused ``enablerepo`` to not be honored when used with disablerepo all wildcard/glob (https://github.com/ansible/ansible/issues/66549)
+- yum - fixed the handling of releasever parameter
+- yum - performance bugfix, the YumBase object was being  instantiated multiple times unnecessarily, which lead to considerable overhead when operating against large sets of packages.
+- yum - yum tasks can no longer end up running non-yum modules
+- yum/dnf - check type of elements in a name
+
+New Plugins
+-----------
+
+Lookup
+~~~~~~
+
+- unvault - read vaulted file(s) contents

changelogs/fragments/32386_debconf_password.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- debconf - add a note about no_log=True since module might expose sensitive information to logs (https://github.com/ansible/ansible/issues/32386).

changelogs/fragments/38760-slackware-os-dist.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - facts - account for Slackware OS with ``+`` in the name (https://github.com/ansible/ansible/issues/38760)

changelogs/fragments/565_default_callback.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- default_callback - moving 'check_mode_markers' documentation in default_callback doc_fragment (https://github.com/ansible-collections/community.general/issues/565).

changelogs/fragments/67794-atomic_move-default-perms.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >
+    **security issue** atomic_move - change default permissions when creating
+    temporary files so they are not world readable (https://github.com/ansible/ansible/issues/67794) (CVE-2020-1736)

changelogs/fragments/67794-default-permissions-warning-fix.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >
+    Fix warning for default permission change when no mode is specified. Follow up
+    to https://github.com/ansible/ansible/issues/67794. (CVE-2020-1736)

changelogs/fragments/68275-vault-module-args.yml

@@ -0,0 +1,3 @@
+bugfixes:
+- Vault - Allow single vault encrypted values to be used directly as module
+  parameters. (https://github.com/ansible/ansible/issues/68275)

changelogs/fragments/69578-shell-remote_tmp-quoting.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).

changelogs/fragments/70000-playbook-detect-failure-in-always.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - playbooks - detect and propagate failures in ``always`` blocks after ``rescue`` (https://github.com/ansible/ansible/issues/70000)

changelogs/fragments/70017-avoid-params-to-callable-checkers.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >-
+    if the ``type`` for a module parameter in the argument spec is callable,
+    do not pass ``kwargs`` to avoid errors (https://github.com/ansible/ansible/issues/70017)

changelogs/fragments/70045-ansible-doc-yaml-anchors.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - improve man page formatting to avoid problems when YAML anchors are used (https://github.com/ansible/ansible/pull/70045)."

changelogs/fragments/70046-ansible-doc-description-crash.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046)."

changelogs/fragments/70099-make-apt-errors-more-transparent.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - apt - include exception message from apt python library in error output

changelogs/fragments/70122-improve-error-message-ssh-client-is-not-found.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - SSH plugin - Improve error message when ssh client is not found on the host

changelogs/fragments/70168-fix-delegate_facts-without-interpreter-set.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - "Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)"

changelogs/fragments/70238_selector.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- basic - use PollSelector implementation when DefaultSelector fails (https://github.com/ansible/ansible/issues/70238).

changelogs/fragments/70240-fix-fatal-post_validate-error.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - TaskExecutor - Handle unexpected errors as failed while post validating loops (https://github.com/ansible/ansible/issues/70050).

changelogs/fragments/70261_pipe_lookup.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- pipe lookup - update docs for Popen with shell=True usages (https://github.com/ansible/ansible/issues/70159).

changelogs/fragments/70344-plugin-deprecation-collection-name.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - collection name for plugin top-level deprecation was not inserted when deprecating by version (https://github.com/ansible/ansible/pull/70344)."

changelogs/fragments/70426-allow-non-ascii-chars-in-cron.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - cron - encode and decode crontab files in UTF-8 explicitly to allow non-ascii chars in cron filepath and job (https://github.com/ansible/ansible/issues/69492)

changelogs/fragments/70437-ssh-args.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >
+    ssh connection plugin - use ``get_option()`` rather than ``_play_context`` to
+    ensure ``ANSBILE_SSH_ARGS`` are applied properly (https://github.com/ansible/ansible/issues/70437)

changelogs/fragments/70465-assemble-fix-decrypt-argument.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - assemble - fix decrypt argument in the module (https://github.com/ansible/ansible/issues/65450).

changelogs/fragments/70484-bool-ansible-become.yaml

@@ -0,0 +1,3 @@
+bugfixes:
+  - The `ansible_become` value was not being treated as a boolean value when set in an INI format
+    inventory file (fixes bug https://github.com/ansible/ansible/issues/70476).

changelogs/fragments/70507-validate-null-author.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Fixes ansible-test traceback when plugin author is not a string or a list of strings (https://github.com/ansible/ansible/pull/70507)

changelogs/fragments/70524-fix-download-collections.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-galaxy collection download - fix downloading tar.gz files and collections in git repositories (https://github.com/ansible/ansible/issues/70429)

changelogs/fragments/70525-setuptools-disutils-reorder.yml

@@ -0,0 +1,7 @@
+bugfixes:
+- >
+  Address the deprecation of the use of stdlib
+  distutils in packaging. It's a short-term hotfix for the problem
+  (https://github.com/ansible/ansible/issues/70456,
+  https://github.com/pypa/setuptools/issues/2230,
+  https://github.com/pypa/setuptools/commit/bd110264)

changelogs/fragments/70600-user-module-dont-create-home-when-create_home-is-false.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "user - don't create home directory and missing parents when create_home == false (https://github.com/ansible/ansible/pull/70600)."

changelogs/fragments/70649_time_clock.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- api - time.clock is removed in Python 3.8, add backward compatible code (https://github.com/ansible/ansible/issues/70649).

changelogs/fragments/70657-template-connection-vars.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Template connection variables before using them (https://github.com/ansible/ansible/issues/70598).

changelogs/fragments/70683-terminal-ansi-re.yaml

@@ -0,0 +1,3 @@
+---
+bugfixes:
+  - Terminal plugins - add "\e[m" to the list of ANSI sequences stripped from device output

changelogs/fragments/70704-void-linux-reboot.yml

@@ -0,0 +1,3 @@
+bugfixes:
+ - reboot - Add support for the runit init system, used on Void Linux, that
+   does not support the normal Linux syntax.

changelogs/fragments/70762-sanitize-uri-keys.yml

@@ -0,0 +1,2 @@
+security_fixes:
+  - Sanitize no_log values from any response keys that might be returned from the uri module (CVE-2020-14330).

changelogs/fragments/70784-vault-is-string.yml

@@ -0,0 +1,3 @@
+bugfixes:
+- JSON Encoder - Ensure we treat single vault encrypted values as strings
+  (https://github.com/ansible/ansible/issues/70784)

changelogs/fragments/70869-ansible-test-validate-modules-version-added.yml

@@ -0,0 +1,3 @@
+bugfixes:
+- "ansible-test validate-modules - return correct error codes ``option-invalid-version-added`` resp. ``return-invalid-version-added`` instead of the wrong error ``deprecation-either-date-or-version`` when an invalid value of ``version_added`` is specified for an option or a return value (https://github.com/ansible/ansible/pull/70869)."
+- "ansible-test validate-modules - ``version_added`` on module level was not validated for modules in collections (https://github.com/ansible/ansible/pull/70869)."

changelogs/fragments/ansible-boilerplate.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - Add standard Python 2/3 compatibility boilerplate to setup script, module_utils and docs_fragments which were missing them.

changelogs/fragments/ansible-doc-collection-name.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - include the collection name in the text output (https://github.com/ansible/ansible/pull/70401)."

changelogs/fragments/ansible-galaxy-stdout.yml

@@ -0,0 +1,3 @@
+minor_changes:
+  - ansible-galaxy - Change the output verbosity level of the download message from 3 to 0 (https://github.com/ansible/ansible/issues/70010)
+  - ansible-galaxy - Add installation successful message

changelogs/fragments/ansible-test-plugin-classification.yml

@@ -0,0 +1,2 @@
+bugfixes:
+    - ansible-test - integration and unit test change detection now works for filter, lookup and test plugins

changelogs/fragments/ansible-test-pytest-cap.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Add ``pytest < 6.0.0`` constraint for managed installations on Python 3.x to avoid issues with relative imports.

changelogs/fragments/ansible-test-relative-import-analysis.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - Change detection now properly resolves relative imports instead of treating them as absolute imports.

changelogs/fragments/ansible-test-rhel-requirements.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Provisioning of RHEL instances now includes installation of pinned versions of ``packaging`` and ``pyparsing`` to match the downstream vendored versions.

changelogs/fragments/ansible-test-sanity-ansible-doc.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - The ``ansible-doc`` sanity test now works for ``netconf`` plugins.

changelogs/fragments/ansible-test-sanity-yamllint-lineno.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Report the correct line number in the ``yamllint`` sanity test when reporting ``libyaml`` parse errors in module documentation.

changelogs/fragments/ansible-test-virtualenv-fix.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test now always uses the ``--python`` option for ``virtualenv`` to select the correct interpreter when creating environments with the ``--venv`` option

changelogs/fragments/better_winrm_putfile_error.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - winrm - preserve winrm forensic data on put_file failures

changelogs/fragments/changed_when_group_by.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Restore the ability for changed_when/failed_when to function with group_by (#70844).

changelogs/fragments/changelog-yaml.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "The machine-readable changelog ``changelogs/changelog.yaml`` is now contained in the release."

changelogs/fragments/collection_meta_use_libyaml.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - collection metadata - ensure collection loader uses libyaml/CSafeLoader to parse collection metadata if available

changelogs/fragments/command-module-warnings.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - "Command module: Removed suggestions to use modules which have moved to collections and out of ansible-base"

changelogs/fragments/date-time-facts-fix-utctime.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - facts - fix incorrect UTC timestamp in ``iso8601_micro`` and ``iso8601``

changelogs/fragments/delegate_has_hostvars.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - ensure delegated vars can resolve hostvars object and access vars from hostvars[inventory_hostname].
+  - fix issue with inventory_hostname and delegated host vars mixing on connection settings.
+  - add magic/connection vars updates from delegated host info.

changelogs/fragments/display-stdout-column-width.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Ansible output now uses stdout to determine column width instead of stdin

changelogs/fragments/fq_action_module_resolution.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - action plugins - change all action/module delegations to use FQ names while allowing overrides (https://github.com/ansible/ansible/issues/69788)

changelogs/fragments/galaxy-collection-fallback.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- ansible-galaxy collection install - fix fallback mechanism if the AH server did not have the collection requested - https://github.com/ansible/ansible/issues/70940

changelogs/fragments/handle_undefined_in_type_errors_filters.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Allow TypeErrors on Undefined variables in filters to be handled or deferred when processing for loops.

changelogs/fragments/keep_log_at_info.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - reset logging level to INFO due to CVE-2019-14846.

changelogs/fragments/lineinfile_exc_fix.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - lineinfile - fix not subscriptable error in exception handling around file creation

changelogs/fragments/module_utils_finder_refactor.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Python module_utils finder - refactor logic to eliminate many corner cases, remove recursion, fix base module_utils redirections

changelogs/fragments/pause-catch-error-when-no-std-exists.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - pause - handle exception when there is no stdout (https://github.com/ansible/ansible/pull/47851)

changelogs/fragments/plugin-loader-collection-name.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- "The plugin loader now keeps track of the collection where a plugin was resolved to, in particular whether the plugin was loaded from ansible-base's internal paths (``ansible.builtin``) or from user-supplied paths (no collection name)."

changelogs/fragments/rpmfluff-compat-fixes.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- Address compat with rpmfluff-0.6 for integration tests

changelogs/fragments/runtime-splunk-redirect.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - splunk httpapi plugin - switch from splunk.enterprise_security to splunk.es in runtime.yml to reflect upstream change of Collection Name

changelogs/fragments/set_fact-connection_vars.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- Stop adding the connection variables to the output results

changelogs/fragments/test-ansible-runner-pin-psutil.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - add constraints file for ``anisble_runner`` test since an update to ``psutil`` is now causing test failures

changelogs/fragments/update-conditionals-bare-vars-default.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - conditionals - change the default of CONDITIONAL_BARE_VARS to False (https://github.com/ansible/ansible/issues/70682).

changelogs/fragments/v2.10.0_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-08-13
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0b1_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-06-17
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0rc1_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-07-23
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0rc2_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-07-23
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0rc3_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-07-24
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0rc4_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-07-30
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/validate-modules-ps-doc-blacklist.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+- ansible-test - Do not try to validate PowerShell modules ``setup.ps1``, ``slurp.ps1``, and ``async_status.ps1``

changelogs/fragments/win_async_full_path.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- windows async - use full path when calling PowerShell to reduce reliance on environment vars being correct - https://github.com/ansible/ansible/issues/70655

changelogs/fragments/win_setup-redirection.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+- win setup - Fix redirection path for the windows setup module

changelogs/fragments/with_seq_example.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- Add an example for using var in with_sequence (https://github.com/ansible/ansible/issues/68836).

docs/docsite/.gitignore

@@ -14,4 +14,6 @@ objects.inv
 rst/dev_guide/testing/sanity/index.rst
 rst/modules/*.rst
 rst/playbooks_keywords.rst
+rst/collections/
+
 *.min.css

docs/docsite/Makefile

@@ -1,6 +1,6 @@
 OS := $(shell uname -s)
 SITELIB = $(shell python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"):
-PLUGIN_FORMATTER=../../hacking/build-ansible.py document-plugins
+PLUGIN_FORMATTER=../../hacking/build-ansible.py docs-build
 TESTING_FORMATTER=../bin/testing_formatter.sh
 KEYWORD_DUMPER=../../hacking/build-ansible.py document-keywords
 CONFIG_DUMPER=../../hacking/build-ansible.py document-config
@@ -12,23 +12,35 @@ else
 CPUS ?= $(shell nproc)
 endif
 
-# Sets the build output directory if it's not already specified
+# Sets the build output directory for the main docsite if it's not already specified
 ifndef BUILDDIR
 	BUILDDIR = _build
 endif
 
-MODULE_ARGS=
-ifdef MODULES
-	MODULE_ARGS = -l $(MODULES)
-endif
-
+# Backwards compat for separate VARS
 PLUGIN_ARGS=
+ifdef MODULES
+ifndef PLUGINS
+	PLUGIN_ARGS = -l $(MODULES)
+else
+	PLUGIN_ARGS = -l $(MODULES),$(PLUGINS)
+endif
+else
 ifdef PLUGINS
 	PLUGIN_ARGS = -l $(PLUGINS)
 endif
+endif
+
 
 DOC_PLUGINS ?= become cache callback cliconf connection httpapi inventory lookup netconf shell strategy vars
 
+PYTHON=python
+# fetch version from project release.py as single source-of-truth
+VERSION := $(shell $(PYTHON) ../../packaging/release/versionhelper/version_helper.py --raw || echo error)
+ifeq ($(findstring error,$(VERSION)), error)
+$(error "version_helper failed")
+endif
+
 assertrst:
 ifndef rst
 	$(error specify document or pattern with rst=somefile.rst)
@@ -38,7 +50,8 @@ all: docs
 
 docs: htmldocs
 
-generate_rst: collections_meta config cli keywords modules plugins testing
+generate_rst: collections_meta config cli keywords plugins testing
+base_generate_rst: collections_meta config cli keywords base_plugins testing
 
 htmldocs: generate_rst
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx html
@@ -46,9 +59,12 @@ htmldocs: generate_rst
 singlehtmldocs: generate_rst
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx singlehtml
 
+base_singlehtmldocs: base_generate_rst
+	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx singlehtml
+
 linkcheckdocs: generate_rst
 		CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx linkcheck
-		
+
 webdocs: docs
 
 #TODO: leaving htmlout removal for those having older versions, should eventually be removed also
@@ -58,7 +74,7 @@ clean:
 	-rm -rf $(BUILDDIR)/html
 	-rm -rf htmlout
 	-rm -rf module_docs
-	-rm -rf _build
+	-rm -rf $(BUILDDIR)
 	-rm -f .buildinfo
 	-rm -f objects.inv
 	-rm -rf *.doctrees
@@ -70,43 +86,44 @@ clean:
 	find . -type f \( -name "*~" -or -name "#*" \) -delete
 	find . -type f \( -name "*.swp" \) -delete
 	@echo "Cleaning up generated rst"
-	rm -f rst/modules/*_by_category.rst
-	rm -f rst/modules/list_of_*.rst
-	rm -f rst/modules/*_maintained.rst
-	rm -f rst/modules/*_module.rst
-	rm -f rst/modules/*_plugin.rst
 	rm -f rst/playbooks_directives.rst
-	rm -f rst/plugins/*/*.rst
 	rm -f rst/reference_appendices/config.rst
 	rm -f rst/reference_appendices/playbooks_keywords.rst
 	rm -f rst/dev_guide/collections_galaxy_meta.rst
 	rm -f rst/cli/*.rst
+	rm -rf rst/collections/*
+	@echo "Cleaning up legacy generated rst locations"
+	rm -rf rst/modules
+	rm -f rst/plugins/*/*.rst
 
 .PHONY: docs clean
 
 collections_meta: ../templates/collections_galaxy_meta.rst.j2
-	PYTHONPATH=../../lib $(COLLECTION_DUMPER) --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
+	$(COLLECTION_DUMPER) --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
 
 # TODO: make generate_man output dir cli option
 cli:
 	mkdir -p rst/cli
-	PYTHONPATH=../../lib $(GENERATE_CLI) --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
+	$(GENERATE_CLI) --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
 
 keywords: ../templates/playbooks_keywords.rst.j2
-	PYTHONPATH=../../lib $(KEYWORD_DUMPER) --template-dir=../templates --output-dir=rst/reference_appendices/ ./keyword_desc.yml
+	$(KEYWORD_DUMPER) --template-dir=../templates --output-dir=rst/reference_appendices/ ./keyword_desc.yml
 
 config: ../templates/config.rst.j2
-	PYTHONPATH=../../lib $(CONFIG_DUMPER) --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
+	$(CONFIG_DUMPER) --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
 
-modules: ../templates/plugin.rst.j2
-	PYTHONPATH=../../lib $(PLUGIN_FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/modules/ $(MODULE_ARGS)
+# For now, if we're building on devel, just build base docs.  In the future we'll want to build docs that
+# are the latest versions on galaxy (using a different antsibull-docs subcommand)
+plugins:
+	if expr match "$(VERSION)" '.*[.]dev[0-9]\+$$' &> /dev/null; then \
+		$(PLUGIN_FORMATTER) base -o rst $(PLUGIN_ARGS);\
+	else \
+		$(PLUGIN_FORMATTER) full -o rst $(PLUGIN_ARGS);\
+	fi
 
-plugins: ../templates/plugin.rst.j2
-	@echo "looping over doc plugins"
-	for plugin in $(DOC_PLUGINS); \
-	do \
-		PYTHONPATH=../../lib $(PLUGIN_FORMATTER) -t rst --plugin-type $$plugin --template-dir=../templates --module-dir=../../lib/ansible/plugins/$$plugin -o rst $(PLUGIN_ARGS); \
-	done
+# This only builds the plugin docs included with ansible-base
+base_plugins:
+	$(PLUGIN_FORMATTER) base -o rst $(PLUGIN_ARGS);\
 
 testing:
 	$(TESTING_FORMATTER)

docs/docsite/_themes/sphinx_rtd_theme/ansible_banner.html

@@ -1,5 +1,8 @@
 <!--- Based on sphinx versionwarning extension. Extension currently only works on READTHEDOCS -->
   <script>
+    startsWith = function(str, needle) {
+      return str.slice(0, needle.length) == needle
+    }
     // Create a banner if we're not on the official docs site
     if (location.host == "docs.testing.ansible.com") {
       document.write('<div id="testing_banner_id" class="admonition important">');
@@ -10,26 +13,27 @@
       element.appendChild(para);
       document.write('</div>');
     }
-
-    // Create a banner if we're not the latest version
-    current_url = window.location.href;
-    if ((current_url.search("latest") > -1) || (current_url.search("/{{ latest_version }}/") > -1)) {
-     // no banner for latest release
-    } else if (current_url.search("devel") > -1) {
-      document.write('<div id="banner_id" class="admonition caution">');
-      para = document.createElement('p');
-      banner_text=document.createTextNode("You are reading the *devel* version of the Ansible documentation - most module documentation is currently missing as the modules have moved to collections. Until docs catches up to this change, use the version selection to the left if you want module documentation or the latest stable release version. The *devel* version is not guaranteed stable.");
-      para.appendChild(banner_text);
-      element = document.getElementById('banner_id');
-      element.appendChild(para);
-      document.write('</div>');
-    } else {
-      document.write('<div id="banner_id" class="admonition caution">');
-      para = document.createElement('p');
-      banner_text=document.createTextNode("You are reading an older version of the Ansible documentation. Use the version selection to the left if you want the latest stable released version.");
-      para.appendChild(banner_text);
-      element = document.getElementById('banner_id');
-      element.appendChild(para);
-      document.write('</div>');
-    }
+    {% if (not READTHEDOCS) and (available_versions is defined) %}
+      // Create a banner if we're not the latest version
+      current_url_path = window.location.pathname;
+      if (startsWith(current_url_path, "/ansible/latest/") || startsWith(current_url_path, "/ansible/{{ latest_version }}/")) {
+       // no banner for latest release
+      } else if (startsWith(current_url_path, "/ansible/devel/")) {
+        document.write('<div id="banner_id" class="admonition caution">');
+        para = document.createElement('p');
+        banner_text=document.createTextNode("You are reading the *devel* version of the Ansible documentation - this version is not guaranteed stable. Use the version selection to the left if you want the latest stable released version.");
+        para.appendChild(banner_text);
+        element = document.getElementById('banner_id');
+        element.appendChild(para);
+        document.write('</div>');
+      } else {
+        document.write('<div id="banner_id" class="admonition caution">');
+        para = document.createElement('p');
+        banner_text=document.createTextNode("You are reading an older version of the Ansible documentation. Use the version selection to the left if you want the latest stable released version.");
+        para.appendChild(banner_text);
+        element = document.getElementById('banner_id');
+        element.appendChild(para);
+        document.write('</div>');
+      }
+    {% endif %}
   </script>

docs/docsite/_themes/sphinx_rtd_theme/ansible_versions.html

@@ -1,7 +1,7 @@
 <!--- Based on https://github.com/rtfd/sphinx_rtd_theme/pull/438/files -->
 {# Creates dropdown version selection in the top-left navigation. #}
 <div class="version">
-  {% if not READTHEDOCS %}
+  {% if (not READTHEDOCS) and (available_versions is defined) %}
     <div class="version-dropdown">
       <select class="version-list" id="version-list" onchange="javascript:location.href = this.value;">
         <script> x = document.getElementById("version-list"); </script>

docs/docsite/_themes/sphinx_rtd_theme/breadcrumbs.html

@@ -44,10 +44,14 @@
               <a href="{{ meta['github_url'] }}" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
             {% else %}
               <!-- Ansible-specific additions for modules etc -->
-                {% if pagename.endswith('_module') %}
-                  <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_module_version }}{{ meta.get('source', '') }}?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
-                {% elif pagename.startswith('plugins') and meta.get('source', None) %}
-                  <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_root_dir }}/{{ pagename }}.py?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
+                {% if (pagename.endswith('_module')) or (pagename.endswith('_become'))
+                  or (pagename.endswith('_cache')) or (pagename.endswith('_callback'))
+                  or (pagename.endswith('_connection')) or (pagename.endswith('_inventory'))
+                  or (pagename.endswith('_lookup')) or (pagename.endswith('_shell'))
+                  or (pagename.endswith('_strategy')) or (pagename.endswith('_vars'))
+                 %}
+                <!--  <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_module_version }}{{ meta.get('source', '') }}?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a> -->
+                  <br>
                 {% elif pagename.startswith('cli') and meta.get('source', None) %}
                   <a href="https://{{ github_host|default("github.com") }}/{{ github_user }}/{{ github_repo }}/{{ theme_vcs_pageview_mode|default("blob") }}/{{ github_cli_version }}{{ meta.get('source', '') }}?description=%23%23%23%23%23%20SUMMARY%0A%3C!---%20Your%20description%20here%20--%3E%0A%0A%0A%23%23%23%23%23%20ISSUE%20TYPE%0A-%20Docs%20Pull%20Request%0A%0A%2Blabel:%20docsite_pr" class="fa fa-github"> {{ _('Edit on GitHub') }}</a>
                 {% elif (not 'list_of' in pagename) and (not 'category' in pagename) %}

docs/docsite/collection-plugins.yml

@@ -0,0 +1,17 @@
+# We also need an example of modules hosted in Automation Hub
+# We'll likely move to data hosted in botmeta instead of a standalone file but
+# we'll need all of these same details.
+module:
+    purefa_user:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'purestorage.flasharray'
+    purefa_vg:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'purestorage.flasharray'
+    gcp_compute_firewall_info:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'google.cloud'
+module_utils:
+    purefa:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'purestorage.flasharray'

docs/docsite/keyword_desc.yml

@@ -6,23 +6,25 @@ args: "A secondary way to add arguments into a task. Takes a dictionary in which
 always: List of tasks, in a block, that execute no matter if there is an error in the block or not.
 any_errors_fatal: Force any un-handled task errors on any host to propagate to all hosts and end the play.
 async: Run a task asynchronously if the C(action) supports this; value is maximum runtime in seconds.
-become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution.
+become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution. Implemented by the become plugin. See :ref:`become_plugins`.
+become_exe: Path to the executable used to elevate privileges. Implemented by the become plugin. See :ref:`become_plugins`.
 become_flags: A string of flag(s) to pass to the privilege escalation program when :term:`become` is True.
 become_method: Which method of privilege escalation to use (such as sudo or su).
 become_user: "User that you 'become' after using privilege escalation. The remote/login user must have permissions to become this user."
 block: List of tasks in a block.
 changed_when: "Conditional expression that overrides the task's normal 'changed' status."
-check_mode: |
-    A boolean that controls if a task is executed in 'check' mode
+check_mode: A boolean that controls if a task is executed in 'check' mode. See :ref:`check_mode_dry`.
+collections: |
 
-    .. seealso:: :ref:`check_mode_dry`
+    List of collection namespaces to search for modules, plugins, and roles. See :ref:`collections_using_playbook`
 
-connection: |
-    Allows you to change the connection plugin used for tasks to execute on the target.
+    .. note::
 
-    .. seealso:: :ref:`using_connection`
+        Tasks within a role do not inherit the value of ``collections`` from the play. To have a role search a list of collections, use the ``collections`` keyword in ``meta/main.yml`` within a role.
 
-debugger: Enable debugging tasks based on state of the task result. See :ref:`playbook_debugger`
+
+connection: Allows you to change the connection plugin used for tasks to execute on the target. See :ref:`using_connection`.
+debugger: Enable debugging tasks based on state of the task result. See :ref:`playbook_debugger`.
 delay: Number of seconds to delay between retries. This setting is only used in combination with :term:`until`.
 delegate_facts: Boolean that allows you to apply facts to a delegated host instead of inventory_hostname.
 delegate_to: Host to execute task instead of the target (inventory_hostname). Connection vars from the delegated host will also be used for the task.

docs/docsite/requirements.txt

@@ -6,4 +6,4 @@ sphinx==2.1.2
 sphinx-notfound-page
 Pygments >= 2.4.0
 straight.plugin # Needed for hacking/build-ansible.py which is the backend build script
-make
+antsibull >= 0.15.0

docs/docsite/rst/community/communication.rst

@@ -24,7 +24,7 @@ Ansible has several mailing lists.  Your first post to the mailing list will be
 * `Ansible Lockdown List <https://groups.google.com/forum/#!forum/ansible-lockdown>`_ is for all things related to Ansible Lockdown projects, including DISA STIG automation and CIS Benchmarks.
 * `Ansible Outreach List <https://groups.google.com/forum/#!forum/ansible-outreach>`_ help with promoting Ansible and `Ansible Meetups <https://ansible.meetup.com/>`_
 * `Ansible Project List <https://groups.google.com/forum/#!forum/ansible-project>`_ is for sharing Ansible tips, answering questions, and general user discussion.
-* `Molecule List <https://groups.google.com/forum/#!forum/molecule-users>`_ is designed to aid with the development and testing of Ansible roles with Molecule.
+* `Molecule Discussions <https://github.com/ansible-community/molecule/discussions>`_ is designed to aid with the development and testing of Ansible roles with Molecule.
 
 To subscribe to a group from a non-Google account, you can send an email to the subscription address requesting the subscription. For example: ``ansible-devel+subscribe@googlegroups.com``
 

docs/docsite/rst/community/contributing_maintained_collections.rst

@@ -67,15 +67,15 @@ The following table shows:
       <td><a href="https://galaxy.ansible.com/amazon/aws">amazon.aws</a></td>
       <td><a href="https://galaxy.ansible.com/community/aws">community.aws</a></td>
       <td><a href="https://github.com/ansible/community/tree/master/group-aws">Cloud</a></td>
-      <td>✓</td>
-      <td>✓</td>
+      <td>✓**</td>
+      <td>**</td>
       <td>✓</td>
       <td>Shippable</td>
       <td>✓</td>
       <td><a href="https://docs.ansible.com/ansible/devel/dev_guide/platforms/aws_guidelines.html">AWS guide</a></td>
     </tr>
     <tr>
-      <td><a href="https://galaxy.ansible.com/ansible/netcommon">ansible.netcommon**</a></td>
+      <td><a href="https://galaxy.ansible.com/ansible/netcommon">ansible.netcommon***</a></td>
       <td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
       <td>✓</td>
@@ -101,7 +101,7 @@ The following table shows:
       <td><a href="https://galaxy.ansible.com/community/windows">community.windows</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Windows">Windows</a></td>
       <td>✓</td>
-      <td>✓***</td>
+      <td>✓****</td>
       <td>✓</td>
       <td>Shippable</td>
       <td>✓</td>
@@ -120,7 +120,7 @@ The following table shows:
     </tr>
     <tr>
       <td><a href="https://galaxy.ansible.com/cisco/asa">cisco.asa</a></td>
-      <td>community.asa</td>
+      <td><a href="https://github.com/ansible-collections/community.asa">community.asa</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
       <td>✓</td>
       <td>✓</td>
@@ -164,7 +164,7 @@ The following table shows:
     </tr>
     <tr>
       <td><a href="https://galaxy.ansible.com/ibm/qradar">ibm.qradar</a></td>
-      <td>community.qradar</td>
+      <td><a href="https://github.com/ansible-collections/community.qradar">community.qradar</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
       <td>✓</td>
       <td></td>
@@ -197,7 +197,7 @@ The following table shows:
     </tr>
     <tr>
       <td><a href="https://github.com/ansible-collections/splunk.es">splunk.es</a></td>
-      <td>community.es</td>
+      <td><a href="https://github.com/ansible-collections/community.es">community.es</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
       <td>✓</td>
       <td></td>
@@ -224,9 +224,11 @@ The following table shows:
 
   \* A ✓  under **Open to PRs** means the collection welcomes GitHub issues and PRs for any changes to existing collection content (plugins, roles, and so on).
 
-  \*\* ``ansible.netcommon`` contains all foundational components for enabling many network and security :ref:`platform <platform_options>` collections. It contains all connection and filter plugins required, and installs as a dependency when you install the the platform collection.
+  \*\* Integration tests are required and unit tests are welcomed but not required for the AWS collections.  An exception to this is made in cases where integration tests are logistically not feasible due to external requirements.  An example of this is AWS Direct Connect, as this service can not be functionally tested without the establishment of network peering connections.  Unit tests are therefore required for modules that interact with AWS Direct Connect.  Exceptions to ``amazon.aws`` must be approved by Red Hat, and exceptions to ``community.aws`` must be approved by the AWS community.
 
-  \*\*\* Unit tests for Windows PowerShell modules are an exception to testing, but unit tests are valid and required for the remainder of the collection, including Ansible-side plugins.
+  \*\*\* ``ansible.netcommon`` contains all foundational components for enabling many network and security :ref:`platform <platform_options>` collections. It contains all connection and filter plugins required, and installs as a dependency when you install the the platform collection.
+
+  \*\*\*\* Unit tests for Windows PowerShell modules are an exception to testing, but unit tests are valid and required for the remainder of the collection, including Ansible-side plugins.
 
 
 .. _which_collection:
@@ -248,7 +250,7 @@ The following scenarios use the ``arista.eos`` to help explain when to contribut
     * Propose a new module in the ``community.network`` collection (requires network community approval).
     * Place your new module in a collection in your own namespace (no approvals required).
 
-  
+
 Most new content should go into either a related community collection or your own collection first so that is well established in the community before you can propose adding it to the  ``arista`` namespace, where inclusion and maintenance criteria are much higher.
 
 

docs/docsite/rst/community/development_process.rst

@@ -27,7 +27,7 @@ If you want to follow the conversation about what features will be added to Ansi
 Micro development: the lifecycle of a PR
 ========================================
 
-Ansible accepts code through **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works <https://help.github.com/articles/about-pull-requests/>`_ in general. The ultimate goal of any pull request is to get merged and become part of Ansible Core.
+Ansible accepts code through **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works <https://help.github.com/articles/about-pull-requests/>`_ in general. The ultimate goal of any pull request is to get merged and become part of a collection or ansible-base.
 Here's an overview of the PR lifecycle:
 
 * Contributor opens a PR
@@ -43,7 +43,7 @@ Here's an overview of the PR lifecycle:
 Automated PR review: ansibullbot
 --------------------------------
 
-Because Ansible receives many pull requests, and because we love automating things, we've automated several steps of the process of reviewing and merging pull requests with a tool called Ansibullbot, or Ansibot for short.
+Because Ansible receives many pull requests, and because we love automating things, we have automated several steps of the process of reviewing and merging pull requests with a tool called Ansibullbot, or Ansibot for short.
 
 `Ansibullbot <https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md>`_ serves many functions:
 
@@ -139,11 +139,7 @@ We don't merge every PR. Here are some tips for making your PR useful, attractiv
 Changelogs
 ----------
 
-Changelogs help users and developers keep up with changes to Ansible.
-Ansible builds a changelog for each release from fragments.
-You **must** add a changelog fragment to any PR that changes functionality or fixes a bug in ansible-base.
-You don't have to add a changelog fragment for PRs that add new
-modules and plugins, because our tooling does that for you automatically.
+Changelogs help users and developers keep up with changes to Ansible. Ansible builds a changelog for each release from fragments. You **must** add a changelog fragment to any PR that changes functionality or fixes a bug in ansible-base. You do not have to add a changelog fragment for PRs that add new modules and plugins, because our tooling does that for you automatically.
 
 We build short summary changelogs for minor releases as well as for major releases. If you backport a bugfix, include a changelog fragment with the backport PR.
 
@@ -152,35 +148,29 @@ We build short summary changelogs for minor releases as well as for major releas
 Creating a changelog fragment
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-A basic changelog fragment is a ``.yaml`` file placed in the
-``changelogs/fragments/`` directory.  Each file contains a yaml dict with
-keys like ``bugfixes`` or ``major_changes`` followed by a list of
-changelog entries of bugfixes or features.  Each changelog entry is
-rst embedded inside of the yaml file which means that certain
-constructs would need to be escaped so they can be interpreted by rst
-and not by yaml (or escaped for both yaml and rst if that's your
-desire).  Each PR **must** use a new fragment file rather than adding to
-an existing one, so we can trace the change back to the PR that introduced it.
+A basic changelog fragment is a ``.yaml`` file placed in the ``changelogs/fragments/`` directory.  Each file contains a yaml dict with keys like ``bugfixes`` or ``major_changes`` followed by a list of changelog entries of bugfixes or features.  Each changelog entry is rst embedded inside of the yaml file which means that certain constructs would need to be escaped so they can be interpreted by rst and not by yaml (or escaped for both yaml and rst if you prefer).  Each PR **must** use a new fragment file rather than adding to an existing one, so we can trace the change back to the PR that introduced it.
 
-To create a changelog entry, create a new file with a unique name in the ``changelogs/fragments/`` directory of corresponding repository.
-The file name should include the PR number and a description of the change.
-It must end with the file extension ``.yaml``. For example: ``40696-user-backup-shadow-file.yaml``
+To create a changelog entry, create a new file with a unique name in the ``changelogs/fragments/`` directory of the corresponding repository. The file name should include the PR number and a description of the change. It must end with the file extension ``.yaml``. For example: ``40696-user-backup-shadow-file.yaml``
 
-A single changelog fragment may contain multiple sections but most will only contain one section.
-The toplevel keys (bugfixes, major_changes, and so on) are defined in the
-`config file <https://github.com/ansible/ansible/blob/devel/changelogs/config.yaml>`_ for our release note tool. Here are the valid sections and a description of each:
+A single changelog fragment may contain multiple sections but most will only contain one section. The toplevel keys (bugfixes, major_changes, and so on) are defined in the `config file <https://github.com/ansible/ansible/blob/devel/changelogs/config.yaml>`_ for our `release note tool <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst>`_. Here are the valid sections and a description of each:
+
+**breaking_changes**
+  Changes that break existing playbooks or roles. This includes any change to existing behavior that forces users to update tasks. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
 
 **major_changes**
-  Major changes to Ansible itself. Generally does not include module or plugin changes.
+  Major changes to Ansible itself. Generally does not include module or plugin changes. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
 
 **minor_changes**
   Minor changes to Ansible, modules, or plugins. This includes new features, new parameters added to modules, or behavior changes to existing parameters.
 
 **deprecated_features**
-  Features that have been deprecated and are scheduled for removal in a future release.
+  Features that have been deprecated and are scheduled for removal in a future release. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
 
 **removed_features**
-  Features that were previously deprecated and are now removed.
+  Features that were previously deprecated and are now removed. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`.
+
+**security_fixes**
+  Fixes that address CVEs or resolve security concerns. Include links to CVE information.
 
 **bugfixes**
   Fixes that resolve issues.
@@ -188,8 +178,7 @@ The toplevel keys (bugfixes, major_changes, and so on) are defined in the
 **known_issues**
   Known issues that are currently not fixed or will not be fixed.
 
-Each changelog entry must contain a link to its issue between parentheses at the end.
-If there is no corresponding issue, the entry must contain a link to the PR itself.
+Each changelog entry must contain a link to its issue between parentheses at the end. If there is no corresponding issue, the entry must contain a link to the PR itself.
 
 Most changelog entries will be ``bugfixes`` or ``minor_changes``. When writing a changelog entry that pertains to a particular module, start the entry with ``- [module name] -`` and the following sentence with a lowercase letter.
 
@@ -209,7 +198,7 @@ Here are some examples:
 .. code-block:: yaml
 
   bugfixes:
-    - copy module - The copy module was attempting to change the mode of files for
+    - copy - the module was attempting to change the mode of files for
       remote_src=True even if mode was not set as a parameter.  This failed on
       filesystems which do not have permission bits (https://github.com/ansible/ansible/issues/29444).
 
@@ -222,9 +211,7 @@ Once you've written the changelog fragment for your PR, commit the file and incl
 Backporting merged PRs
 ======================
 
-All Ansible PRs must be merged to the ``devel`` branch first.
-After a pull request has been accepted and merged to the ``devel`` branch, the following instructions will help you create a
-pull request to backport the change to a previous stable branch.
+All Ansible PRs must be merged to the ``devel`` branch first. After a pull request has been accepted and merged to the ``devel`` branch, the following instructions will help you create a pull request to backport the change to a previous stable branch.
 
 We do **not** backport features.
 

docs/docsite/rst/community/documentation_contributions.rst

@@ -102,6 +102,10 @@ You can drop ``--user`` if you have set up a virtual environment (venv/virtenv).
 
     On macOS with Xcode, you may need to install ``six`` and ``pyparsing`` with ``--ignore-installed`` to get versions that work with ``sphinx``.
 
+.. note::
+
+  	After checking out ``ansible/ansible``, make sure the ``docs/docsite/rst`` directory has strict enough permissions. It should only be writable by the owner's account. If your default ``umask`` is not 022, you can use ``chmod go-w docs/docsite/rst`` to set the permissions correctly in your new branch.  Optionally, you can set your ``umask`` to 022 to make all newly created files on your system (including those created by ``git clone``) have the correct permissions.
+
 .. _testing_documentation_locally:
 
 Testing the documentation locally

docs/docsite/rst/community/how_can_I_help.rst

@@ -4,7 +4,8 @@
 How can I help?
 ***************
 
-.. contents:: Topics
+.. contents::
+   :local:
 
 Thanks for being interested in helping the Ansible project!
 
@@ -50,19 +51,19 @@ File and verify issues
 All software has bugs, and Ansible is no exception. When you find a bug, you can help tremendously by :ref:`telling us about it <reporting_bugs_and_features>`.
 
 
-If you should discover that the bug you're trying to file already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information.
+If the bug you found already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information.
 
 Review and submit pull requests
 ===============================
 
-As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you've got a solution to a bug you've found in Ansible, or if you've got a new feature that you've written and would like to share with millions of Ansible users, read all about the :ref:`Ansible development process <community_development_process>` to learn how to get your code accepted into Ansible.
+As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you have a fix for a bug in Ansible, or if you have a new feature that you would like to share with millions of Ansible users, read all about the :ref:`Ansible development process <community_development_process>` and and :ref:`how to contribute to collections <contributing_maintained_collections>` to learn how to get your code accepted into Ansible.
 
 Another good way to help is to review pull requests that other Ansible users have submitted. The Ansible community keeps a full list of `open pull requests by file <https://ansible.sivel.net/pr/byfile.html>`_, so if there's a particular module or plug-in that particularly interests you, you can easily keep track of all the relevant new pull requests and provide testing or feedback.
 
-Become a module maintainer
-==========================
+Become a collection maintainer
+==============================
 
-Once you've learned about the development process and have contributed code to a particular module, we encourage you to become a maintainer of that module. There are hundreds of different modules in Ansible, and the vast majority of them are written and maintained entirely by members of the Ansible community.
+Once you have learned about the development process and have contributed code to a collection, we encourage you to become a maintainer of that collection. There are hundreds of modules in dozens of Ansible collections, and the vast majority of them are written and maintained entirely by members of the Ansible community.
 
 To learn more about the responsibilities of being an Ansible module maintainer, please read our :ref:`module maintainer guidelines <maintainers>`.
 
@@ -77,7 +78,7 @@ Working groups are a way for Ansible community members to self-organize around p
 Teach Ansible to others
 =======================
 
-We're working on a standardized Ansible workshop called `Lightbulb <https://github.com/ansible/lightbulb>`_ that can provide a good hands-on introduction to Ansible usage and concepts.
+We are working on a standardized `Ansible workshop <https://ansible.github.io/workshops/>`_ that can provide a good hands-on introduction to Ansible usage and concepts.
 
 Social media
 ============

docs/docsite/rst/community/index.rst

@@ -14,9 +14,9 @@ This page outlines the most common situations and questions that bring readers t
 Getting started
 ===============
 
-* I'm new to the community. Where can I find the Ansible :ref:`code_of_conduct`?
-* I'd like to know what I'm agreeing to when I contribute to Ansible. Does Ansible have a :ref:`contributor_license_agreement`?
-* I'd like to contribute but I'm not sure how. Are there :ref:`easy ways to contribute <how_can_i_help>`?
+* I am new to the community. Where can I find the Ansible :ref:`code_of_conduct`?
+* I would like to know what I am agreeing to when I contribute to Ansible. Does Ansible have a :ref:`contributor_license_agreement`?
+* I would like to contribute but I am not sure how. Are there :ref:`easy ways to contribute <how_can_i_help>`?
 * I want to talk to other Ansible users. How do I find an `Ansible Meetup near me <https://www.meetup.com/topics/ansible/>`_?
 * I have a question. Which :ref:`Ansible email lists and IRC channels <communication>` will help me find answers?
 * I want to learn more about Ansible. What can I do?
@@ -27,7 +27,7 @@ Getting started
   * `Review getting started guides <https://www.ansible.com/resources/get-started>`_.
   * `Watch videos <https://www.ansible.com/resources/videos>`_ - includes Ansible Automates, AnsibleFest & webinar recordings.
 
-* I'd like updates about new Ansible versions. How are `new releases announced <https://groups.google.com/forum/#!forum/ansible-announce>`_?
+* I would like updates about new Ansible versions. How are `new releases announced <https://groups.google.com/forum/#!forum/ansible-announce>`_?
 * I want to use the current release. How do I know which :ref:`releases are current <release_schedule>`?
 
 Going deeper
@@ -36,9 +36,9 @@ Going deeper
 * I think Ansible is broken. How do I :ref:`report a bug <reporting_bugs>`?
 * I need functionality that Ansible doesn't offer. How do I :ref:`request a feature <request_features>`?
 * How do I :ref:`contribute to an Ansible-maintained collection <contributing_maintained_collections>`?
-* I'm waiting for a particular feature. How do I see what's :ref:`planned for future Ansible Releases <roadmaps>`?
+* I am waiting for a particular feature. How do I see what is :ref:`planned for future Ansible Releases <roadmaps>`?
 * I have a specific Ansible interest or expertise (for example, VMware, Linode, and so on.). How do I get involved in a :ref:`working group <working_group_list>`?
-* I'd like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests?
+* I would like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests?
 * I found a typo or another problem on docs.ansible.com. How can I :ref:`improve the documentation <community_documentation_contributions>`?
 
 
@@ -46,15 +46,15 @@ Working with the Ansible repo
 =============================
 
 * I want to code my first changes to Ansible. How do I :ref:`set up my Python development environment <environment_setup>`?
-* I'd like to get more efficient as a developer. How can I find :ref:`editors, linters, and other tools <other_tools_and_programs>` that will support my Ansible development efforts?
+* I would like to get more efficient as a developer. How can I find :ref:`editors, linters, and other tools <other_tools_and_programs>` that will support my Ansible development efforts?
 * I want my PR to meet Ansible's guidelines. Where can I find  guidance on :ref:`coding in Ansible <developer_guide>`?
 * I want to learn more about Ansible roadmaps, releases, and projects. How do I find information on :ref:`the development cycle <community_development_process>`?
-* I'd like to connect Ansible to a new API or other resource. How do I :ref:`contribute a group of related modules <developing_modules_in_groups>`?
+* I would like to connect Ansible to a new API or other resource. How do I :ref:`create a collection <developing_modules_in_groups>`?
 * My pull request is marked ``needs_rebase``. How do I :ref:`rebase my PR <rebase_guide>`?
-* I'm using an older version of Ansible and want a bug fixed in my version that's already been fixed on the ``devel`` branch. How do I :ref:`backport a bugfix PR <backport_process>`?
+* I am using an older version of Ansible and want a bug fixed in my version that has already been fixed on the ``devel`` branch. How do I :ref:`backport a bugfix PR <backport_process>`?
 * I have an open pull request with a failing test. How do I learn about Ansible's :ref:`testing (CI) process <developing_testing>`?
-* I'm ready to step up as a module maintainer. What are the :ref:`guidelines for maintainers <maintainers>`?
-* A module I maintain is obsolete. How do I :ref:`deprecate a module <deprecating_modules>`?
+* I am ready to step up as a collection maintainer. What are the :ref:`guidelines for maintainers <maintainers>`?
+* A module in a collection I maintain is obsolete. How do I :ref:`deprecate a module <deprecating_modules>`?
 
 Traditional Table of Contents
 =============================

docs/docsite/rst/community/maintainers.rst

@@ -1,12 +1,13 @@
 .. _maintainers:
 
-****************************
-Module Maintainer Guidelines
-****************************
+********************************
+Collection maintainer guidelines
+********************************
 
-.. contents:: Topics
+Thank you for being a community collection maintainer. This guide offers an overview of your responsibilities as a maintainer along with resources for additional information. The Ansible community hopes that you will find that maintaining a collection is as rewarding for you as having the collection content is for the wider community.
 
-Thank you for being a maintainer of part of Ansible's codebase. This guide provides module maintainers an overview of their responsibilities, resources for additional information, and links to helpful tools.
+.. contents::
+   :local:
 
 In addition to the information below, module maintainers should be familiar with:
 
@@ -17,57 +18,17 @@ In addition to the information below, module maintainers should be familiar with
 Maintainer responsibilities
 ===========================
 
-When you contribute a new module to the `ansible/ansible <https://github.com/ansible/ansible>`_ repository, you become the maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
+When you contribute a module to a collection included in the ``ansible`` package, you become a maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
 
-Maintainers of Ansible modules are expected to provide feedback, responses, or actions on pull requests or issues to the module(s) they maintain in a reasonably timely manner.
+Maintainers of Ansible collections are expected to provide feedback, responses, or actions on pull requests or issues to the collection(s) they maintain in a reasonably timely manner. You can also update the contributor guidelines for that collection, in collaboration with the Ansible community team and the other maintainers of that collection.
 
-It is also recommended that you occasionally revisit the `contribution guidelines <https://github.com/ansible/ansible/blob/devel/.github/CONTRIBUTING.md>`_, as they are continually refined. Occasionally, you may be requested to update your module to move it closer to the general accepted standard requirements. We hope for this to be infrequent, and will always be a request with a fair amount of lead time (ie: not by tomorrow!).
+Resources
+=========
 
-Finally, following the `ansible-devel <https://groups.google.com/forum/#!forum/ansible-devel>`_ mailing list can be a great way to participate in the broader Ansible community, and a place where you can influence the overall direction, quality, and goals of Ansible and its modules. If you're not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
+Please see :ref:`communication` for ways to contact the broader Ansible community. For maintainers, following the `ansible-devel <https://groups.google.com/forum/#!forum/ansible-devel>`_ mailing list is a great way to participate in conversations about coding, get assistance when you need it, and influence the overall direction, quality, and goals of Ansible and the collections. If you are not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
 
-The Ansible community hopes that you will find that maintaining your module is as rewarding for you as having the module is for the wider community.
 
 Pull requests, issues, and workflow
 ===================================
 
-Pull requests
--------------
-
-Module pull requests are located in the `main Ansible repository <https://github.com/ansible/ansible/pulls>`_.
-
-Because of the high volume of pull requests, notification of PRs to specific modules are routed by an automated bot to the appropriate maintainer for handling. It is recommended that you set an appropriate notification process to receive notifications which mention your GitHub ID.
-
-Issues
-------
-
-Issues for modules, including bug reports, documentation bug reports, and feature requests, are tracked in the `ansible repository <https://github.com/ansible/ansible/issues>`_.
-
-Issues for modules are routed to their maintainers by an automated process. This process is still being refined, and currently depends upon the issue creator to provide adequate details (specifically, providing the proper module name) in order to route it correctly. If you are a maintainer of a specific module, it is recommended that you periodically search module issues for issues which mention your module's name (or some variation on that name), as well as setting an appropriate notification process for receiving notification of mentions of your GitHub ID.
-
-PR workflow
------------
-
-Automated routing of pull requests is handled by a tool called `Ansibot <https://github.com/ansible/ansibullbot>`_.
-
-Being moderately familiar with how the workflow behind the bot operates can be helpful to you, and -- should things go awry -- your feedback can be helpful to the folks that continually help Ansibullbot to evolve.
-
-A detailed explanation of the PR workflow can be seen in the :ref:`community_development_process`.
-
-Maintainers (BOTMETA.yml)
--------------------------
-
-The full list of maintainers is located in `BOTMETA.yml <https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml>`_.
-
-Adding and removing maintainers
-===============================
-
-Communities change over time, and no one maintains a module forever. If you'd like to propose an additional maintainer for your module, please submit a PR to ``BOTMETA.yml`` with the GitHub username of the new maintainer.
-
-If you'd like to step down as a maintainer, please submit a PR to the ``BOTMETA.yml`` removing your GitHub ID from the module in question. If that would leave the module with no maintainers, put "ansible" as the maintainer.  This will indicate that the module is temporarily without a maintainer, and the Ansible community team will search for a new maintainer.
-
-Tools and other Resources
-=========================
-
-* `PRs in flight, organized by directory <https://ansible.sivel.net/pr/byfile.html>`_
-* `Ansibullbot <https://github.com/ansible/ansibullbot>`_
-* :ref:`community_development_process`
+Each collection community can set its own rules and workflow for managing pull requests, bug reports, documentation issues, and feature requests, as well as adding and replacing maintainers.

docs/docsite/rst/community/reporting_bugs_and_features.rst

@@ -1,39 +1,56 @@
 .. _reporting_bugs_and_features:
 
 **************************************
-Reporting Bugs And Requesting Features
+Reporting bugs and requesting features
 **************************************
 
-.. contents:: Topics
+.. contents::
+   :local:
 
 .. _reporting_bugs:
 
 Reporting a bug
 ===============
 
+Security bugs
+-------------
+
 Ansible practices responsible disclosure - if this is a security-related bug, email `security@ansible.com <mailto:security@ansible.com>`_ instead of filing a ticket or posting to any public groups, and you will receive a prompt response.
 
-Ansible bugs should be reported to `github.com/ansible/ansible/issues <https://github.com/ansible/ansible/issues>`_ after
-signing up for a free GitHub account.  Before reporting a bug, please use the bug/issue search
-to see if the issue has already been reported.
+Bugs in ansible-base
+--------------------
 
-Knowing your Ansible version and the exact commands you are running, and what you expect, saves time and helps us help everyone with their issues more quickly. For that reason, we provide an issue template; please fill it out as completely and as accurately as possible.
+If you find a bug that affects multiple plugins, a plugin that remained in the ansible/ansible repo, or the overall functioning of Ansible, report it to `github.com/ansible/ansible/issues <https://github.com/ansible/ansible/issues>`_. You need a free GitHub account.  Before reporting a bug, use the bug/issue search to see if the issue has already been reported. If you are not sure if something is a bug yet, you can report the behavior on the :ref:`mailing list or IRC first <communication>`.
 
-Do not use the issue tracker for "how do I do this" type questions.  These are great candidates for IRC or the mailing list instead where things are likely to be more of a discussion.
+Do not open issues for "how do I do this" type questions.  These are great topics for IRC or the mailing list, where things are likely to be more of a discussion.
 
-To be respectful of reviewers' time and allow us to help everyone efficiently, please  provide minimal well-reduced and well-commented examples rather than sharing your entire production playbook.  Include playbook snippets and output where possible.
+If you find a bug, open the issue yourself to ensure we have a record of it. Do not rely on someone else in the community to file the bug report for you. We have created an issue template, which saves time and helps us help everyone with their issues more quickly. Please fill it out as completely and as accurately as possible:
 
-When sharing YAML in playbooks, formatting can be preserved by using `code blocks  <https://help.github.com/articles/creating-and-highlighting-code-blocks/>`_.
+  * Include the Ansible version
+  * Include any relevant configuration
+  * Include the exact commands or tasks you are running
+  * Describe the behavior you expected
+  * Provide steps to reproduce the bug
+    * Use minimal well-reduced and well-commented examples, not your entire production playbook
+    * When sharing YAML in playbooks, preserve the formatting by using `code blocks  <https://help.github.com/articles/creating-and-highlighting-code-blocks/>`_.
+  * Document the behavior you got
+  * Include output where possible
+  * For multiple-file content, use gist.github.com, which is more durable than pastebin content
 
-For multiple-file content, we encourage use of gist.github.com.  Online pastebin content can expire, so it's nice to have things around for a longer term if they are referenced in a ticket.
+Bugs in collections
+-------------------
 
-If you are not sure if something is a bug yet, you are welcome to ask about something on the :ref:`mailing list or IRC first <communication>`.
+Many bugs only affect a single module or plugin. If you find a bug that affects a module or plugin hosted in a collection, file the bug in the repository of the :ref:`collection <collections>`:
 
-As we are a very high volume project, if you determine that you do have a bug, please be sure to open the issue yourself to ensure we have a record of it. Don't rely on someone else in the community to file the bug report for you.
+  #. Find the collection on Galaxy.
+  #. Click on the Issue Tracker link for that collection.
+  #. Follow the contributor guidelines or instructions in the collection repo.
+
+If you are not sure whether a bug is in ansible-base or in a collection, you can report the behavior on the :ref:`mailing list or IRC first <communication>`.
 
 .. _request_features:
 
 Requesting a feature
 ====================
 
-The best way to get a feature into Ansible is to :ref:`submit a pull request <community_pull_requests>`.
+The best way to get a feature into Ansible is to :ref:`submit a pull request <community_pull_requests>`, either against ansible-base or against a collection. See also :ref:`ansible_collection_merge_requirements`.

docs/docsite/rst/conf.py

@@ -147,7 +147,7 @@ html_context = {
     'current_version': version,
     'latest_version': '2.9',
     # list specifically out of order to make latest work
-    'available_versions': ('latest', '2.8', '2.7', 'devel'),
+    'available_versions': ('latest', '2.9_ja', '2.8', '2.7', 'devel'),
     'css_files': ('_static/ansible.css',  # overrides to the standard theme
                   ),
 }

docs/docsite/rst/dev_guide/debugging.rst

@@ -1,140 +1,112 @@
-.. _debugging:
+.. _debugging_modules:
 
 *****************
 Debugging modules
 *****************
 
-Debugging (local)
-=================
+.. contents::
+   :local:
 
-To break into a module running on ``localhost`` and step through with the debugger:
+.. _detailed_debugging:
 
-- Set a breakpoint in the module: ``import pdb; pdb.set_trace()``
-- Run the module on the local machine: ``$ python -m pdb ./my_new_test_module.py ./args.json``
+Detailed debugging steps
+========================
 
-Example
--------
+Ansible modules are put together as a zip file consisting of the module file and the various Python module boilerplate inside of a wrapper script. To see what is actually happening in the module, you need to extract the file from the wrapper. The wrapper script provides helper methods that let you do that.
 
-`echo '{"msg": "hello"}' | python ./my_new_test_module.py`
-
-Debugging (remote)
-==================
-
-To debug a module running on a remote target (i.e. not ``localhost``):
-
-#. On your controller machine (running Ansible) set ``ANSIBLE_KEEP_REMOTE_FILES=1`` to tell Ansible to retain the modules it sends to the remote machine instead of removing them after you playbook runs.
-#. Run your playbook targeting the remote machine and specify ``-vvvv`` (verbose) to display the remote location Ansible is using for the modules (among many other things).
-#. Take note of the directory Ansible used to store modules on the remote host. This directory is usually under the home directory of your ``ansible_user``, in the form ``~/.ansible/tmp/ansible-tmp-...``.
-#. SSH into the remote target after the playbook runs.
-#. Navigate to the directory you noted in step 3.
-#. Extract the module you want to debug from the zipped file that Ansible sent to the remote host: ``$ python AnsiballZ_my_test_module.py explode``. Ansible will expand the module into ``./debug_dir``. You can optionally run the zipped file by specifying ``python AnsiballZ_my_test_module.py``.
-#. Navigate to the debug directory: ``$ cd debug_dir``.
-#. Modify or set a breakpoint in ``__main__.py``.
-#. Ensure that the unzipped module is executable: ``$ chmod 755 __main__.py``.
-#. Run the unzipped module directly, passing the ``args`` file that contains the params that were originally passed: ``$ ./__main__.py args``. This approach is good for reproducing behavior as well as modifying the parameters for debugging.
+The following steps use ``localhost`` as the target host, but you can use the same steps to debug against remote hosts as well. For a simpler approach to debugging without using the temporary files, see :ref:`simple debugging <simple_debugging>`.
 
 
-.. _debugging_ansiblemodule_based_modules:
+#. Set :envvar:`ANSIBLE_KEEP_REMOTE_FILES` to ``1`` on the control host so Ansible will keep the remote module files instead of deleting them after the module finishes executing. Use the ``-vvv`` option to make Ansible more verbose. This will display the file name of the temporary module file.
 
-Debugging AnsibleModule-based modules
-=====================================
+    .. code-block:: shell-session
 
-.. tip::
+      $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible localhost -m ping -a 'data=debugging_session' -vvv
+      <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: badger
+      <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" )'
+      <127.0.0.1> PUT /var/tmp/tmpjdbJ1w TO /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/AnsiballZ_ping.py
+      <127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/AnsiballZ_ping.py && sleep 0'
+      localhost | SUCCESS => {
+          "changed": false,
+          "invocation": {
+              "module_args": {
+                  "data": "debugging_session"
+              },
+              "module_name": "ping"
+          },
+          "ping": "debugging_session"
+      }
 
-    If you're using the :file:`hacking/test-module.py` script then most of this
-    is taken care of for you.  If you need to do some debugging of the module
-    on the remote machine that the module will actually run on or when the
-    module is used in a playbook then you may need to use this information
-    instead of relying on :file:`test-module.py`.
+#. Navigate to the temporary directory from the previous step. If the previous command was run against a remote host, connect to that host first before trying to navigate to the temporary directory.
 
-Starting with Ansible 2.1, AnsibleModule-based modules are put together as
-a zip file consisting of the module file and the various python module
-boilerplate inside of a wrapper script instead of as a single file with all of
-the code concatenated together.  Without some help, this can be harder to
-debug as the file needs to be extracted from the wrapper in order to see
-what's actually going on in the module.  Luckily the wrapper script provides
-some helper methods to do just that.
+   .. code-block:: shell-session
 
-If you are using Ansible with the :envvar:`ANSIBLE_KEEP_REMOTE_FILES`
-environment variables to keep the remote module file, here's a sample of how
-your debugging session will start:
+      $ ssh remotehost  # only if not debugging against localhost
+      $ cd /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595
 
-.. code-block:: shell-session
+#. Run the wrapper's ``explode`` command to turn the string into some Python files that you can work with.
 
-    $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible localhost -m ping -a 'data=debugging_session' -vvv
-    <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: badger
-    <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595 `" )'
-    <127.0.0.1> PUT /var/tmp/tmpjdbJ1w TO /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping
-    <127.0.0.1> EXEC /bin/sh -c 'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping'
-    localhost | SUCCESS => {
-        "changed": false,
-        "invocation": {
-            "module_args": {
-                "data": "debugging_session"
-            },
-            "module_name": "ping"
-        },
-        "ping": "debugging_session"
-    }
+    .. code-block:: shell-session
 
-Setting :envvar:`ANSIBLE_KEEP_REMOTE_FILES` to ``1`` tells Ansible to keep the
-remote module files instead of deleting them after the module finishes
-executing.  Giving Ansible the ``-vvv`` option makes Ansible more verbose.
-That way it prints the file name of the temporary module file for you to see.
+      $ python AnsiballZ_ping.py explode
+      Module expanded into:
+      /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/debug_dir
 
-If you want to examine the wrapper file you can.  It will show a small python
-script with a large, base64 encoded string.  The string contains the module
-that is going to be executed.  Run the wrapper's explode command to turn the
-string into some python files that you can work with:
+    If you want to examine the wrapper file you can. It will show a small Python script with a large base64 encoded string. The string contains the module to execute.
 
-.. code-block:: shell-session
+#. When you look into the temporary directory you'll see a structure like this:
 
-    $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping explode
-    Module expanded into:
-    /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/debug_dir
+    .. code-block:: shell-session
 
-When you look into the debug_dir you'll see a directory structure like this::
+      ├── AnsiballZ_ping.py
+      └── debug_dir
+          ├── ansible
+          │   ├── __init__.py
+          │   ├── module_utils
+          │   │   ├── __init__.py
+          │   │   ├── _text.py
+          │   │   ├── basic.py
+          │   │   ├── common
+          │   │   ├── compat
+          │   │   ├── distro
+          │   │   ├── parsing
+          │   │   ├── pycompat24.py
+          │   │   └── six
+          │   └── modules
+          │       ├── __init__.py
+          │       └── ping.py
+          └── args
 
-    ├── AnsiballZ_ping.py
-    ├── args
-    └── ansible
-        ├── __init__.py
-        └── module_utils
-            ├── basic.py
-            └── __init__.py
+    * ``AnsiballZ_ping.py`` is the Python script with the the module code stored in a base64 encoded string. It contains various helper functions for executing the module.
 
-* :file:`AnsiballZ_ping.py` is the code for the module itself.  The name
-  is based on the name of the module with a prefix so that we don't clash with
-  any other python module names.  You can modify this code to see what effect
-  it would have on your module.
+    * ``ping.py`` is the code for the module itself. You can modify this code to see what effect it would have on your module, or for debugging purposes.
 
-* The :file:`args` file contains a JSON string.  The string is a dictionary
-  containing the module arguments and other variables that Ansible passes into
-  the module to change its behaviour.  If you want to modify the parameters
-  that are passed to the module, this is the file to do it in.
+    * The ``args`` file contains a JSON string. The string is a dictionary containing the module arguments and other variables that Ansible passes into the module to change its behavior. Modify this file to change the parameters passed to the module.
 
-* The :file:`ansible` directory contains code from
-  :mod:`ansible.module_utils` that is used by the module.  Ansible includes
-  files for any :mod:`ansible.module_utils` imports in the module but not
-  any files from any other module.  So if your module uses
-  :mod:`ansible.module_utils.url` Ansible will include it for you, but if
-  your module includes `requests <https://requests.readthedocs.io/en/master/api/>`_ then you'll have to make sure that
-  the python `requests library <https://pypi.org/project/requests/>`_ is installed on the system before running the
-  module.  You can modify files in this directory if you suspect that the
-  module is having a problem in some of this boilerplate code rather than in
-  the module code you have written.
+    * The ``ansible`` directory contains the module code in ``modules`` as well as code from :mod:`ansible.module_utils` that is used by the module. Ansible includes files for any :mod:`ansible.module_utils` imports in the module but not any files from any other module. If your module uses :mod:`ansible.module_utils.url` Ansible will include it for you. But if your module includes `requests <https://requests.readthedocs.io/en/master/api/>`_, then you'll have to make sure that the Python `requests library <https://pypi.org/project/requests/>`_ is installed on the system before running the module.
 
-Once you edit the code or arguments in the exploded tree you need some way to
-run it.  There's a separate wrapper subcommand for this:
+    You can modify files in this directory if you suspect that the module is having a problem in some of this boilerplate code rather than in the module code you have written.
 
-.. code-block:: shell-session
+#. Once you edit the code or arguments in the exploded tree, use the ``execute`` subcommand to run it:
 
-    $ python /home/badger/.ansible/tmp/ansible-tmp-1461434734.35-235318071810595/ping execute
-    {"invocation": {"module_args": {"data": "debugging_session"}}, "changed": false, "ping": "debugging_session"}
+    .. code-block:: shell-session
+
+      $ python AnsiballZ_ping.py execute
+      {"invocation": {"module_args": {"data": "debugging_session"}}, "changed": false, "ping": "debugging_session"}
+
+    This subcommand inserts the absolute path to ``debug_dir`` as the first item in ``sys.path`` and invokes the script using the arguments in the ``args`` file. You can continue to run the module like this until you understand the problem. Then you can copy the changes back into your real module file and test that the real module works via ``ansible`` or ``ansible-playbook``.
+
+
+.. _simple_debugging:
+
+Simple debugging
+================
+
+The easiest way to run a debugger in a module, either local or remote, is to use `epdb <https://pypi.org/project/epdb/>`_. Add ``import epdb; epdb.serve()`` in the module code on the control node at the desired break point. To connect to the debugger, run ``epdb.connect()``. See the `epdb documentation <https://pypi.org/project/epdb/>`_ for how to specify the ``host`` and ``port``. If connecting to a remote node, make sure to use a port that is allowed by any firewall between the control node and the remote node.
+
+This technique should work with any remote debugger, but we do not guarantee any particual remote debugging tool will work.
+
+The `q <https://pypi.org/project/q/>`_ library is another very useful debugging tool.
+
+Since ``print()`` statements do not work inside modules, raising an exception is a good approach if you just want to see some specific data. Put ``raise Exception(some_value)`` somewhere in the module and run it normally. Ansible will handle this exception, pass the message back to the control node, and display it.
 
-This subcommand takes care of setting the PYTHONPATH to use the exploded
-:file:`debug_dir/ansible/module_utils` directory and invoking the script using
-the arguments in the :file:`args` file.  You can continue to run it like this
-until you understand the problem.  Then you can copy it back into your real
-module file and test that the real module works via :command:`ansible` or
-:command:`ansible-playbook`.

docs/docsite/rst/dev_guide/developing_api.rst

@@ -6,7 +6,7 @@ Python API
 
 .. contents:: Topics
 
-.. note:: This API is intended for internal Ansible use. Ansible may make changes to this API at any time that could break backward compatibility with older versions of the API. Because of this, external use is not supported by Ansible.
+.. note:: This API is intended for internal Ansible use. Ansible may make changes to this API at any time that could break backward compatibility with older versions of the API. Because of this, external use is not supported by Ansible. If you want to use Python API only for executing playbooks or modules, consider `ansible-runner <https://ansible-runner.readthedocs.io/en/latest/>`_ first.
 
 There are several ways to use Ansible from an API perspective.   You can use
 the Ansible Python API to control nodes, you can extend Ansible to respond to various Python events, you can
@@ -23,88 +23,10 @@ or have access control and logging demands, please see the `Ansible Tower docume
 Python API example
 ==================
 
-This example is a simple demonstration that shows how to minimally run a couple of tasks::
-
-    #!/usr/bin/env python
-
-    import json
-    import shutil
-    from ansible.module_utils.common.collections import ImmutableDict
-    from ansible.parsing.dataloader import DataLoader
-    from ansible.vars.manager import VariableManager
-    from ansible.inventory.manager import InventoryManager
-    from ansible.playbook.play import Play
-    from ansible.executor.task_queue_manager import TaskQueueManager
-    from ansible.plugins.callback import CallbackBase
-    from ansible import context
-    import ansible.constants as C
-
-    class ResultCallback(CallbackBase):
-        """A sample callback plugin used for performing an action as results come in
-
-        If you want to collect all results into a single object for processing at
-        the end of the execution, look into utilizing the ``json`` callback plugin
-        or writing your own custom callback plugin
-        """
-        def v2_runner_on_ok(self, result, **kwargs):
-            """Print a json representation of the result
-
-            This method could store the result in an instance attribute for retrieval later
-            """
-            host = result._host
-            print(json.dumps({host.name: result._result}, indent=4))
-
-    # since the API is constructed for CLI it expects certain options to always be set in the context object
-    context.CLIARGS = ImmutableDict(connection='local', module_path=['/to/mymodules'], forks=10, become=None,
-                                    become_method=None, become_user=None, check=False, diff=False)
-
-    # initialize needed objects
-    loader = DataLoader() # Takes care of finding and reading yaml, json and ini files
-    passwords = dict(vault_pass='secret')
-
-    # Instantiate our ResultCallback for handling results as they come in. Ansible expects this to be one of its main display outlets
-    results_callback = ResultCallback()
-
-    # create inventory, use path to host config file as source or hosts in a comma separated string
-    inventory = InventoryManager(loader=loader, sources='localhost,')
-
-    # variable manager takes care of merging all the different sources to give you a unified view of variables available in each context
-    variable_manager = VariableManager(loader=loader, inventory=inventory)
-
-    # create data structure that represents our play, including tasks, this is basically what our YAML loader does internally.
-    play_source =  dict(
-            name = "Ansible Play",
-            hosts = 'localhost',
-            gather_facts = 'no',
-            tasks = [
-                dict(action=dict(module='shell', args='ls'), register='shell_out'),
-                dict(action=dict(module='debug', args=dict(msg='{{shell_out.stdout}}')))
-             ]
-        )
-
-    # Create play object, playbook objects use .load instead of init or new methods,
-    # this will also automatically create the task objects from the info provided in play_source
-    play = Play().load(play_source, variable_manager=variable_manager, loader=loader)
-
-    # Run it - instantiate task queue manager, which takes care of forking and setting up all objects to iterate over host list and tasks
-    tqm = None
-    try:
-        tqm = TaskQueueManager(
-                  inventory=inventory,
-                  variable_manager=variable_manager,
-                  loader=loader,
-                  passwords=passwords,
-                  stdout_callback=results_callback,  # Use our custom callback instead of the ``default`` callback plugin, which prints to stdout
-              )
-        result = tqm.run(play) # most interesting data for a play is actually sent to the callback's methods
-    finally:
-        # we always need to cleanup child procs and the structures we use to communicate with them
-        if tqm is not None:
-            tqm.cleanup()
-
-        # Remove ansible tmpdir
-        shutil.rmtree(C.DEFAULT_LOCAL_TMP, True)
+This example is a simple demonstration that shows how to minimally run a couple of tasks:
 
+.. literalinclude:: ../../../../examples/scripts/uptime.py
+   :language: python
 
 .. note:: Ansible emits warnings and errors via the display object, which prints directly to stdout, stderr and the Ansible log.