New release v2.10.0rc2

Co-authored-by: Matt Clay <mclay@redhat.com>

.github/ISSUE_TEMPLATE/config.yml

@@ -20,7 +20,7 @@ contact_links:
   about: Please ask and answer usage questions here
 - name: ⚡ Working groups
   url: https://github.com/ansible/community/wiki
-  about: Interedted in improving a specific area? Become a part of a working group!
+  about: Interested in improving a specific area? Become a part of a working group!
 - name: 💼 For Enterprise
   url: https://www.ansible.com/products/engine?utm_medium=github&utm_source=issue_template_chooser
-  about: Red Hat offers supported builds of Ansible Engine
+  about: Red Hat offers support for the Ansible Automation Platform

.gitignore

@@ -37,6 +37,8 @@ docs/docsite/rst/cli/ansible.rst
 docs/docsite/rst/dev_guide/collections_galaxy_meta.rst
 docs/docsite/rst/dev_guide/testing/sanity/index.rst.new
 docs/docsite/rst/modules/*.rst
+docs/docsite/rst/collections/*.rst
+docs/docsite/rst/collections/*/*.rst
 docs/docsite/rst/playbooks_directives.rst
 docs/docsite/rst/plugins_by_category.rst
 docs/docsite/rst/plugins/*/*.rst

MANIFEST.in

@@ -32,5 +32,6 @@ recursive-include test/units *
 include Makefile
 include MANIFEST.in
 include changelogs/CHANGELOG*.rst
+include changelogs/changelog.yaml
 recursive-include hacking/build_library *.py
 include hacking/build-ansible.py

Makefile

@@ -275,7 +275,7 @@ linkcheckdocs:
 .PHONY: generate_rst
 generate_rst: lib/ansible/cli/*.py
 	mkdir -p ./docs/man/man1/ ; \
-	PYTHONPATH=./lib $(GENERATE_CLI) --template-file=docs/templates/man.j2 --output-dir=docs/man/man1/ --output-format man lib/ansible/cli/*.py
+	$(GENERATE_CLI) --template-file=docs/templates/man.j2 --output-dir=docs/man/man1/ --output-format man lib/ansible/cli/*.py
 
 
 docs: generate_rst

changelogs/CHANGELOG-v2.10.rst

@@ -0,0 +1,690 @@
+=======================================================
+Ansible Base 2.10 "When the Levee Breaks" Release Notes
+=======================================================
+
+.. contents:: Topics
+
+
+v2.10.0rc2
+==========
+
+Release Summary
+---------------
+
+| Release Date: 2020-07-23
+| `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__
+
+
+Bugfixes
+--------
+
+- Stop adding the connection variables to the output results
+
+v2.10.0rc1
+==========
+
+Release Summary
+---------------
+
+| Release Date: 2020-07-23
+| `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__
+
+
+Minor Changes
+-------------
+
+- Add an example for using var in with_sequence (https://github.com/ansible/ansible/issues/68836).
+- Add standard Python 2/3 compatibility boilerplate to setup script, module_utils and docs_fragments which were missing them.
+- Command module: Removed suggestions to use modules which have moved to collections and out of ansible-base
+- The plugin loader now keeps track of the collection where a plugin was resolved to, in particular whether the plugin was loaded from ansible-base's internal paths (``ansible.builtin``) or from user-supplied paths (no collection name).
+- ansible-galaxy - Add installation successful message
+- ansible-galaxy - Change the output verbosity level of the download message from 3 to 0 (https://github.com/ansible/ansible/issues/70010)
+- ansible-test - Provisioning of RHEL instances now includes installation of pinned versions of ``packaging`` and ``pyparsing`` to match the downstream vendored versions.
+- ansible-test - Report the correct line number in the ``yamllint`` sanity test when reporting ``libyaml`` parse errors in module documentation.
+- conditionals - change the default of CONDITIONAL_BARE_VARS to False (https://github.com/ansible/ansible/issues/70682).
+- debconf - add a note about no_log=True since module might expose sensitive information to logs (https://github.com/ansible/ansible/issues/32386).
+- pipe lookup - update docs for Popen with shell=True usages (https://github.com/ansible/ansible/issues/70159).
+
+Bugfixes
+--------
+
+- **security issue** atomic_move - change default permissions when creating temporary files so they are not world readable (https://github.com/ansible/ansible/issues/67794) (CVE-2020-1736)
+- Address the deprecation of the use of stdlib distutils in packaging. It's a short-term hotfix for the problem (https://github.com/ansible/ansible/issues/70456, https://github.com/pypa/setuptools/issues/2230, https://github.com/pypa/setuptools/commit/bd110264)
+- Allow TypeErrors on Undefined variables in filters to be handled or deferred when processing for loops.
+- Ansible output now uses stdout to determine column width instead of stdin
+- Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)
+- JSON Encoder - Ensure we treat single vault encrypted values as strings (https://github.com/ansible/ansible/issues/70784)
+- Python module_utils finder - refactor logic to eliminate many corner cases, remove recursion, fix base module_utils redirections
+- SSH plugin - Improve error message when ssh client is not found on the host
+- Sanitize no_log values from any response keys that might be returned from the uri module.
+- TaskExecutor - Handle unexpected errors as failed while post validating loops (https://github.com/ansible/ansible/issues/70050).
+- Template connection variables before using them (https://github.com/ansible/ansible/issues/70598).
+- Terminal plugins - add "\e[m" to the list of ANSI sequences stripped from device output
+- The `ansible_become` value was not being treated as a boolean value when set in an INI format inventory file (fixes bug https://github.com/ansible/ansible/issues/70476).
+- The machine-readable changelog ``changelogs/changelog.yaml`` is now contained in the release.
+- Vault - Allow single vault encrypted values to be used directly as module parameters. (https://github.com/ansible/ansible/issues/68275)
+- action plugins - change all action/module delegations to use FQ names while allowing overrides (https://github.com/ansible/ansible/issues/69788)
+- add constraints file for ``anisble_runner`` test since an update to ``psutil`` is now causing test failures
+- add magic/connection vars updates from delegated host info.
+- ansible-doc - collection name for plugin top-level deprecation was not inserted when deprecating by version (https://github.com/ansible/ansible/pull/70344).
+- ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046).
+- ansible-doc - improve man page formatting to avoid problems when YAML anchors are used (https://github.com/ansible/ansible/pull/70045).
+- ansible-doc - include the collection name in the text output (https://github.com/ansible/ansible/pull/70401).
+- ansible-test - Do not try to validate PowerShell modules ``setup.ps1``, ``slurp.ps1``, and ``async_status.ps1``
+- ansible-test - The ``ansible-doc`` sanity test now works for ``netconf`` plugins.
+- ansible-test - integration and unit test change detection now works for filter, lookup and test plugins
+- ansible-test now always uses the ``--python`` option for ``virtualenv`` to select the correct interpreter when creating environments with the ``--venv`` option
+- api - time.clock is removed in Python 3.8, add backward compatible code (https://github.com/ansible/ansible/issues/70649).
+- apt - include exception message from apt python library in error output
+- assemble - fix decrypt argument in the module (https://github.com/ansible/ansible/issues/65450).
+- basic - use PollSelector implementation when DefaultSelector fails (https://github.com/ansible/ansible/issues/70238).
+- collection metadata - ensure collection loader uses libyaml/CSafeLoader to parse collection metadata if available
+- cron - encode and decode crontab files in UTF-8 explicitly to allow non-ascii chars in cron filepath and job (https://github.com/ansible/ansible/issues/69492)
+- ensure delegated vars can resolve hostvars object and access vars from hostvars[inventory_hostname].
+- facts - account for Slackware OS with ``+`` in the name (https://github.com/ansible/ansible/issues/38760)
+- fix issue with inventory_hostname and delegated host vars mixing on connection settings.
+- if the ``type`` for a module parameter in the argument spec is callable, do not pass ``kwargs`` to avoid errors (https://github.com/ansible/ansible/issues/70017)
+- pause - handle exception when there is no stdout (https://github.com/ansible/ansible/pull/47851)
+- playbooks - detect and propagate failures in ``always`` blocks after ``rescue`` (https://github.com/ansible/ansible/issues/70000)
+- shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).
+- splunk httpapi plugin - switch from splunk.enterprise_security to splunk.es in runtime.yml to reflect upstream change of Collection Name
+- ssh connection plugin - use ``get_option()`` rather than ``_play_context`` to ensure ``ANSBILE_SSH_ARGS`` are applied properly (https://github.com/ansible/ansible/issues/70437)
+- user - don't create home directory and missing parents when create_home == false (https://github.com/ansible/ansible/pull/70600).
+- win setup - Fix redirection path for the windows setup module
+- windows async - use full path when calling PowerShell to reduce reliance on environment vars being correct - https://github.com/ansible/ansible/issues/70655
+- winrm - preserve winrm forensic data on put_file failures
+
+v2.10.0b1
+=========
+
+Release Summary
+---------------
+
+| Release Date: 2020-06-17
+| `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__
+
+
+Major Changes
+-------------
+
+- Both ansible-doc and ansible-console's help command will error for modules and plugins whose return documentation cannot be parsed as YAML. All modules and plugins passing ``ansible-test sanity --test yamllint`` will not be affected by this.
+- Collections may declare a list of supported/tested Ansible versions for the collection. A warning is issued if a collection does not support the Ansible version that loads it (can also be configured as silent or a fatal error). Collections that do not declare supported Ansible versions do not issue a warning/error.
+- Plugin routing allows collections to declare deprecation, redirection targets, and removals for all plugin types.
+- Plugins that import module_utils and other ansible namespaces that have moved to collections should continue to work unmodified.
+- Routing data built into Ansible 2.10 ensures that 2.9 content should work unmodified on 2.10. Formerly included modules and plugins that were moved to collections are still accessible by their original unqualified names, so long as their destination collections are installed.
+- When deprecations are done in code, they to specify a ``collection_name`` so that deprecation warnings can mention which collection - or ansible-base - is deprecating a feature. This affects all ``Display.deprecated()`` or ``AnsibleModule.deprecate()`` or ``Ansible.Basic.Deprecate()`` calls, and ``removed_in_version``/``removed_at_date`` or ``deprecated_aliases`` in module argument specs.
+- ansible-test now uses a different ``default`` test container for Ansible Collections
+
+Minor Changes
+-------------
+
+- 'Edit on GitHub' link for plugin, cli documentation fixed to navigate to correct plugin, cli source.
+- Add 'auth_url' field to galaxy server config stanzas in ansible.cfg The url should point to the token_endpoint of a Keycloak server.
+- Add --ask-vault-password and --vault-pass-file options to ansible cli commands
+- Add ``--pre`` flag to ``ansible-galaxy collection install`` to allow pulling in the most recent pre-release version of a collection (https://github.com/ansible/ansible/issues/64905)
+- Add a global toggle to control when vars plugins are executed (per task by default for backward compatibility or after importing inventory).
+- Add a new config parameter, WIN_ASYNC_STARTUP_TIMEOUT, which allows configuration of the named pipe connection timeout under Windows when launching async tasks.
+- Add a per-plugin stage option to override the global toggle to control the execution of individual vars plugins (per task, after inventory, or both).
+- Add an additional check for importing journal from systemd-python module (https://github.com/ansible/ansible/issues/60595).
+- Add new magic variable ``ansible_collection`` that contains the collection name
+- Add new magic variable ``ansible_role_name`` that contains the FQCN of the role
+- Added PopOS as a part of Debian OS distribution family (https://github.com/ansible/ansible/issues/69286).
+- Added hostname support for PopOS in hostname module.
+- Added openEuler OS in RedHat OS Family.
+- Added the ability to set ``DEFAULT_NO_TARGET_SYSLOG`` through the ``ansible_no_target_syslog`` variable on a task
+- Ansible CLI fails with warning if extra_vars parameter is used with filename without @ sign (https://github.com/ansible/ansible/issues/51857).
+- Ansible modules created with ``add_file_common_args=True`` added a number of undocumented arguments which were mostly there to ease implementing certain action plugins. The undocumented arguments ``src``, ``follow``, ``force``, ``content``, ``backup``, ``remote_src``, ``regexp``, ``delimiter``, and ``directory_mode`` are now no longer added. Modules relying on these options to be added need to specify them by themselves. Also, action plugins relying on these extra elements in ``FILE_COMMON_ARGUMENTS`` need to be adjusted.
+- Ansible now allows deprecation by date instead of deprecation by version. This is possible for plugins and modules (``meta/runtime.yml`` and ``deprecated.removed_at_date`` in ``DOCUMENTATION``, instead of ``deprecated.removed_in``), for plugin options (``deprecated.date`` instead of ``deprecated.version`` in ``DOCUMENTATION``), for module options (``removed_at_date`` instead of ``removed_in_version`` in argument spec), and for module option aliases (``deprecated_aliases.date`` instead of ``deprecated_aliases.version`` in argument spec).
+- Ansible should fail with error when non-existing limit file is provided in command line.
+- Ansible.Basic - Added the ability to specify multiple fragments to load in a generic way for modules that use a module_util with fragment options
+- Ansible.Basic.cs - Added support for ``deprecated_aliases`` to deprecated aliases in a standard way
+- Ansible.ModuleUtils.WebRequest - Move username and password aliases out of util to avoid option name collision
+- Change order of arguments in ansible cli to use --ask-vault-password and --vault-password-file by default
+- CollectionRequirement - Add a metadata property to update and retrieve the _metadata attribute.
+- Enable Ansible Collections loader to discover and import collections from ``site-packages`` dir and ``PYTHONPATH``-added locations.
+- Enable testing the AIX platform as a remote OS in ansible-test
+- Flatten the directory hierarchy of modules
+- Ignore plesk-release file while parsing distribution release (https://github.com/ansible/ansible/issues/64101).
+- Openstack inventory script is migrated to ansible-openstack-collection, adjusted the link in documentation accordingly.
+- Openstack inventory script is moved to openstack.cloud from community.general.
+- PowerShell Add-Type - Add an easier way to reference extra types when compiling C# code on PowerShell Core
+- PowerShell Add-Type - Added the ``X86`` and ``AMD64`` preprocessor symbols for conditional compiling
+- Prevent losing useful error information by including both the loop and the conditional error messages (https://github.com/ansible/ansible/issues/66529)
+- Provides additional information about collection namespace name restrictions (https://github.com/ansible/ansible/issues/65151).
+- Raise error when no task file is provided to import_tasks (https://github.com/ansible/ansible/issues/54095).
+- Refactor test_distribution_version testcases.
+- Remove the deprecation message for the ``TRANSFORM_INVALID_GROUP_CHARS`` setting. (https://github.com/ansible/ansible/issues/61889)
+- Removed extras_require support from setup.py (and [azure] extra). Requirements will float with the collections, so it's not appropriate for ansible-base to host requirements for them any longer.
+- Simplify dict2items filter example in loop documentation (https://github.com/ansible/ansible/issues/65505).
+- Templating - Add globals to the jinja2 environment at ``Templar`` instantiation, instead of customizing the template object. Only customize the template object, to disable lookups. (https://github.com/ansible/ansible/pull/69278)
+- Templating - Add support to auto unroll generators produced by jinja2 filters, to prevent the need of explicit use of ``|list`` (https://github.com/ansible/ansible/pull/68014)
+- The results queue and counter for results are now split for standard / handler results. This allows the governing strategy to be truly independent from the handler strategy, which basically follows the linear methodology.
+- Update required library message with correct grammer in basic.py.
+- Updated inventory script location for EC2, Openstack, and Cobbler after collection (https://github.com/ansible/ansible/issues/68897).
+- Updated inventory script location for infoblox, ec2 and other after collection migration (https://github.com/ansible/ansible/issues/69139).
+- Updates ``ansible_role_names``, ``ansible_play_role_names``, and ``ansible_dependent_role_names`` to include the FQCN
+- Use OrderedDict by default when importing mappings from YAML.
+- Windows - Add a check for the minimum PowerShell version so we can create a friendly error message on older hosts
+- Windows - add deprecation notice in the Windows setup module when running on Server 2008, 2008 R2, and Windows 7
+- `AnsibleModule.fail_json()` has always required that a message be passed in which informs the end user why the module failed.  In the past this message had to be passed as the `msg` keyword argument but it can now be passed as the first positional argument instead.
+- ``AnsibleModule.load_file_common_arguments`` now allows to simply override ``path``.
+- add mechanism for storing warnings and deprecations globally and not attached to an ``AnsibleModule`` object (https://github.com/ansible/ansible/pull/58993)
+- added more ways to configure new uri options in 2.10.
+- ansible-doc - improve suboptions formatting (https://github.com/ansible/ansible/pull/69795).
+- ansible-doc - now indicates if an option is added by a doc fragment from another collection by prepending the collection name, or ``ansible.builtin`` for ansible-base, to the version number.
+- ansible-doc - return values will be properly formatted (https://github.com/ansible/ansible/pull/69796).
+- ansible-galaxy - Add ``download`` option for ``ansible-galaxy collection`` to download collections and their dependencies for an offline install
+- ansible-galaxy - Add a `verify` subcommand to `ansible-galaxy collection`. The collection found on the galaxy server is downloaded to a tempfile to compare the checksums of the files listed in the MANIFEST.json and the FILES.json with the contents of the installed collection.
+- ansible-galaxy - Added the ability to display the progress wheel through the C.GALAXY_DISPLAY_PROGRESS config option. Also this now defaults to displaying the progress wheel if stdout has a tty.
+- ansible-galaxy - Added the ability to ignore further files and folders using a pattern with the ``build_ignore`` key in a collection's ``galaxy.yml`` (https://github.com/ansible/ansible/issues/59228).
+- ansible-galaxy - Allow installing collections from git repositories.
+- ansible-galaxy - Always ignore the ``tests/output`` directory when building a collection as it is used by ``ansible-test`` for test output (https://github.com/ansible/ansible/issues/59228).
+- ansible-galaxy - Display message if both collections and roles are specified in a requirements file but can't be installed together.
+- ansible-galaxy - Install both collections and roles with ``ansible-galaxy install -r requirements.yml`` in certain scenarios.
+- ansible-galaxy - Requirement entries for collections now support a 'type' key to indicate whether the collection is a galaxy artifact, file, url, or git repo.
+- ansible-galaxy - Support both 'galaxy.yml' and 'galaxy.yaml' files for collections.
+- ansible-galaxy - add ``--token`` argument which is the same as ``--api-key`` (https://github.com/ansible/ansible/issues/65955)
+- ansible-galaxy - add ``collection list`` command for listing installed collections (https://github.com/ansible/ansible/pull/65022)
+- ansible-galaxy - add ``validate_collection_path()`` utility function ()
+- ansible-galaxy - add collections path argument
+- ansible-galaxy - allow role to define dependency requirements that will be only installed by defining them in ``meta/requirements.yml`` (https://github.com/ansible/proposals/issues/57)
+- ansible-test - --docker flag now has an associated --docker-terminate flag which controls if and when the docker container is removed following tests
+- ansible-test - Add a test to prevent ``state=get``
+- ansible-test - Add a test to prevent ``state=list`` and ``state=info``
+- ansible-test - Add a verbosity option for displaying warnings.
+- ansible-test - Add support for Python 3.9.
+- ansible-test - Added CI provider support for Azure Pipelines.
+- ansible-test - Added a ``ansible-test coverage analyze targets filter`` command to filter aggregated coverage reports by path and/or target name.
+- ansible-test - Added a ``ansible-test coverage analyze targets`` command to analyze integration test code coverage by test target.
+- ansible-test - Added support for Ansible Core CI request signing for Shippable.
+- ansible-test - Added support for testing on Fedora 32.
+- ansible-test - General code cleanup.
+- ansible-test - Now includes testing support for RHEL 8.2
+- ansible-test - Refactor code to consolidate filesystem access and improve handling of encoding.
+- ansible-test - Refactored CI related logic into a basic provider abstraction.
+- ansible-test - Remove obsolete support for provisioning remote vCenter instances. The supporting services are no longer available.
+- ansible-test - Support writing compact JSON files instead of formatting and indenting the output.
+- ansible-test - Update Ubuntu 18.04 test container to version 1.13 which includes ``venv``
+- ansible-test - Update ``default-test-container`` to version 1.11, which includes Python 3.9.0a4.
+- ansible-test - Updated the default test containers to include Python 3.9.0b3.
+- ansible-test - Upgrade OpenSUSE containers to use Leap 15.1.
+- ansible-test - Upgrade distro test containers from 1.16.0 to 1.17.0
+- ansible-test - Upgrade from ansible-base-test-container 1.1 to 2.2
+- ansible-test - Upgrade from default-test-container 2.1 to 2.2
+- ansible-test - ``mutually_exclusive``, ``required_if``, ``required_by``, ``required_together`` and ``required_one_of`` in modules are now validated.
+- ansible-test - ``validate-modules`` now also accepts an ISO 8601 formatted date as ``deprecated.removed_at_date``, instead of requiring a version number in ``deprecated.removed_in``.
+- ansible-test - ``validate-modules`` now makes sure that module documentation deprecation removal version and/or date matches with removal version and/or date in meta/runtime.yml.
+- ansible-test - ``validate-modules`` now validates all version numbers in documentation and argument spec. Version numbers for collections are checked for being valid semantic versioning version number strings.
+- ansible-test - add ``validate-modules`` tests for ``removed_in_version`` and ``deprecated_aliases`` (https://github.com/ansible/ansible/pull/66920/).
+- ansible-test - add check for ``print()`` calls in modules and module_utils.
+- ansible-test - added a ``--no-pip-check`` option
+- ansible-test - added a ``--venv-system-site-packages`` option for use with the ``--venv`` option
+- ansible-test - added new ``changelog`` test, which runs if a `antsibull-changelog <https://pypi.org/project/antsibull-changelog/>`_ configuration or files in ``changelogs/fragments/`` are found (https://github.com/ansible/ansible/pull/69313).
+- ansible-test - allow delegation config to specify equivalents to the ``--no-pip-check``, ``--disable-httptester`` and `--no-temp-unicode`` options
+- ansible-test - allow sanity tests to check for optional errors by specifying ``--enable-optional-errors`` (https://github.com/ansible/ansible/pull/66920/).
+- ansible-test - also run the ``ansible-doc`` sanity test with ``--json`` to ensure that the documentation does not contain something that cannot be exported as JSON (https://github.com/ansible/ansible/issues/69238).
+- ansible-test - enable deprecated version testing for modules and ``module.deprecate()`` calls (https://github.com/ansible/ansible/pull/66920/).
+- ansible-test - extend alias validation.
+- ansible-test - fixed ``units`` command with ``--docker`` to (mostly) work under podman
+- ansible-test - improve module validation so that ``default``, ``sample`` and ``example`` contain JSON values and not arbitrary YAML values, like ``datetime`` objects or dictionaries with non-string keys.
+- ansible-test - module validation will now consider arguments added by ``add_file_common_arguments=True`` correctly.
+- ansible-test - switch from testing RHEL 8.0 and RHEL 8.1 Beta to RHEL 8.1
+- ansible-test - the argument spec of modules is now validated by a YAML schema.
+- ansible-test - the module validation code now checks whether ``elements`` documentation for options matches the argument_spec.
+- ansible-test - the module validation code now checks whether ``elements`` is defined when ``type=list``
+- ansible-test - the module validation code now checks whether ``requirement`` for options is documented correctly.
+- ansible-test add pyparsing constraint for Python 2.x to avoid compatibility issues with the upcoming pyparsing 3 release
+- ansible-test defaults to redacting sensitive values (disable with the ``--no-redact`` option)
+- ansible-test has been updated to use ``default-test-container:1.13`` which includes fewer Python requirements now that most modules and tests have been migrated to collections.
+- ansible-test no longer detects ``git`` submodule directories as files.
+- ansible-test no longer provides a ``--tox`` option. Use the ``--venv`` option instead. This only affects testing the Ansible source. The feature was never available for Ansible Collections or when running from an Ansible install.
+- ansible-test no longer tries to install sanity test dependencies on unsupported Python versions
+- ansible-test now checks for the minimum and maximum supported versions when importing ``coverage``
+- ansible-test now filters out unnecessary warnings and messages from pip when installing its own requirements
+- ansible-test now has a ``--list-files`` option to list files using the ``env`` command.
+- ansible-test now includes the ``pylint`` plugin ``mccabe`` in optional sanity tests enabled with ``--enable-optional-errors``
+- ansible-test now places the ansible source and collections content in separate directories when using the ``--docker`` or ``--remote`` options.
+- ansible-test now provides a more helpful error when loading coverage files created by ``coverage`` version 5 or later
+- ansible-test now supports provisioning of network resources when testing network collections
+- ansible-test now supports skip aliases in the format ``skip/{arch}/{platform}`` and ``skip/{arch}/{platform}/{version}`` where ``arch`` can be ``power``. These aliases are only effective for the ``--remote`` option.
+- ansible-test now supports skip aliases in the format ``skip/{platform}/{version}`` for the ``--remote`` option. This is preferred over the older ``skip/{platform}{version}`` format which included no ``/`` between the platform and version.
+- ansible-test now supports testing against RHEL 7.8 when using the ``--remote`` option.
+- ansible-test now supports the ``--remote power/centos/7`` platform option.
+- ansible-test now validates the schema of ansible_builtin_runtime.yml and a collections meta/runtime.yml file.
+- ansible-test provides clearer error messages when failing to detect the provider to use with the ``--remote`` option.
+- ansible-test provisioning of network devices for ``network-integration`` has been updated to use collections.
+- ansible_native_concat() - use ``to_text`` function rather than Jinja2's ``text_type`` which has been removed in Jinja2 master branch.
+- apt - Implemented an exponential backoff behaviour when retrying to update the cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
+- apt_repository - Implemented an exponential backoff behaviour when retrying to update the apt cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
+- blockinfile - Update module documentation to clarify insertbefore/insertafter usage.
+- callbacks - Allow modules to return `None` as before/after entries for diff. This should make it easier for modules to report the "not existing" state of the entity they touched.
+- combine filter - now accept a ``list_merge`` argument which modifies its behaviour when the hashes to merge contain arrays/lists.
+- config - accept singular version of ``collections_path`` ini setting and ``ANSIBLE_COLLECTIONS_PATH`` environment variable setting
+- core filters - Adding ``path_join`` filter to the core filters list
+- distro - Update bundled version of distro from 1.4.0 to 1.5.0
+- dnf - Properly handle idempotent transactions with package name wildcard globs (https://github.com/ansible/ansible/issues/62809)
+- dnf - Properly handle module AppStreams that don't define stream (https://github.com/ansible/ansible/issues/63683)
+- dnf param to pass allowerasing
+- downstream packagers may install packages under ansible._vendor, which will be added to head of sys.path at ansible package load
+- file - specifying ``src`` without ``state`` is now an error
+- get_bin_path() - change the interface to always raise ``ValueError`` if the command is not found (https://github.com/ansible/ansible/pull/56813)
+- get_url - Remove deprecated string format support for the headers option (https://github.com/ansible/ansible/issues/61891)
+- git - added an ``archive_prefix`` option to set a prefix to add to each file path in archive
+- host_group_vars plugin - Require whitelisting and whitelist by default.
+- new magic variable - ``ansible_config_file`` - full path of used Ansible config file
+- package_facts.py - Add support for Pacman package manager.
+- plugin loader - Add MODULE_IGNORE_EXTS config option to skip over certain extensions when looking for script and binary modules.
+- powershell (shell plugin) - Fix `join_path` to support UNC paths (https://github.com/ansible/ansible/issues/66341)
+- regexp_replace filter - add multiline support for regex_replace filter (https://github.com/ansible/ansible/issues/61985)
+- rename ``_find_existing_collections()`` to ``find_existing_collections()`` to reflect its use across multiple files
+- reorganized code for the ``ansible-test coverage`` command for easier maintenance and feature additions
+- service_facts - Added undocumented 'indirect' and 'static' as service status (https://github.com/ansible/ansible/issues/69752).
+- ssh - connection plugin now supports a new variable ``sshpass_prompt`` which gets passed to ``sshpass`` allowing the user to set a custom substring to search for a password prompt (requires sshpass 1.06+)
+- systemd - default scope is now explicitly "system"
+- tests - Add new ``truthy`` and ``falsy`` jinja2 tests to evaluate the truthiness or falsiness of a value
+- to_nice_json filter - Removed now-useless exception handler
+- to_uuid - add a named parameter to let the user optionally set a custom namespace
+- update ansible-test default-test-container from version 1.13 to 1.14, which includes an update from Python 3.9.0a6 to Python 3.9.0b1
+- update ansible-test default-test-container from version 1.9.1 to 1.9.2
+- update ansible-test default-test-container from version 1.9.2 to 1.9.3
+- update ansible-test default-test-container from version 1.9.3 to 1.10.1
+- update ansible-test images to 1.16.0, which includes system updates and pins CentOS versions
+- uri/galaxy - Add new ``prepare_multipart`` helper function for creating a ``multipart/form-data`` body (https://github.com/ansible/ansible/pull/69376)
+- url_lookup_plugin - add parameters to match what is available in ``module_utils/urls.py``
+- user - allow groups, append parameters with local
+- user - usage of ``append: True`` without setting a list of groups. This is currently a no-op with a warning, and will change to an error in 2.14. (https://github.com/ansible/ansible/pull/65795)
+- validate-modules checks for deprecated in collections against meta/runtime.yml
+- validation - Sort missing parameters in exception message thrown by check_required_arguments
+- vars plugins - Support vars plugins in collections by adding the ability to whitelist plugins.
+- vars_prompt - throw error when encountering unsupported key
+- win_package - Added proxy support for retrieving packages from a URL - https://github.com/ansible/ansible/issues/43818
+- win_package - Added support for ``.appx``, ``.msix``, ``.appxbundle``, and ``.msixbundle`` package - https://github.com/ansible/ansible/issues/50765
+- win_package - Added support for ``.msp`` packages - https://github.com/ansible/ansible/issues/22789
+- win_package - Added support for specifying the HTTP method when getting files from a URL - https://github.com/ansible/ansible/issues/35377
+- win_package - Read uninstall strings from the ``QuietUninstallString`` if present to better support argumentless uninstalls of registry based packages.
+- win_package - Scan packages in the current user's registry hive - https://github.com/ansible/ansible/issues/45950
+- windows collections - Support relative module util imports in PowerShell modules and module_utils
+
+Deprecated Features
+-------------------
+
+- Using the DefaultCallback without the correspodning doc_fragment or copying the documentation.
+- hash_behaviour - Deprecate ``hash_behaviour`` for future removal.
+- script inventory plugin - The 'cache' option is deprecated and will be removed in 2.12. Its use has been removed from the plugin since it has never had any effect.
+
+Removed Features (previously deprecated)
+----------------------------------------
+
+- core - remove support for ``check_invalid_arguments`` in ``AnsibleModule``, ``AzureModule`` and ``UTMModule``.
+
+Security Fixes
+--------------
+
+- **security issue** - Convert CLI provided passwords to text initially, to prevent unsafe context being lost when converting from bytes->text during post processing of PlayContext. This prevents CLI provided passwords from being incorrectly templated (CVE-2019-14856)
+- **security issue** - Redact cloud plugin secrets in ansible-test when running integration tests using cloud plugins. Only present in 2.9.0b1.
+- **security issue** - TaskExecutor - Ensure we don't erase unsafe context in TaskExecutor.run on bytes. Only present in 2.9.0beta1 (https://github.com/ansible/ansible/issues/62237)
+- **security issue** - The ``subversion`` module provided the password via the svn command line option ``--password`` and can be retrieved from the host's /proc/<pid>/cmdline file. Update the module to use the secure ``--password-from-stdin`` option instead, and add a warning in the module and in the documentation if svn version is too old to support it. (CVE-2020-1739)
+- **security issue** - Update ``AnsibleUnsafeText`` and ``AnsibleUnsafeBytes`` to maintain unsafe context by overriding ``.encode`` and ``.decode``. This prevents future issues with ``to_text``, ``to_bytes``, or ``to_native`` removing the unsafe wrapper when converting between string types (CVE-2019-14856)
+- **security issue** - properly hide parameters marked with ``no_log`` in suboptions when invalid parameters are passed to the module (CVE-2019-14858)
+- **security issue** win_unzip - normalize paths in archive to ensure extracted files do not escape from the target directory (CVE-2020-1737)
+- **security_issue** - create temporary vault file with strict permissions when editing and prevent race condition (CVE-2020-1740)
+- Ensure we get an error when creating a remote tmp if it already exists. CVE-2020-1733
+- In fetch action, avoid using slurp return to set up dest, also ensure no dir traversal CVE-2020-1735.
+- ansible-galaxy - Error when install finds a tar with a file that will be extracted outside the collection install directory - CVE-2020-10691
+
+Bugfixes
+--------
+
+- ActionBase - Add new ``cleanup`` method that is explicitly run by the ``TaskExecutor`` to ensure that the shell plugins ``tmpdir`` is always removed. This change means that individual action plugins need not be responsible for removing the temporary directory, which ensures that we don't have code paths that accidentally leave behind the temporary directory.
+- Add example setting for ``collections_paths`` parameter to ``examples/ansible.cfg``
+- Add missing gcp modules to gcp module defaults group
+- Added support for Flatcar Container Linux in distribution and hostname modules. (https://github.com/ansible/ansible/pull/69627)
+- Added support for OSMC distro in hostname module (https://github.com/ansible/ansible/issues/66189).
+- Allow tasks to notify a fqcn handler name (https://github.com/ansible/ansible/issues/68181)
+- An invalid value is hard to track down if you don't know where it came from, return field name instead.
+- Ansible.Basic - Fix issue when setting a ``no_log`` parameter to an empty string - https://github.com/ansible/ansible/issues/62613
+- Ansible.ModuleUtils.WebRequest - actually set no proxy when ``use_proxy: no`` is set on a Windows module - https://github.com/ansible/ansible/issues/68528
+- AnsibleDumper - Add a representer for AnsibleUnsafeBytes (https://github.com/ansible/ansible/issues/62562).
+- AnsibleModule.run_command() - set ``close_fds`` to ``False`` on Python 2 if ``pass_fds`` are passed to ``run_command()``. Since ``subprocess.Popen()`` on Python 2 does not have the ``pass_fds`` option, there is no way to exclude a specific list of file descriptors from being closed.
+- Avoid bare select() for running commands to avoid too large file descriptor numbers failing tasks
+- Avoid running subfunctions that are passed to show_vars function when it will be a noop.
+- By passing the module_tmpdir as a parameter in the write_ssh_wrapper function instead of initalizing module_tmpdir via get_module_path()
+- CLI - the `ANSIBLE_PLAYBOOK_DIR` envvar or `playbook_dir` config can now substitute for the --playbook-dir arg on CLIs that support it (https://github.com/ansible/ansible/issues/59464)
+- Check NoneType for raw_params before proceeding in include_vars (https://github.com/ansible/ansible/issues/64939).
+- Collections - Allow a collection role to call a stand alone role, without needing to explicitly add ``ansible.legacy`` to the collection search order within the collection role. (https://github.com/ansible/ansible/issues/69101)
+- Correctly process raw_params in add_hosts.
+- Create an ``import_module`` compat util, for use across the codebase, to allow collection loading to work properly on Python26
+- DUPLICATE_YAML_DICT_KEY - Fix error output when configuration option DUPLICATE_YAML_DICT_KEY is set to error (https://github.com/ansible/ansible/issues/65366)
+- Do not keep empty blocks in PlayIterator after skipping tasks with tags.
+- Ensure DataLoader temp files are removed at appropriate times and that we observe the LOCAL_TMP setting.
+- Ensure that ``--version`` works with non-ascii ansible project paths (https://github.com/ansible/ansible/issues/66617)
+- Ensure that keywords defined as booleans are correctly interpreting their input, before patch any random string would be interpreted as False
+- Ensure we don't allow ansible_facts subkey of ansible_facts to override top level, also fix 'deprefixing' to prevent key transforms.
+- Fact Delegation - Add ability to indicate which facts must always be delegated. Primarily for ``discovered_interpreter_python`` right now, but extensible later. (https://github.com/ansible/ansible/issues/61002)
+- Fix a bug when a host was not removed from a play after ``meta: end_host`` and as a result the host was still present in ``ansible_play_hosts`` and ``ansible_play_batch`` variables.
+- Fix an issue with the ``fileglob`` plugin where passing a subdirectory of non-existent directory would cause it to fail - https://github.com/ansible/ansible/issues/69450
+- Fix case sensitivity for ``lookup()`` (https://github.com/ansible/ansible/issues/66464)
+- Fix collection install error that happened if a dependency specified dependencies to be null (https://github.com/ansible/ansible/issues/67574).
+- Fix https://github.com/ansible/galaxy-dev/issues/96 Add support for automation-hub authentication to ansible-galaxy
+- Fix incorrect "Could not match supplied host pattern" warning (https://github.com/ansible/ansible/issues/66764)
+- Fix issue git module cannot use custom `key_file` or `ssh_opts` as non-root user on system with noexec `/tmp` (https://github.com/ansible/ansible/issues/30064).
+- Fix issue git module ignores remote_tmp (https://github.com/ansible/ansible/issues/33947).
+- Fix issue where the collection loader tracebacks if ``collections_paths = ./`` is set in the config
+- Fix issue with callbacks ``set_options`` method that was not called with collections
+- Fix label lookup in the default callback for includes (https://github.com/ansible/ansible/issues/65904)
+- Fix regression when ``ansible_failed_task`` and ``ansible_failed_result`` are not defined in the rescue block (https://github.com/ansible/ansible/issues/64789)
+- Fix string parsing of inline vault strings for plugin config variable sources
+- Fix traceback when printing ``HostVars`` on native Jinja2 (https://github.com/ansible/ansible/issues/65365)
+- Fixed a bug with the copy action plugin where mode=preserve was being passed on symlink files and causing a traceback (https://github.com/ansible/ansible/issues/68471).
+- Fixed the equality check for IncludedFiles to ensure they are not accidently merged when process_include_results runs.
+- Fixes in network action plugins load from collections using module prefix (https://github.com/ansible/ansible/issues/65071)
+- Force collection names to be static so that a warning is generated because templating currently does not work (see https://github.com/ansible/ansible/issues/68704).
+- Handle empty extra vars in ansible cli (https://github.com/ansible/ansible/issues/61497).
+- Handle empty roles and empty collections in requirements.yml in ansible-galaxy install command (https://github.com/ansible/ansible/issues/68186).
+- Handle exception encountered while parsing the argument description in module when invoked via ansible-doc command (https://github.com/ansible/ansible/issues/60587).
+- Handle exception when /etc/shadow file is missing or not found, while operating user operation in user module (https://github.com/ansible/ansible/issues/63490).
+- HostVarsVars - Template the __repr__ value (https://github.com/ansible/ansible/issues/64128).
+- Make netconf plugin configurable to set ncclient device handler name in netconf plugin (https://github.com/ansible/ansible/pull/65718)
+- Make sure if a collection is supplied as a string that we transform it into a list.
+- Misc typo fixes in various documentation pages.
+- Module arguments in suboptions which were marked as deprecated with ``removed_in_version`` did not result in a warning.
+- On HTTP status code 304, return status_code
+- Plugin Metadata is supposed to have default values.  When the metadata was missing entirely, we were properly setting the defaults.  Fixed the metadata parsing so that the defaults are also set when we were missing just a few fields.
+- Prevent a race condition when running handlers using a combination of the free strategy and include_role.
+- Prevent rewriting nested Block's data in filter_tagged_tasks
+- Prevent templating unused variables for {% include %} (https://github.com/ansible/ansible/issues/68699)
+- Properly handle unicode in ``safe_eval``. (https://github.com/ansible/ansible/issues/66943)
+- Remove a temp directory created by wait_for_connection action plugin (https://github.com/ansible/ansible/issues/62407).
+- Remove the unnecessary warning about aptitude not being installed (https://github.com/ansible/ansible/issues/56832).
+- Remove unused Python imports in ``ansible-inventory``.
+- Role Installation - Ensure that a role containing files with non-ascii characters can be installed (https://github.com/ansible/ansible/issues/69133)
+- RoleRequirement - include stderr in the error message if a scm command fails (https://github.com/ansible/ansible/issues/41336)
+- Skipping of become for ``network_cli`` connections now works when ``network_cli`` is sourced from a collection.
+- Strictly check string datatype for 'tasks_from', 'vars_from', 'defaults_from', and 'handlers_from' in include_role (https://github.com/ansible/ansible/issues/68515).
+- Strip no log values from module response keys (https://github.com/ansible/ansible/issues/68400)
+- TaskQueueManager - Explicitly set the mutliprocessing start method to ``fork`` to avoid issues with the default on macOS now being ``spawn``.
+- Templating - Ansible was caching results of Jinja2 expressions in some cases where these expressions could have dynamic results, like password generation (https://github.com/ansible/ansible/issues/34144).
+- The ansible-galaxy publish command was using an incorrect URL for v3 servers. The configuration for v3 servers includes part of the path fragment that was added in the new test.
+- Update ActionBase._low_level_execute_command to honor executable (https://github.com/ansible/ansible/issues/68054)
+- Update the warning message for ``CONDITIONAL_BARE_VARS`` to list the original conditional not the value of the original conditional (https://github.com/ansible/ansible/issues/67735)
+- Use ``sys.exit`` instead of ``exit`` in ``ansible-inventory``.
+- Use fqcr from command module invocation using shell module. Fixes https://github.com/ansible/ansible/issues/69788
+- Use hostnamectl command to get current hostname for host while using systemd strategy (https://github.com/ansible/ansible/issues/59438).
+- Using --start-at-task would fail when it attempted to skip over tasks with no name.
+- Validate include args in handlers.
+- Vault - Make the single vaulted value ``AnsibleVaultEncryptedUnicode`` class work more like a string by replicating the behavior of ``collections.UserString`` from Python. These changes don't allow it to be considered a string, but most common python string actions will now work as expected. (https://github.com/ansible/ansible/pull/67823)
+- ``AnsibleUnsafe``/``AnsibleContext``/``Templar`` - Do not treat ``AnsibleUndefined`` as being "unsafe" (https://github.com/ansible/ansible/issues/65198)
+- account for empty strings in when splitting the host pattern (https://github.com/ansible/ansible/issues/61964)
+- add parameter name to warning message when values are converted to strings (https://github.com/ansible/ansible/pull/57145)
+- add_host action now correctly shows idempotency/changed status
+- added 'unimplemented' prefix to file based caching
+- added new option for default callback to compat variable to avoid old 3rd party plugins from erroring out.
+- adhoc CLI - when playbook-dir is specified and inside a collection, use default collection logic to resolve modules/actions
+- allow external collections to be created in the 'ansible' collection namespace (https://github.com/ansible/ansible/issues/59988)
+- also strip spaces around config values in pathlist as we do in list types
+- ansiballz - remove '' and '.' from sys.path to fix a permissions issue on OpenBSD with pipelining (#69320)
+- ansible command now correctly sends v2_playbook_on_start to callbacks
+- ansible-connection persists even after playbook run is completed (https://github.com/ansible/ansible/pull/61591)
+- ansible-doc - Allow and give precedence to `removed_at_date` for deprecated modules.
+- ansible-doc now properly handles removed modules/plugins
+- ansible-galaxy - Default collection install path to first path in COLLECTIONS_PATHS (https://github.com/ansible/ansible/pull/62870)
+- ansible-galaxy - Display proper error when invalid token is used for Galaxy servers
+- ansible-galaxy - Ensure we preserve the new URL when appending ``/api`` for the case where the GET succeeds on galaxy.ansible.com
+- ansible-galaxy - Expand the ``User-Agent`` to include more information and add it to more calls to Galaxy endpoints.
+- ansible-galaxy - Fix ``collection install`` when installing from a URL or a file - https://github.com/ansible/ansible/issues/65109
+- ansible-galaxy - Fix ``multipart/form-data`` body to include extra CRLF (https://github.com/ansible/ansible/pull/67942)
+- ansible-galaxy - Fix issue when compared installed dependencies with a collection having no ``MANIFEST.json`` or an empty version string in the json
+- ansible-galaxy - Fix pagination issue when retrieving role versions for install - https://github.com/ansible/ansible/issues/64355
+- ansible-galaxy - Fix up pagination searcher for collection versions on Automation Hub
+- ansible-galaxy - Fix url building to not truncate the URL (https://github.com/ansible/ansible/issues/61624)
+- ansible-galaxy - Handle the different task resource urls in API responses from publishing collection artifacts to galaxy servers using v2 and v3 APIs.
+- ansible-galaxy - Preserve symlinks when building and installing a collection
+- ansible-galaxy - Remove uneeded verbose messages when accessing local token file
+- ansible-galaxy - Return the HTTP code reason if no error msg was returned by the server - https://github.com/ansible/ansible/issues/64850
+- ansible-galaxy - Send SHA256 hashes when publishing a collection
+- ansible-galaxy - Set ``User-Agent`` to Ansible version when interacting with Galaxy or Automation Hub
+- ansible-galaxy - Treat the ``GALAXY_SERVER_LIST`` config entry that is defined but with no values as an empty list
+- ansible-galaxy - Utilize ``Templar`` for templating skeleton files, so that they have access to Ansible filters/tests/lookups (https://github.com/ansible/ansible/issues/69104)
+- ansible-galaxy - fix a bug where listing a specific role if it was not in the first path failed to find the role
+- ansible-galaxy - fix regression that prenented roles from being listed
+- ansible-galaxy - hide warning during collection installation if other installed collections do not contain a ``MANIFEST.json`` (https://github.com/ansible/ansible/issues/67490)
+- ansible-galaxy - properly list roles when the role name also happens to be in the role path (https://github.com/ansible/ansible/issues/67365)
+- ansible-galaxy - properly show the role description when running offline (https://github.com/ansible/ansible/issues/60167)
+- ansible-galaxy cli - fixed ``--version`` argument
+- ansible-galaxy collection - Preserve executable bit on build and preserve mode on install from what tar member is set to - https://github.com/ansible/ansible/issues/68415
+- ansible-galaxy role - Fix issue where ``--server`` was not being used for certain ``ansible-galaxy role`` actions - https://github.com/ansible/ansible/issues/61609
+- ansible-galaxy- On giving an invalid subcommand to ansible-galaxy, the help would be shown only for role subcommand (collection subcommand help is not shown). With this change, the entire help for ansible-galaxy (same as ansible-galaxy --help) is displayed along with the help for role subcommand. (https://github.com/ansible/ansible/issues/69009)
+- ansible-inventory - Fix long standing bug not loading vars plugins for group vars relative to the playbook dir when the '--playbook-dir' and '--export' flags are used together.
+- ansible-inventory - Fix regression loading vars plugins. (https://github.com/ansible/ansible/issues/65064)
+- ansible-inventory - Properly hide arguments that should not be shown (https://github.com/ansible/ansible/issues/61604)
+- ansible-inventory - Restore functionality to allow ``--graph`` to be limited by a host pattern
+- ansible-test - Code cleanup.
+- ansible-test - Disabled the ``duplicate-code`` and ``cyclic-import`` checks for the ``pylint`` sanity test due to inconsistent results.
+- ansible-test - Do not warn on missing PowerShell or C# util that are in other collections
+- ansible-test - Fix PowerShell module util analysis to properly detect the names of a util when running in a collection
+- ansible-test - Fix regression introduced in https://github.com/ansible/ansible/pull/67063 which caused module_utils analysis to fail on Python 2.x.
+- ansible-test - Fix traceback in validate-modules test when argument_spec is None.
+- ansible-test - Make sure import sanity test virtual environments also remove ``pkg-resources`` if it is not removed by uninstalling ``setuptools``.
+- ansible-test - Remove out-of-date constraint on installing paramiko versions 2.5.0 or later in tests.
+- ansible-test - The ``import`` sanity test now correctly blocks access to python modules, not just packages, in the ``ansible`` package.
+- ansible-test - The ``import`` sanity test now correctly provides an empty ``ansible`` package.
+- ansible-test - The shebang sanity test now correctly identifies modules in subdirectories in collections.
+- ansible-test - Updated Python constraints for installing ``coverage`` to resolve issues on multiple Python versions when using the ``--coverage`` option.
+- ansible-test - Updated requirements to limit ``boto3`` and ``botocore`` versions on Python 2.6 to supported versions.
+- ansible-test - Use ``sys.exit`` instead of ``exit``.
+- ansible-test - Use ``virtualenv`` versions before 20 on provisioned macOS instances to remain compatible with an older pip install.
+- ansible-test - avoid use of deprecated junit_xml method
+- ansible-test - bump version of ACME test container. The new version includes updated dependencies.
+- ansible-test - during module validation, handle add_file_common_args only for top-level arguments.
+- ansible-test - during module validation, improve alias handling.
+- ansible-test - for local change detection, allow to specify branch to compare to with ``--base-branch`` for all types of tests (https://github.com/ansible/ansible/pull/69508).
+- ansible-test - improve ``deprecate()`` call checker.
+- ansible-test can now install argparse with ``--requirements`` or delegation when the pip version in use is older than version 7.1
+- ansible-test change detection - Run only sanity tests on ``docs/`` and ``changelogs/`` in collections, to avoid triggering full CI runs of integration and unit tests when files in these directories change.
+- ansible-test coverage - Fix the ``--all`` argument when generating coverage reports - https://github.com/ansible/ansible/issues/62096
+- ansible-test import sanity test now consistently reports errors against the file being tested.
+- ansible-test import sanity test now consistently reports warnings as errors.
+- ansible-test import sanity test now properly handles relative imports.
+- ansible-test import sanity test now properly invokes Ansible modules as scripts.
+- ansible-test is now able to find its ``egg-info`` directory when it contains the Ansible version number
+- ansible-test no longer errors reporting coverage when no Python coverage exists. This fixes issues reporting on PowerShell only coverage from collections.
+- ansible-test no longer fails when downloading test results for a collection without a ``tests`` directory when using the ``--docker`` option.
+- ansible-test no longer optimizes setting ``PATH`` by prepending the directory containing the selected Python interpreter when it is named ``python``. This avoids unintentionally making other programs available on ``PATH``, including an already installed version of Ansible.
+- ansible-test no longer tracebacks during change analysis due to processing an empty python file
+- ansible-test no longer tries to install ``coverage`` 5.0+ since those versions are unsupported
+- ansible-test no longer tries to install ``setuptools`` 45+ on Python 2.x since those versions are unsupported
+- ansible-test now correctly collects code coverage on the last task in a play. This should resolve issues with missing code coverage, empty coverage files and corrupted coverage files resulting from early worker termination.
+- ansible-test now correctly enumerates submodules when a collection resides below the repository root
+- ansible-test now correctly excludes the test results temporary directory when copying files from the remote test system to the local system
+- ansible-test now correctly includes inventory files ignored by git when running tests with the ``--docker`` option
+- ansible-test now correctly installs the requirements specified by the collection's unit and integration tests instead of the requirements specified for Ansible's own unit and integration tests
+- ansible-test now correctly recognizes imports in collections when using the ``--changed`` option.
+- ansible-test now correctly rewrites coverage paths for PowerShell files when testing collections
+- ansible-test now creates its integration test temporary directory within the collection so ansible-playbook can properly detect the default collection
+- ansible-test now enables color ``ls`` on a remote host only if the host supports the feature
+- ansible-test now ignores empty ``*.py`` files when analyzing module_utils imports for change detection
+- ansible-test now ignores version control within subdirectories of collections. Previously this condition was an error.
+- ansible-test now ignores warnings when comparing pip versions before and after integration tests run
+- ansible-test now installs sanity test requirements specific to each test instead of installing requirements for all sanity tests
+- ansible-test now installs the correct version of ``cryptography`` with ``--requirements`` or delegation when setuptools is older than version 18.5
+- ansible-test now limits Jinja2 installs to version 2.10 and earlier on Python 2.6
+- ansible-test now limits ``pathspec`` to versions prior to 0.6.0 on Python 2.6 to avoid installation errors
+- ansible-test now limits installation of ``hcloud`` to Python 2.7 and 3.5 - 3.8 since other versions are unsupported
+- ansible-test now limits the version of ``setuptools`` on Python 2.6 to versions older than 37
+- ansible-test now loads the collection loader plugin early enough for ansible_collections imports to work in unit test conftest.py modules
+- ansible-test now preserves existing SSH authorized keys when provisioning a remote host
+- ansible-test now properly activates the vcenter plugin for vcenter tests when docker is available
+- ansible-test now properly activates virtual environments created using the --venv option
+- ansible-test now properly creates a virtual environment using ``venv`` when running in a ``virtualenv`` created virtual environment
+- ansible-test now properly excludes the ``tests/output/`` directory from code coverage
+- ansible-test now properly handles creation of Python execv wrappers when the selected interpreter is a script
+- ansible-test now properly handles enumeration of git submodules. Enumeration is now done with ``git submodule status --recursive`` without specifying ``.`` for the path, since that could cause the command to fail. Instead, relative paths outside the current directory are filtered out of the results. Errors from ``git`` commands will now once again be reported as errors instead of warnings.
+- ansible-test now properly handles warnings for removed modules/plugins
+- ansible-test now properly ignores the ``tests/output//`` directory when not using git
+- ansible-test now properly installs requirements for multiple Python versions when running sanity tests
+- ansible-test now properly recognizes modules and module_utils in collections when using the ``blacklist`` plugin for the ``pylint`` sanity test
+- ansible-test now properly registers its own code in a virtual environment when running from an install
+- ansible-test now properly reports import errors for collections when running the import sanity test
+- ansible-test now properly searches for ``pythonX.Y`` instead of ``python`` when looking for the real python that created a ``virtualenv``
+- ansible-test now properly sets PYTHONPATH for tests when running from an Ansible installation
+- ansible-test now properly sets ``ANSIBLE_PLAYBOOK_DIR`` for integration tests so unqualified collection references work for adhoc ``ansible`` usage
+- ansible-test now properly uses a fresh copy of environment variables for each command invocation to avoid mixing vars between commands
+- ansible-test now shows sanity test doc links when installed (previously the links were only visible when running from source)
+- ansible-test now shows the correct source path instead of ``%s`` for collection role based test targets when the ``-v`` option is used
+- ansible-test now supports submodules using older ``git`` versions which require querying status from the top level directory of the repo.
+- ansible-test now updates SSH keys it generates with newer versions of ssh-keygen to function with Paramiko
+- ansible-test now upgrades ``pip`` with `--requirements`` or delegation as needed when the pip version in use is older than version 7.1
+- ansible-test now uses GNU tar format instead of the Python default when creating payloads for remote systems
+- ansible-test now uses ``pycodestyle`` frozen at version 2.6.0 for consistent test results.
+- ansible-test now uses modules from the ``ansible.windows`` collection for setup and teardown of ``windows-integration`` tests and code coverage
+- ansible-test once again properly collects code coverage for ``ansible-connection``
+- ansible-test validate-modules - Fix arg spec collector for PowerShell to find utils in both a collection and base.
+- ansible-test validate-modules sanity test code ``missing-module-utils-import-c#-requirements`` is now ``missing-module-utils-import-csharp-requirements`` (fixes ignore bug).
+- ansible-test validate-modules sanity test code ``multiple-c#-utils-per-requires`` is now ``multiple-csharp-utils-per-requires`` (fixes ignore bug).
+- ansible-test validate-modules sanity test now checks for AnsibleModule initialization instead of module_utils imports, which did not work in many cases.
+- ansible-test validate-modules sanity test now properly handles collections imports using the Ansible collection loader.
+- ansible-test validate-modules sanity test now properly handles relative imports.
+- ansible-test validate-modules sanity test now properly handles sys.exit in modules.
+- ansible-test validate-modules sanity test now properly invokes Ansible modules as scripts.
+- ansible-test windows coverage - Ensure coverage reports are UTF-8 encoded without a BOM
+- ansible-test windows coverage - Output temp files as UTF-8 with BOM to standardise against non coverage runs
+- ansible-vault - Fix ``encrypt_string`` output in a tty when using ``--sdtin-name`` option (https://github.com/ansible/ansible/issues/65121)
+- ansible-vault create - Fix exception on no arguments given
+- apt - Fixed the issue the cache being updated while auto-installing its dependencies even when ``update_cache`` is set to false.
+- assemble module - fix documentation - the remote_src property specified a default value of no but it's actually yes.
+- avoid fatal traceback when a bad FQCN for a callback is supplied in the whitelist (#69401).
+- become - Fix various plugins that still used play_context to get the become password instead of through the plugin - https://github.com/ansible/ansible/issues/62367
+- blockinfile - fix regression that results in incorrect block in file when the block to be inserted does not end in a line separator (https://github.com/ansible/ansible/pull/69734)
+- blockinfile - preserve line endings on update (https://github.com/ansible/ansible/issues/64966)
+- clean_facts - use correct variable to avoid unnecessary handling of ``AttributeError``
+- code - removes some Python compatibility code for dealing with socket timeouts in ``wait_for``
+- collection loader - ensure Jinja function cache is fully-populated before lookup
+- collection loader - fixed relative imports on Python 2.7, ensure pluginloader caches use full name to prevent names from being clobbered (https://github.com/ansible/ansible/pull/60317)
+- collection_loader - sort Windows modules below other plugin types so the correct builtin plugin inside a role is selected (https://github.com/ansible/ansible/issues/65298)
+- collections - Handle errors better for filters and tests in collections, where a non-existent collection is specified, or importing the plugin results in an exception (https://github.com/ansible/ansible/issues/66721)
+- combine filter - ``[dict1, [dict2]] | combine`` now raise an error; previously ``combine`` had an undocumented behaviour where it was flattening the list before combining it (https://github.com/ansible/ansible/pull/57894#discussion_r339517518).
+- config - encoding failures on config values should be non-fatal (https://github.com/ansible/ansible/issues/63310)
+- copy - Fix copy modes when using remote_src=yes and src is a directory with trailing slash.
+- copy - Fixed copy module not working in case that remote_src is enabled and dest ends in a / (https://github.com/ansible/ansible/pull/47238)
+- copy - recursive copy with ``remote_src=yes`` now recurses beyond first level. (Fixes https://github.com/ansible/ansible/issues/58284)
+- core - remove unneeded Python version checks.
+- core - replace a compatibility import of pycompat24.literal_eval with ast.literal_eval.
+- core filters - fix ``extract()`` filter when key does not exist in container (https://github.com/ansible/ansible/issues/64957)
+- cron and cronvar - use get_bin_path utility to locate the default crontab executable instead of the hardcoded /usr/bin/crontab. (https://github.com/ansible/ansible/pull/59765)
+- cron cronvar - only run ``get_bin_path()`` once
+- cronvar - use correct binary name (https://github.com/ansible/ansible/issues/63274)
+- deal with cases in which just a file is pased and not a path with directories, now fileglob correctly searches in 'files/' subdirs.
+- debug - fixed an issue introduced in Ansible 2.4 where a loop of debug tasks would lose the "changed" status on each item.
+- discovery will NOT update incorrect host anymore when in delegate_to task.
+- display - Improve method of removing extra new line after warnings so it does not break Tower/Runner (https://github.com/ansible/ansible/pull/68517)
+- display - remove extra new line after warnings (https://github.com/ansible/ansible/pull/65199)
+- display - remove leading space when displaying WARNING messages
+- display logging - Fix issue where 3rd party modules will print tracebacks when attempting to log information when ``ANSIBLE_LOG_PATH`` is set - https://github.com/ansible/ansible/issues/65249
+- display logging - Fixed up the logging formatter to use the proper prefixes for ``u=user`` and ``p=process``
+- display logging - Re-added the ``name`` attribute to the log formatter so that the source of the log can be seen
+- dnf - Fix idempotence of `state: installed` (https://github.com/ansible/ansible/issues/64963)
+- dnf - Unified error messages when trying to install a nonexistent package with newer dnf (4.2.18) vs older dnf (4.2.9)
+- dnf - Unified error messages when trying to remove a wildcard name that is not currently installed, with newer dnf (4.2.18) vs older dnf (4.2.9)
+- dnf - enable logging using setup_loggers() API in dnf-4.2.17-6 or later
+- dnf - remove custom ``fetch_rpm_from_url`` method in favor of more general ``ansible.module_utils.urls.fetch_file``.
+- dnf module - Ensure the modules exit_json['msg'] response is always string, not sometimes a tuple.
+- ensure we pass on interpreter discovery values to delegated host.
+- env lookup plugin - Fix handling of environment variables values containing utf-8 characters. (https://github.com/ansible/ansible/issues/65298)
+- fact gathering - Display warnings and deprecation messages that are created during the fact gathering phase
+- facts - fix detection of virtualization type when dmi product name is KVM Server
+- facts - introduce fact "ansible_processor_nproc" which reflects the number of vcpus available to processes (falls back to the number of vcpus available to the scheduler)
+- file - Removed unreachable code in module
+- file - change ``_diff_peek`` in argument spec to be the correct type, which is ``bool`` (https://github.com/ansible/ansible/issues/59433)
+- file - return ``'state': 'absent'`` when a file does not exist (https://github.com/ansible/ansible/issues/66171)
+- find - clarify description of ``contains`` (https://github.com/ansible/ansible/issues/61983)
+- fix issue in which symlinked collection cannot be listed, though the docs/plugins can be loaded if referenced directly.
+- fix wrong command line length calculation in ``ansible-console`` when long command inputted
+- for those running uids for invalid users (containers), fallback to uid=<uid> when logging fixes #68007
+- free strategy - Include failed hosts when filtering notified hosts for handlers. The strategy base should determine whether or not to run handlers on those hosts depending on whether forcing handlers is enabled (https://github.com/ansible/ansible/issues/65254).
+- galaxy - Fix an AttributeError on ansible-galaxy install with an empty requirements.yml (https://github.com/ansible/ansible/issues/66725).
+- get_url - Don't treat no checksum as a checksum match (https://github.com/ansible/ansible/issues/61978)
+- get_url pass incorrect If-Modified-Since header (https://github.com/ansible/ansible/issues/67417)
+- git - when force=True, apply --force flag to git fetches as well
+- group - The group module was not correctly detecting whether a local group is existing or not with local set to yes if the same group exists in a non local group repository e.g. LDAP. (https://github.com/ansible/ansible/issues/58619)
+- group_by now should correctly refect changed status.
+- hostname - Fixed an issue where the hostname on the cloudlinux 6 server could not be set.
+- hostname - make module work on Manjaro Linux (https://github.com/ansible/ansible/issues/61382)
+- hurd - Address FIXMEs. Extract functionality and exit early.
+- include_vars - fix stack trace when passing ``dirs`` in an ad-hoc command (https://github.com/ansible/ansible/issues/62633)
+- interpreter discovery will now use correct vars (from delegated host) when in delegate_to task.
+- junit callback - avoid use of deprecated junit_xml method
+- lineinfile - add example of using alternative backrefs syntax (https://github.com/ansible/ansible/issues/42794)
+- lineinfile - don't attempt mkdirs when path doesn't contain directory path
+- lineinfile - fix bug that caused multiple line insertions (https://github.com/ansible/ansible/issues/58923).
+- lineinfile - properly handle inserting a line when backrefs are enabled and the line already exists in the file (https://github.com/ansible/ansible/issues/63756)
+- lineinfile - use ``module.tmpdir`` to allow configuration of the remote temp directory (https://github.com/ansible/ansible/issues/68218)
+- lineinfile - use correct index value when inserting a line at the end of a file (https://github.com/ansible/ansible/issues/63684)
+- loops - Do not indiscriminately mark loop items as unsafe, only apply unsafe to ``with_`` style loops. The items from ``loop`` should not be explicitly wrapped in unsafe. The underlying templating mechanism should dictate this. (https://github.com/ansible/ansible/issues/64379)
+- make ``no_log=False`` on a module option silence the ``no_log`` warning (https://github.com/ansible/ansible/issues/49465 https://github.com/ansible/ansible/issues/64656)
+- match docs for ssh and ensure pipelining is configurable per connection plugin.
+- module executor - Address issue where changes to Ansiballz module code, change the behavior of module execution as it pertains to ``__file__`` and ``sys.modules`` (https://github.com/ansible/ansible/issues/64664)
+- module_defaults - support candidate action names for relocated content
+- module_defaults - support short group names for content relocated to collections
+- now correclty merge and not just overwrite facts when gathering using multiple modules.
+- objects - Remove FIXME comment because no fix is needed.
+- optimize 'smart' detection from being run over and over and preferably do it at config time.
+- package_facts - fix value of ``vital`` attribute which is returned when ``pkg`` manager is used
+- package_facts - use module warnings rather than a custom implementation for reporting warnings
+- packaging_yum - replace legacy file handling with a file manager.
+- paramiko - catch and handle exception to prevent stack trace when running in FIPS mode
+- paramiko_ssh - Removed redundant conditional statement in ``_parse_proxy_command`` that always evaluated to True.
+- paramiko_ssh - improve authentication error message so it is less confusing
+- paramiko_ssh - optimized file handling by using a context manager.
+- pip - The virtualenv_command option can now include arguments without requiring the full path to the binary. (https://github.com/ansible/ansible/issues/52275)
+- pip - check_mode with ``state: present`` now returns the correct state for pre-release versioned packages
+- plugins - Allow ensure_type to decrypt the value for string types (and implicit string types) when value is an inline vault.
+- psexec - Fix issue where the Kerberos package was not detected as being available.
+- psexec - Fix issue where the ``interactive`` option was not being passed down to the library.
+- reboot, win_reboot - add ``boot_time_command`` parameter to override the default command used to determine whether or not a system was rebooted (https://github.com/ansible/ansible/issues/58868)
+- remove update/restore of vars from play_context as it is now redundant.
+- replace use of deprecated functions from ``ansible.module_utils.basic``.
+- roles - Ensure that ``allow_duplicates: true`` enables to run single role multiple times (https://github.com/ansible/ansible/issues/64902)
+- runas - Fix the ``runas`` ``become_pass`` variable fallback from ``ansible_runas_runas`` to ``ansible_runas_pass``
+- service_facts - Now correctly parses systemd list-unit-files for systemd >=245
+- setup - properly detect yum package manager for IBM i.
+- setup - service_mgr - detect systemd even if it isn't running, such as during a container build
+- shell cmd - Properly escape double quotes in the command argument
+- synchronize - allow data to be passed between two managed nodes when using the docker connection plugin (https://github.com/ansible/ansible/pull/65698)
+- synchronize - fix password authentication on Python 2 (https://github.com/ansible/ansible/issues/56629)
+- sysctl - Remove FIXME comments to avoid confusion
+- systemd - don't require systemd to be running to enable/disable or mask/unmask units
+- systemd - the module should fail in check_mode when service not found on host (https://github.com/ansible/ansible/pull/68136).
+- sysvinit - Add missing parameter ``module`` in call to ``daemonize()``.
+- template lookup - ensure changes to the templar in the lookup, do not affect the templar context outside of the lookup (https://github.com/ansible/ansible/issues/60106)
+- template lookup - fix regression when templating hostvars (https://github.com/ansible/ansible/issues/63940)
+- the default parsing will now show existing JSON errors and not just YAML (last attempted), also we avoid YAML parsing when we know we only want JSON issue
+- throttle: the linear strategy didn't always stuck with the throttle limit
+- unarchive - Remove incorrect and unused function arguments.
+- unsafe_proxy - Ensure that data within a tuple is marked as unsafe (https://github.com/ansible/ansible/issues/65722)
+- update ``user`` module to support silencing ``no_log`` warnings in the future (see: https://github.com/ansible/ansible/pull/64733)
+- uri - Don't return the body even if it failed (https://github.com/ansible/ansible/issues/21003)
+- user - allow 13 asterisk characters in password field without warning
+- user - fix comprasion on macOS so module does not improperly report a change (https://github.com/ansible/ansible/issues/62969)
+- user - fix stack trace on AIX when attempting to parse shadow file that does not exist (https://github.com/ansible/ansible/issues/62510)
+- user - on systems using busybox, honor the ``on_changed`` parameter to prevent unnecessary password changing (https://github.com/ansible/ansible/issues/65711)
+- user - update docs to reflect proper way to remove account from all groups
+- validate-modules - Fix hang when inspecting module with a delegate args spec type
+- virtual facts - detect generic container environment based on non-empty "container" env var
+- wait_for_connection - with pipelining enabled, interpreter discovery would fail if the first connection attempt was not successful
+- win_exec_wrapper - Be more defensive when it comes to getting unhandled exceptions
+- win_package - Handle quoted and unquoted strings in the registry ``UninstallString`` value - https://github.com/ansible/ansible/issues/40973
+- win_uri win_get_url - Fix the behaviour of ``follow_redirects: safe`` to actual redirect on ``GET`` and ``HEAD`` requests - https://github.com/ansible/ansible/issues/65556
+- windows environment - Support env vars that contain the unicode variant of single quotes - https://github.com/ansible-collections/ansible.windows/issues/45
+- yum - fix bug that caused ``enablerepo`` to not be honored when used with disablerepo all wildcard/glob (https://github.com/ansible/ansible/issues/66549)
+- yum - fixed the handling of releasever parameter
+- yum - performance bugfix, the YumBase object was being  instantiated multiple times unnecessarily, which lead to considerable overhead when operating against large sets of packages.
+- yum - yum tasks can no longer end up running non-yum modules
+- yum/dnf - check type of elements in a name
+
+New Plugins
+-----------
+
+Lookup
+~~~~~~
+
+- unvault - read vaulted file(s) contents

changelogs/fragments/32386_debconf_password.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- debconf - add a note about no_log=True since module might expose sensitive information to logs (https://github.com/ansible/ansible/issues/32386).

changelogs/fragments/38760-slackware-os-dist.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - facts - account for Slackware OS with ``+`` in the name (https://github.com/ansible/ansible/issues/38760)

changelogs/fragments/67794-atomic_move-default-perms.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >
+    **security issue** atomic_move - change default permissions when creating
+    temporary files so they are not world readable (https://github.com/ansible/ansible/issues/67794) (CVE-2020-1736)

changelogs/fragments/68275-vault-module-args.yml

@@ -0,0 +1,3 @@
+bugfixes:
+- Vault - Allow single vault encrypted values to be used directly as module
+  parameters. (https://github.com/ansible/ansible/issues/68275)

changelogs/fragments/69578-shell-remote_tmp-quoting.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - shell - fix quoting of mkdir command in creation of remote_tmp in order to allow spaces and other special characters (https://github.com/ansible/ansible/issues/69577).

changelogs/fragments/70000-playbook-detect-failure-in-always.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - playbooks - detect and propagate failures in ``always`` blocks after ``rescue`` (https://github.com/ansible/ansible/issues/70000)

changelogs/fragments/70017-avoid-params-to-callable-checkers.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >-
+    if the ``type`` for a module parameter in the argument spec is callable,
+    do not pass ``kwargs`` to avoid errors (https://github.com/ansible/ansible/issues/70017)

changelogs/fragments/70045-ansible-doc-yaml-anchors.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - improve man page formatting to avoid problems when YAML anchors are used (https://github.com/ansible/ansible/pull/70045)."

changelogs/fragments/70046-ansible-doc-description-crash.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - improve error message in text formatter when ``description`` is missing for a (sub-)option or a return value or its ``contains`` (https://github.com/ansible/ansible/pull/70046)."

changelogs/fragments/70099-make-apt-errors-more-transparent.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - apt - include exception message from apt python library in error output

changelogs/fragments/70122-improve-error-message-ssh-client-is-not-found.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - SSH plugin - Improve error message when ssh client is not found on the host

changelogs/fragments/70168-fix-delegate_facts-without-interpreter-set.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - "Fix ``delegate_facts: true`` when ``ansible_python_interpreter`` is not set. (https://github.com/ansible/ansible/issues/70168)"

changelogs/fragments/70238_selector.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- basic - use PollSelector implementation when DefaultSelector fails (https://github.com/ansible/ansible/issues/70238).

changelogs/fragments/70240-fix-fatal-post_validate-error.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - TaskExecutor - Handle unexpected errors as failed while post validating loops (https://github.com/ansible/ansible/issues/70050).

changelogs/fragments/70261_pipe_lookup.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- pipe lookup - update docs for Popen with shell=True usages (https://github.com/ansible/ansible/issues/70159).

changelogs/fragments/70344-plugin-deprecation-collection-name.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - collection name for plugin top-level deprecation was not inserted when deprecating by version (https://github.com/ansible/ansible/pull/70344)."

changelogs/fragments/70426-allow-non-ascii-chars-in-cron.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - cron - encode and decode crontab files in UTF-8 explicitly to allow non-ascii chars in cron filepath and job (https://github.com/ansible/ansible/issues/69492)

changelogs/fragments/70437-ssh-args.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - >
+    ssh connection plugin - use ``get_option()`` rather than ``_play_context`` to
+    ensure ``ANSBILE_SSH_ARGS`` are applied properly (https://github.com/ansible/ansible/issues/70437)

changelogs/fragments/70465-assemble-fix-decrypt-argument.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - assemble - fix decrypt argument in the module (https://github.com/ansible/ansible/issues/65450).

changelogs/fragments/70484-bool-ansible-become.yaml

@@ -0,0 +1,3 @@
+bugfixes:
+  - The `ansible_become` value was not being treated as a boolean value when set in an INI format
+    inventory file (fixes bug https://github.com/ansible/ansible/issues/70476).

changelogs/fragments/70525-setuptools-disutils-reorder.yml

@@ -0,0 +1,7 @@
+bugfixes:
+- >
+  Address the deprecation of the use of stdlib
+  distutils in packaging. It's a short-term hotfix for the problem
+  (https://github.com/ansible/ansible/issues/70456,
+  https://github.com/pypa/setuptools/issues/2230,
+  https://github.com/pypa/setuptools/commit/bd110264)

changelogs/fragments/70600-user-module-dont-create-home-when-create_home-is-false.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "user - don't create home directory and missing parents when create_home == false (https://github.com/ansible/ansible/pull/70600)."

changelogs/fragments/70649_time_clock.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- api - time.clock is removed in Python 3.8, add backward compatible code (https://github.com/ansible/ansible/issues/70649).

changelogs/fragments/70657-template-connection-vars.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Template connection variables before using them (https://github.com/ansible/ansible/issues/70598).

changelogs/fragments/70683-terminal-ansi-re.yaml

@@ -0,0 +1,3 @@
+---
+bugfixes:
+  - Terminal plugins - add "\e[m" to the list of ANSI sequences stripped from device output

changelogs/fragments/70762-sanitize-uri-keys.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Sanitize no_log values from any response keys that might be returned from the uri module.

changelogs/fragments/70784-vault-is-string.yml

@@ -0,0 +1,3 @@
+bugfixes:
+- JSON Encoder - Ensure we treat single vault encrypted values as strings
+  (https://github.com/ansible/ansible/issues/70784)

changelogs/fragments/ansible-boilerplate.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - Add standard Python 2/3 compatibility boilerplate to setup script, module_utils and docs_fragments which were missing them.

changelogs/fragments/ansible-doc-collection-name.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "ansible-doc - include the collection name in the text output (https://github.com/ansible/ansible/pull/70401)."

changelogs/fragments/ansible-galaxy-stdout.yml

@@ -0,0 +1,3 @@
+minor_changes:
+  - ansible-galaxy - Change the output verbosity level of the download message from 3 to 0 (https://github.com/ansible/ansible/issues/70010)
+  - ansible-galaxy - Add installation successful message

changelogs/fragments/ansible-test-plugin-classification.yml

@@ -0,0 +1,2 @@
+bugfixes:
+    - ansible-test - integration and unit test change detection now works for filter, lookup and test plugins

changelogs/fragments/ansible-test-rhel-requirements.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Provisioning of RHEL instances now includes installation of pinned versions of ``packaging`` and ``pyparsing`` to match the downstream vendored versions.

changelogs/fragments/ansible-test-sanity-ansible-doc.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test - The ``ansible-doc`` sanity test now works for ``netconf`` plugins.

changelogs/fragments/ansible-test-sanity-yamllint-lineno.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Report the correct line number in the ``yamllint`` sanity test when reporting ``libyaml`` parse errors in module documentation.

changelogs/fragments/ansible-test-virtualenv-fix.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - ansible-test now always uses the ``--python`` option for ``virtualenv`` to select the correct interpreter when creating environments with the ``--venv`` option

changelogs/fragments/better_winrm_putfile_error.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - winrm - preserve winrm forensic data on put_file failures

changelogs/fragments/changelog-yaml.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- "The machine-readable changelog ``changelogs/changelog.yaml`` is now contained in the release."

changelogs/fragments/collection_meta_use_libyaml.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - collection metadata - ensure collection loader uses libyaml/CSafeLoader to parse collection metadata if available

changelogs/fragments/command-module-warnings.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - "Command module: Removed suggestions to use modules which have moved to collections and out of ansible-base"

changelogs/fragments/delegate_has_hostvars.yml

@@ -0,0 +1,4 @@
+bugfixes:
+  - ensure delegated vars can resolve hostvars object and access vars from hostvars[inventory_hostname].
+  - fix issue with inventory_hostname and delegated host vars mixing on connection settings.
+  - add magic/connection vars updates from delegated host info.

changelogs/fragments/display-stdout-column-width.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Ansible output now uses stdout to determine column width instead of stdin

changelogs/fragments/fq_action_module_resolution.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - action plugins - change all action/module delegations to use FQ names while allowing overrides (https://github.com/ansible/ansible/issues/69788)

changelogs/fragments/handle_undefined_in_type_errors_filters.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Allow TypeErrors on Undefined variables in filters to be handled or deferred when processing for loops.

changelogs/fragments/module_utils_finder_refactor.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - Python module_utils finder - refactor logic to eliminate many corner cases, remove recursion, fix base module_utils redirections

changelogs/fragments/pause-catch-error-when-no-std-exists.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - pause - handle exception when there is no stdout (https://github.com/ansible/ansible/pull/47851)

changelogs/fragments/plugin-loader-collection-name.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- "The plugin loader now keeps track of the collection where a plugin was resolved to, in particular whether the plugin was loaded from ansible-base's internal paths (``ansible.builtin``) or from user-supplied paths (no collection name)."

changelogs/fragments/runtime-splunk-redirect.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - splunk httpapi plugin - switch from splunk.enterprise_security to splunk.es in runtime.yml to reflect upstream change of Collection Name

changelogs/fragments/set_fact-connection_vars.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- Stop adding the connection variables to the output results

changelogs/fragments/test-ansible-runner-pin-psutil.yml

@@ -0,0 +1,2 @@
+bugfixes:
+  - add constraints file for ``anisble_runner`` test since an update to ``psutil`` is now causing test failures

changelogs/fragments/update-conditionals-bare-vars-default.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - conditionals - change the default of CONDITIONAL_BARE_VARS to False (https://github.com/ansible/ansible/issues/70682).

changelogs/fragments/v2.10.0b1_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-06-17
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0rc1_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-07-23
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/v2.10.0rc2_summary.yaml

@@ -0,0 +1,3 @@
+release_summary: |
+   | Release Date: 2020-07-23
+   | `Porting Guide <https://docs.ansible.com/ansible/devel/porting_guides.html>`__

changelogs/fragments/validate-modules-ps-doc-blacklist.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+- ansible-test - Do not try to validate PowerShell modules ``setup.ps1``, ``slurp.ps1``, and ``async_status.ps1``

changelogs/fragments/win_async_full_path.yml

@@ -0,0 +1,2 @@
+bugfixes:
+- windows async - use full path when calling PowerShell to reduce reliance on environment vars being correct - https://github.com/ansible/ansible/issues/70655

changelogs/fragments/win_setup-redirection.yaml

@@ -0,0 +1,2 @@
+bugfixes:
+- win setup - Fix redirection path for the windows setup module

changelogs/fragments/with_seq_example.yml

@@ -0,0 +1,2 @@
+minor_changes:
+- Add an example for using var in with_sequence (https://github.com/ansible/ansible/issues/68836).

docs/docsite/Makefile

@@ -1,6 +1,6 @@
 OS := $(shell uname -s)
 SITELIB = $(shell python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"):
-PLUGIN_FORMATTER=../../hacking/build-ansible.py document-plugins
+PLUGIN_FORMATTER=../../hacking/build-ansible.py docs-build
 TESTING_FORMATTER=../bin/testing_formatter.sh
 KEYWORD_DUMPER=../../hacking/build-ansible.py document-keywords
 CONFIG_DUMPER=../../hacking/build-ansible.py document-config
@@ -12,23 +12,35 @@ else
 CPUS ?= $(shell nproc)
 endif
 
-# Sets the build output directory if it's not already specified
+# Sets the build output directory for the main docsite if it's not already specified
 ifndef BUILDDIR
 	BUILDDIR = _build
 endif
 
-MODULE_ARGS=
-ifdef MODULES
-	MODULE_ARGS = -l $(MODULES)
-endif
-
+# Backwards compat for separate VARS
 PLUGIN_ARGS=
+ifdef MODULES
+ifndef PLUGINS
+	PLUGIN_ARGS = -l $(MODULES)
+else
+	PLUGIN_ARGS = -l $(MODULES),$(PLUGINS)
+endif
+else
 ifdef PLUGINS
 	PLUGIN_ARGS = -l $(PLUGINS)
 endif
+endif
+
 
 DOC_PLUGINS ?= become cache callback cliconf connection httpapi inventory lookup netconf shell strategy vars
 
+PYTHON=python
+# fetch version from project release.py as single source-of-truth
+VERSION := $(shell $(PYTHON) ../../packaging/release/versionhelper/version_helper.py --raw || echo error)
+ifeq ($(findstring error,$(VERSION)), error)
+$(error "version_helper failed")
+endif
+
 assertrst:
 ifndef rst
 	$(error specify document or pattern with rst=somefile.rst)
@@ -38,7 +50,8 @@ all: docs
 
 docs: htmldocs
 
-generate_rst: collections_meta config cli keywords modules plugins testing
+generate_rst: collections_meta config cli keywords plugins testing
+base_generate_rst: collections_meta config cli keywords base_plugins testing
 
 htmldocs: generate_rst
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx html
@@ -46,9 +59,12 @@ htmldocs: generate_rst
 singlehtmldocs: generate_rst
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx singlehtml
 
+base_singlehtmldocs: base_generate_rst
+	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx singlehtml
+
 linkcheckdocs: generate_rst
 		CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx linkcheck
-		
+
 webdocs: docs
 
 #TODO: leaving htmlout removal for those having older versions, should eventually be removed also
@@ -58,7 +74,7 @@ clean:
 	-rm -rf $(BUILDDIR)/html
 	-rm -rf htmlout
 	-rm -rf module_docs
-	-rm -rf _build
+	-rm -rf $(BUILDDIR)
 	-rm -f .buildinfo
 	-rm -f objects.inv
 	-rm -rf *.doctrees
@@ -70,43 +86,44 @@ clean:
 	find . -type f \( -name "*~" -or -name "#*" \) -delete
 	find . -type f \( -name "*.swp" \) -delete
 	@echo "Cleaning up generated rst"
-	rm -f rst/modules/*_by_category.rst
-	rm -f rst/modules/list_of_*.rst
-	rm -f rst/modules/*_maintained.rst
-	rm -f rst/modules/*_module.rst
-	rm -f rst/modules/*_plugin.rst
 	rm -f rst/playbooks_directives.rst
-	rm -f rst/plugins/*/*.rst
 	rm -f rst/reference_appendices/config.rst
 	rm -f rst/reference_appendices/playbooks_keywords.rst
 	rm -f rst/dev_guide/collections_galaxy_meta.rst
 	rm -f rst/cli/*.rst
+	rm -rf rst/collections/*
+	@echo "Cleaning up legacy generated rst locations"
+	rm -rf rst/modules
+	rm -f rst/plugins/*/*.rst
 
 .PHONY: docs clean
 
 collections_meta: ../templates/collections_galaxy_meta.rst.j2
-	PYTHONPATH=../../lib $(COLLECTION_DUMPER) --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
+	$(COLLECTION_DUMPER) --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/ ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
 
 # TODO: make generate_man output dir cli option
 cli:
 	mkdir -p rst/cli
-	PYTHONPATH=../../lib $(GENERATE_CLI) --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
+	$(GENERATE_CLI) --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
 
 keywords: ../templates/playbooks_keywords.rst.j2
-	PYTHONPATH=../../lib $(KEYWORD_DUMPER) --template-dir=../templates --output-dir=rst/reference_appendices/ ./keyword_desc.yml
+	$(KEYWORD_DUMPER) --template-dir=../templates --output-dir=rst/reference_appendices/ ./keyword_desc.yml
 
 config: ../templates/config.rst.j2
-	PYTHONPATH=../../lib $(CONFIG_DUMPER) --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
+	$(CONFIG_DUMPER) --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ ../../lib/ansible/config/base.yml
 
-modules: ../templates/plugin.rst.j2
-	PYTHONPATH=../../lib $(PLUGIN_FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/modules/ $(MODULE_ARGS)
+# For now, if we're building on devel, just build base docs.  In the future we'll want to build docs that
+# are the latest versions on galaxy (using a different antsibull-docs subcommand)
+plugins:
+	if expr match "$(VERSION)" '.*[.]dev[0-9]\+$$' &> /dev/null; then \
+		$(PLUGIN_FORMATTER) base -o rst $(PLUGIN_ARGS);\
+	else \
+		$(PLUGIN_FORMATTER) full -o rst $(PLUGIN_ARGS);\
+	fi
 
-plugins: ../templates/plugin.rst.j2
-	@echo "looping over doc plugins"
-	for plugin in $(DOC_PLUGINS); \
-	do \
-		PYTHONPATH=../../lib $(PLUGIN_FORMATTER) -t rst --plugin-type $$plugin --template-dir=../templates --module-dir=../../lib/ansible/plugins/$$plugin -o rst $(PLUGIN_ARGS); \
-	done
+# This only builds the plugin docs included with ansible-base
+base_plugins:
+	$(PLUGIN_FORMATTER) base -o rst $(PLUGIN_ARGS);\
 
 testing:
 	$(TESTING_FORMATTER)

docs/docsite/_themes/sphinx_rtd_theme/ansible_banner.html

@@ -10,26 +10,27 @@
       element.appendChild(para);
       document.write('</div>');
     }
-
-    // Create a banner if we're not the latest version
-    current_url = window.location.href;
-    if ((current_url.search("latest") > -1) || (current_url.search("/{{ latest_version }}/") > -1)) {
-     // no banner for latest release
-    } else if (current_url.search("devel") > -1) {
-      document.write('<div id="banner_id" class="admonition caution">');
-      para = document.createElement('p');
-      banner_text=document.createTextNode("You are reading the *devel* version of the Ansible documentation - most module documentation is currently missing as the modules have moved to collections. Until docs catches up to this change, use the version selection to the left if you want module documentation or the latest stable release version. The *devel* version is not guaranteed stable.");
-      para.appendChild(banner_text);
-      element = document.getElementById('banner_id');
-      element.appendChild(para);
-      document.write('</div>');
-    } else {
-      document.write('<div id="banner_id" class="admonition caution">');
-      para = document.createElement('p');
-      banner_text=document.createTextNode("You are reading an older version of the Ansible documentation. Use the version selection to the left if you want the latest stable released version.");
-      para.appendChild(banner_text);
-      element = document.getElementById('banner_id');
-      element.appendChild(para);
-      document.write('</div>');
-    }
+    {% if (not READTHEDOCS) and (available_versions is defined) %}
+      // Create a banner if we're not the latest version
+      current_url = window.location.href;
+      if ((current_url.search("latest") > -1) || (current_url.search("/{{ latest_version }}/") > -1)) {
+       // no banner for latest release
+      } else if (current_url.search("devel") > -1) {
+        document.write('<div id="banner_id" class="admonition caution">');
+        para = document.createElement('p');
+        banner_text=document.createTextNode("You are reading the *devel* version of the Ansible documentation - this version is not guaranteed stable. Use the version selection to the left if you want the latest stable released version.");
+        para.appendChild(banner_text);
+        element = document.getElementById('banner_id');
+        element.appendChild(para);
+        document.write('</div>');
+      } else {
+        document.write('<div id="banner_id" class="admonition caution">');
+        para = document.createElement('p');
+        banner_text=document.createTextNode("You are reading an older version of the Ansible documentation. Use the version selection to the left if you want the latest stable released version.");
+        para.appendChild(banner_text);
+        element = document.getElementById('banner_id');
+        element.appendChild(para);
+        document.write('</div>');
+      }
+    {% endif %}
   </script>

docs/docsite/_themes/sphinx_rtd_theme/ansible_versions.html

@@ -1,7 +1,7 @@
 <!--- Based on https://github.com/rtfd/sphinx_rtd_theme/pull/438/files -->
 {# Creates dropdown version selection in the top-left navigation. #}
 <div class="version">
-  {% if not READTHEDOCS %}
+  {% if (not READTHEDOCS) and (available_versions is defined) %}
     <div class="version-dropdown">
       <select class="version-list" id="version-list" onchange="javascript:location.href = this.value;">
         <script> x = document.getElementById("version-list"); </script>

docs/docsite/collection-plugins.yml

@@ -0,0 +1,17 @@
+# We also need an example of modules hosted in Automation Hub
+# We'll likely move to data hosted in botmeta instead of a standalone file but
+# we'll need all of these same details.
+module:
+    purefa_user:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'purestorage.flasharray'
+    purefa_vg:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'purestorage.flasharray'
+    gcp_compute_firewall_info:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'google.cloud'
+module_utils:
+    purefa:
+        source: 'https://galaxy.ansible.com/'
+        fqcn: 'purestorage.flasharray'

docs/docsite/keyword_desc.yml

@@ -6,23 +6,25 @@ args: "A secondary way to add arguments into a task. Takes a dictionary in which
 always: List of tasks, in a block, that execute no matter if there is an error in the block or not.
 any_errors_fatal: Force any un-handled task errors on any host to propagate to all hosts and end the play.
 async: Run a task asynchronously if the C(action) supports this; value is maximum runtime in seconds.
-become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution.
+become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution. Implemented by the become plugin. See :ref:`become_plugins`.
+become_exe: Path to the executable used to elevate privileges. Implemented by the become plugin. See :ref:`become_plugins`.
 become_flags: A string of flag(s) to pass to the privilege escalation program when :term:`become` is True.
 become_method: Which method of privilege escalation to use (such as sudo or su).
 become_user: "User that you 'become' after using privilege escalation. The remote/login user must have permissions to become this user."
 block: List of tasks in a block.
 changed_when: "Conditional expression that overrides the task's normal 'changed' status."
-check_mode: |
-    A boolean that controls if a task is executed in 'check' mode
+check_mode: A boolean that controls if a task is executed in 'check' mode. See :ref:`check_mode_dry`.
+collections: |
 
-    .. seealso:: :ref:`check_mode_dry`
+    List of collection namespaces to search for modules, plugins, and roles. See :ref:`collections_using_playbook`
 
-connection: |
-    Allows you to change the connection plugin used for tasks to execute on the target.
+    .. note::
 
-    .. seealso:: :ref:`using_connection`
+        Tasks within a role do not inherit the value of ``collections`` from the play. To have a role search a list of collections, use the ``collections`` keyword in ``meta/main.yml`` within a role.
 
-debugger: Enable debugging tasks based on state of the task result. See :ref:`playbook_debugger`
+
+connection: Allows you to change the connection plugin used for tasks to execute on the target. See :ref:`using_connection`.
+debugger: Enable debugging tasks based on state of the task result. See :ref:`playbook_debugger`.
 delay: Number of seconds to delay between retries. This setting is only used in combination with :term:`until`.
 delegate_facts: Boolean that allows you to apply facts to a delegated host instead of inventory_hostname.
 delegate_to: Host to execute task instead of the target (inventory_hostname). Connection vars from the delegated host will also be used for the task.

docs/docsite/requirements.txt

@@ -6,4 +6,4 @@ sphinx==2.1.2
 sphinx-notfound-page
 Pygments >= 2.4.0
 straight.plugin # Needed for hacking/build-ansible.py which is the backend build script
-make
+antsibull >= 0.15.0

docs/docsite/rst/community/communication.rst

@@ -24,7 +24,7 @@ Ansible has several mailing lists.  Your first post to the mailing list will be
 * `Ansible Lockdown List <https://groups.google.com/forum/#!forum/ansible-lockdown>`_ is for all things related to Ansible Lockdown projects, including DISA STIG automation and CIS Benchmarks.
 * `Ansible Outreach List <https://groups.google.com/forum/#!forum/ansible-outreach>`_ help with promoting Ansible and `Ansible Meetups <https://ansible.meetup.com/>`_
 * `Ansible Project List <https://groups.google.com/forum/#!forum/ansible-project>`_ is for sharing Ansible tips, answering questions, and general user discussion.
-* `Molecule List <https://groups.google.com/forum/#!forum/molecule-users>`_ is designed to aid with the development and testing of Ansible roles with Molecule.
+* `Molecule Discussions <https://github.com/ansible-community/molecule/discussions>`_ is designed to aid with the development and testing of Ansible roles with Molecule.
 
 To subscribe to a group from a non-Google account, you can send an email to the subscription address requesting the subscription. For example: ``ansible-devel+subscribe@googlegroups.com``
 

docs/docsite/rst/community/contributing_maintained_collections.rst

@@ -67,15 +67,15 @@ The following table shows:
       <td><a href="https://galaxy.ansible.com/amazon/aws">amazon.aws</a></td>
       <td><a href="https://galaxy.ansible.com/community/aws">community.aws</a></td>
       <td><a href="https://github.com/ansible/community/tree/master/group-aws">Cloud</a></td>
-      <td>✓</td>
-      <td>✓</td>
+      <td>✓**</td>
+      <td>**</td>
       <td>✓</td>
       <td>Shippable</td>
       <td>✓</td>
       <td><a href="https://docs.ansible.com/ansible/devel/dev_guide/platforms/aws_guidelines.html">AWS guide</a></td>
     </tr>
     <tr>
-      <td><a href="https://galaxy.ansible.com/ansible/netcommon">ansible.netcommon**</a></td>
+      <td><a href="https://galaxy.ansible.com/ansible/netcommon">ansible.netcommon***</a></td>
       <td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
       <td>✓</td>
@@ -101,7 +101,7 @@ The following table shows:
       <td><a href="https://galaxy.ansible.com/community/windows">community.windows</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Windows">Windows</a></td>
       <td>✓</td>
-      <td>✓***</td>
+      <td>✓****</td>
       <td>✓</td>
       <td>Shippable</td>
       <td>✓</td>
@@ -120,7 +120,7 @@ The following table shows:
     </tr>
     <tr>
       <td><a href="https://galaxy.ansible.com/cisco/asa">cisco.asa</a></td>
-      <td>community.asa</td>
+      <td><a href="https://github.com/ansible-collections/community.asa">community.asa</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
       <td>✓</td>
       <td>✓</td>
@@ -164,7 +164,7 @@ The following table shows:
     </tr>
     <tr>
       <td><a href="https://galaxy.ansible.com/ibm/qradar">ibm.qradar</a></td>
-      <td>community.qradar</td>
+      <td><a href="https://github.com/ansible-collections/community.qradar">community.qradar</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
       <td>✓</td>
       <td></td>
@@ -197,7 +197,7 @@ The following table shows:
     </tr>
     <tr>
       <td><a href="https://github.com/ansible-collections/splunk.es">splunk.es</a></td>
-      <td>community.es</td>
+      <td><a href="https://github.com/ansible-collections/community.es">community.es</a></td>
       <td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
       <td>✓</td>
       <td></td>
@@ -224,9 +224,11 @@ The following table shows:
 
   \* A ✓  under **Open to PRs** means the collection welcomes GitHub issues and PRs for any changes to existing collection content (plugins, roles, and so on).
 
-  \*\* ``ansible.netcommon`` contains all foundational components for enabling many network and security :ref:`platform <platform_options>` collections. It contains all connection and filter plugins required, and installs as a dependency when you install the the platform collection.
+  \*\* Integration tests are required and unit tests are welcomed but not required for the AWS collections.  An exception to this is made in cases where integration tests are logistically not feasible due to external requirements.  An example of this is AWS Direct Connect, as this service can not be functionally tested without the establishment of network peering connections.  Unit tests are therefore required for modules that interact with AWS Direct Connect.  Exceptions to ``amazon.aws`` must be approved by Red Hat, and exceptions to ``community.aws`` must be approved by the AWS community.
 
-  \*\*\* Unit tests for Windows PowerShell modules are an exception to testing, but unit tests are valid and required for the remainder of the collection, including Ansible-side plugins.
+  \*\*\* ``ansible.netcommon`` contains all foundational components for enabling many network and security :ref:`platform <platform_options>` collections. It contains all connection and filter plugins required, and installs as a dependency when you install the the platform collection.
+
+  \*\*\*\* Unit tests for Windows PowerShell modules are an exception to testing, but unit tests are valid and required for the remainder of the collection, including Ansible-side plugins.
 
 
 .. _which_collection:
@@ -248,7 +250,7 @@ The following scenarios use the ``arista.eos`` to help explain when to contribut
     * Propose a new module in the ``community.network`` collection (requires network community approval).
     * Place your new module in a collection in your own namespace (no approvals required).
 
-  
+
 Most new content should go into either a related community collection or your own collection first so that is well established in the community before you can propose adding it to the  ``arista`` namespace, where inclusion and maintenance criteria are much higher.
 
 

docs/docsite/rst/community/development_process.rst

@@ -27,7 +27,7 @@ If you want to follow the conversation about what features will be added to Ansi
 Micro development: the lifecycle of a PR
 ========================================
 
-Ansible accepts code through **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works <https://help.github.com/articles/about-pull-requests/>`_ in general. The ultimate goal of any pull request is to get merged and become part of Ansible Core.
+Ansible accepts code through **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works <https://help.github.com/articles/about-pull-requests/>`_ in general. The ultimate goal of any pull request is to get merged and become part of a collection or ansible-base.
 Here's an overview of the PR lifecycle:
 
 * Contributor opens a PR
@@ -209,7 +209,7 @@ Here are some examples:
 .. code-block:: yaml
 
   bugfixes:
-    - copy module - The copy module was attempting to change the mode of files for
+    - copy - the module was attempting to change the mode of files for
       remote_src=True even if mode was not set as a parameter.  This failed on
       filesystems which do not have permission bits (https://github.com/ansible/ansible/issues/29444).
 

docs/docsite/rst/community/how_can_I_help.rst

@@ -4,7 +4,8 @@
 How can I help?
 ***************
 
-.. contents:: Topics
+.. contents::
+   :local:
 
 Thanks for being interested in helping the Ansible project!
 
@@ -50,19 +51,19 @@ File and verify issues
 All software has bugs, and Ansible is no exception. When you find a bug, you can help tremendously by :ref:`telling us about it <reporting_bugs_and_features>`.
 
 
-If you should discover that the bug you're trying to file already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information.
+If the bug you found already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information.
 
 Review and submit pull requests
 ===============================
 
-As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you've got a solution to a bug you've found in Ansible, or if you've got a new feature that you've written and would like to share with millions of Ansible users, read all about the :ref:`Ansible development process <community_development_process>` to learn how to get your code accepted into Ansible.
+As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you have a fix for a bug in Ansible, or if you have a new feature that you would like to share with millions of Ansible users, read all about the :ref:`Ansible development process <community_development_process>` and and :ref:`how to contribute to collections <contributing_maintained_collections>` to learn how to get your code accepted into Ansible.
 
 Another good way to help is to review pull requests that other Ansible users have submitted. The Ansible community keeps a full list of `open pull requests by file <https://ansible.sivel.net/pr/byfile.html>`_, so if there's a particular module or plug-in that particularly interests you, you can easily keep track of all the relevant new pull requests and provide testing or feedback.
 
-Become a module maintainer
-==========================
+Become a collection maintainer
+==============================
 
-Once you've learned about the development process and have contributed code to a particular module, we encourage you to become a maintainer of that module. There are hundreds of different modules in Ansible, and the vast majority of them are written and maintained entirely by members of the Ansible community.
+Once you have learned about the development process and have contributed code to a collection, we encourage you to become a maintainer of that collection. There are hundreds of modules in dozens of Ansible collections, and the vast majority of them are written and maintained entirely by members of the Ansible community.
 
 To learn more about the responsibilities of being an Ansible module maintainer, please read our :ref:`module maintainer guidelines <maintainers>`.
 
@@ -77,7 +78,7 @@ Working groups are a way for Ansible community members to self-organize around p
 Teach Ansible to others
 =======================
 
-We're working on a standardized Ansible workshop called `Lightbulb <https://github.com/ansible/lightbulb>`_ that can provide a good hands-on introduction to Ansible usage and concepts.
+We are working on a standardized `Ansible workshop <https://ansible.github.io/workshops/>`_ that can provide a good hands-on introduction to Ansible usage and concepts.
 
 Social media
 ============

docs/docsite/rst/community/index.rst

@@ -14,9 +14,9 @@ This page outlines the most common situations and questions that bring readers t
 Getting started
 ===============
 
-* I'm new to the community. Where can I find the Ansible :ref:`code_of_conduct`?
-* I'd like to know what I'm agreeing to when I contribute to Ansible. Does Ansible have a :ref:`contributor_license_agreement`?
-* I'd like to contribute but I'm not sure how. Are there :ref:`easy ways to contribute <how_can_i_help>`?
+* I am new to the community. Where can I find the Ansible :ref:`code_of_conduct`?
+* I would like to know what I am agreeing to when I contribute to Ansible. Does Ansible have a :ref:`contributor_license_agreement`?
+* I would like to contribute but I am not sure how. Are there :ref:`easy ways to contribute <how_can_i_help>`?
 * I want to talk to other Ansible users. How do I find an `Ansible Meetup near me <https://www.meetup.com/topics/ansible/>`_?
 * I have a question. Which :ref:`Ansible email lists and IRC channels <communication>` will help me find answers?
 * I want to learn more about Ansible. What can I do?
@@ -27,7 +27,7 @@ Getting started
   * `Review getting started guides <https://www.ansible.com/resources/get-started>`_.
   * `Watch videos <https://www.ansible.com/resources/videos>`_ - includes Ansible Automates, AnsibleFest & webinar recordings.
 
-* I'd like updates about new Ansible versions. How are `new releases announced <https://groups.google.com/forum/#!forum/ansible-announce>`_?
+* I would like updates about new Ansible versions. How are `new releases announced <https://groups.google.com/forum/#!forum/ansible-announce>`_?
 * I want to use the current release. How do I know which :ref:`releases are current <release_schedule>`?
 
 Going deeper
@@ -36,9 +36,9 @@ Going deeper
 * I think Ansible is broken. How do I :ref:`report a bug <reporting_bugs>`?
 * I need functionality that Ansible doesn't offer. How do I :ref:`request a feature <request_features>`?
 * How do I :ref:`contribute to an Ansible-maintained collection <contributing_maintained_collections>`?
-* I'm waiting for a particular feature. How do I see what's :ref:`planned for future Ansible Releases <roadmaps>`?
+* I am waiting for a particular feature. How do I see what is :ref:`planned for future Ansible Releases <roadmaps>`?
 * I have a specific Ansible interest or expertise (for example, VMware, Linode, and so on.). How do I get involved in a :ref:`working group <working_group_list>`?
-* I'd like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests?
+* I would like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests?
 * I found a typo or another problem on docs.ansible.com. How can I :ref:`improve the documentation <community_documentation_contributions>`?
 
 
@@ -46,15 +46,15 @@ Working with the Ansible repo
 =============================
 
 * I want to code my first changes to Ansible. How do I :ref:`set up my Python development environment <environment_setup>`?
-* I'd like to get more efficient as a developer. How can I find :ref:`editors, linters, and other tools <other_tools_and_programs>` that will support my Ansible development efforts?
+* I would like to get more efficient as a developer. How can I find :ref:`editors, linters, and other tools <other_tools_and_programs>` that will support my Ansible development efforts?
 * I want my PR to meet Ansible's guidelines. Where can I find  guidance on :ref:`coding in Ansible <developer_guide>`?
 * I want to learn more about Ansible roadmaps, releases, and projects. How do I find information on :ref:`the development cycle <community_development_process>`?
-* I'd like to connect Ansible to a new API or other resource. How do I :ref:`contribute a group of related modules <developing_modules_in_groups>`?
+* I would like to connect Ansible to a new API or other resource. How do I :ref:`create a collection <developing_modules_in_groups>`?
 * My pull request is marked ``needs_rebase``. How do I :ref:`rebase my PR <rebase_guide>`?
-* I'm using an older version of Ansible and want a bug fixed in my version that's already been fixed on the ``devel`` branch. How do I :ref:`backport a bugfix PR <backport_process>`?
+* I am using an older version of Ansible and want a bug fixed in my version that has already been fixed on the ``devel`` branch. How do I :ref:`backport a bugfix PR <backport_process>`?
 * I have an open pull request with a failing test. How do I learn about Ansible's :ref:`testing (CI) process <developing_testing>`?
-* I'm ready to step up as a module maintainer. What are the :ref:`guidelines for maintainers <maintainers>`?
-* A module I maintain is obsolete. How do I :ref:`deprecate a module <deprecating_modules>`?
+* I am ready to step up as a collection maintainer. What are the :ref:`guidelines for maintainers <maintainers>`?
+* A module in a collection I maintain is obsolete. How do I :ref:`deprecate a module <deprecating_modules>`?
 
 Traditional Table of Contents
 =============================

docs/docsite/rst/community/maintainers.rst

@@ -1,12 +1,13 @@
 .. _maintainers:
 
-****************************
-Module Maintainer Guidelines
-****************************
+********************************
+Collection maintainer guidelines
+********************************
 
-.. contents:: Topics
+Thank you for being a community collection maintainer. This guide offers an overview of your responsibilities as a maintainer along with resources for additional information. The Ansible community hopes that you will find that maintaining a collection is as rewarding for you as having the collection content is for the wider community.
 
-Thank you for being a maintainer of part of Ansible's codebase. This guide provides module maintainers an overview of their responsibilities, resources for additional information, and links to helpful tools.
+.. contents::
+   :local:
 
 In addition to the information below, module maintainers should be familiar with:
 
@@ -17,57 +18,17 @@ In addition to the information below, module maintainers should be familiar with
 Maintainer responsibilities
 ===========================
 
-When you contribute a new module to the `ansible/ansible <https://github.com/ansible/ansible>`_ repository, you become the maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
+When you contribute a module to a collection included in the ``ansible`` package, you become a maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
 
-Maintainers of Ansible modules are expected to provide feedback, responses, or actions on pull requests or issues to the module(s) they maintain in a reasonably timely manner.
+Maintainers of Ansible collections are expected to provide feedback, responses, or actions on pull requests or issues to the collection(s) they maintain in a reasonably timely manner. You can also update the contributor guidelines for that collection, in collaboration with the Ansible community team and the other maintainers of that collection.
 
-It is also recommended that you occasionally revisit the `contribution guidelines <https://github.com/ansible/ansible/blob/devel/.github/CONTRIBUTING.md>`_, as they are continually refined. Occasionally, you may be requested to update your module to move it closer to the general accepted standard requirements. We hope for this to be infrequent, and will always be a request with a fair amount of lead time (ie: not by tomorrow!).
+Resources
+=========
 
-Finally, following the `ansible-devel <https://groups.google.com/forum/#!forum/ansible-devel>`_ mailing list can be a great way to participate in the broader Ansible community, and a place where you can influence the overall direction, quality, and goals of Ansible and its modules. If you're not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
+Please see :ref:`communication` for ways to contact the broader Ansible community. For maintainers, following the `ansible-devel <https://groups.google.com/forum/#!forum/ansible-devel>`_ mailing list is a great way to participate in conversations about coding, get assistance when you need it, and influence the overall direction, quality, and goals of Ansible and the collections. If you are not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
 
-The Ansible community hopes that you will find that maintaining your module is as rewarding for you as having the module is for the wider community.
 
 Pull requests, issues, and workflow
 ===================================
 
-Pull requests
--------------
-
-Module pull requests are located in the `main Ansible repository <https://github.com/ansible/ansible/pulls>`_.
-
-Because of the high volume of pull requests, notification of PRs to specific modules are routed by an automated bot to the appropriate maintainer for handling. It is recommended that you set an appropriate notification process to receive notifications which mention your GitHub ID.
-
-Issues
-------
-
-Issues for modules, including bug reports, documentation bug reports, and feature requests, are tracked in the `ansible repository <https://github.com/ansible/ansible/issues>`_.
-
-Issues for modules are routed to their maintainers by an automated process. This process is still being refined, and currently depends upon the issue creator to provide adequate details (specifically, providing the proper module name) in order to route it correctly. If you are a maintainer of a specific module, it is recommended that you periodically search module issues for issues which mention your module's name (or some variation on that name), as well as setting an appropriate notification process for receiving notification of mentions of your GitHub ID.
-
-PR workflow
------------
-
-Automated routing of pull requests is handled by a tool called `Ansibot <https://github.com/ansible/ansibullbot>`_.
-
-Being moderately familiar with how the workflow behind the bot operates can be helpful to you, and -- should things go awry -- your feedback can be helpful to the folks that continually help Ansibullbot to evolve.
-
-A detailed explanation of the PR workflow can be seen in the :ref:`community_development_process`.
-
-Maintainers (BOTMETA.yml)
--------------------------
-
-The full list of maintainers is located in `BOTMETA.yml <https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml>`_.
-
-Adding and removing maintainers
-===============================
-
-Communities change over time, and no one maintains a module forever. If you'd like to propose an additional maintainer for your module, please submit a PR to ``BOTMETA.yml`` with the GitHub username of the new maintainer.
-
-If you'd like to step down as a maintainer, please submit a PR to the ``BOTMETA.yml`` removing your GitHub ID from the module in question. If that would leave the module with no maintainers, put "ansible" as the maintainer.  This will indicate that the module is temporarily without a maintainer, and the Ansible community team will search for a new maintainer.
-
-Tools and other Resources
-=========================
-
-* `PRs in flight, organized by directory <https://ansible.sivel.net/pr/byfile.html>`_
-* `Ansibullbot <https://github.com/ansible/ansibullbot>`_
-* :ref:`community_development_process`
+Each collection community can set its own rules and workflow for managing pull requests, bug reports, documentation issues, and feature requests, as well as adding and replacing maintainers.

docs/docsite/rst/community/reporting_bugs_and_features.rst

@@ -1,39 +1,56 @@
 .. _reporting_bugs_and_features:
 
 **************************************
-Reporting Bugs And Requesting Features
+Reporting bugs and requesting features
 **************************************
 
-.. contents:: Topics
+.. contents::
+   :local:
 
 .. _reporting_bugs:
 
 Reporting a bug
 ===============
 
+Security bugs
+-------------
+
 Ansible practices responsible disclosure - if this is a security-related bug, email `security@ansible.com <mailto:security@ansible.com>`_ instead of filing a ticket or posting to any public groups, and you will receive a prompt response.
 
-Ansible bugs should be reported to `github.com/ansible/ansible/issues <https://github.com/ansible/ansible/issues>`_ after
-signing up for a free GitHub account.  Before reporting a bug, please use the bug/issue search
-to see if the issue has already been reported.
+Bugs in ansible-base
+--------------------
 
-Knowing your Ansible version and the exact commands you are running, and what you expect, saves time and helps us help everyone with their issues more quickly. For that reason, we provide an issue template; please fill it out as completely and as accurately as possible.
+If you find a bug that affects multiple plugins, a plugin that remained in the ansible/ansible repo, or the overall functioning of Ansible, report it to `github.com/ansible/ansible/issues <https://github.com/ansible/ansible/issues>`_. You need a free GitHub account.  Before reporting a bug, use the bug/issue search to see if the issue has already been reported. If you are not sure if something is a bug yet, you can report the behavior on the :ref:`mailing list or IRC first <communication>`.
 
-Do not use the issue tracker for "how do I do this" type questions.  These are great candidates for IRC or the mailing list instead where things are likely to be more of a discussion.
+Do not open issues for "how do I do this" type questions.  These are great topics for IRC or the mailing list, where things are likely to be more of a discussion.
 
-To be respectful of reviewers' time and allow us to help everyone efficiently, please  provide minimal well-reduced and well-commented examples rather than sharing your entire production playbook.  Include playbook snippets and output where possible.
+If you find a bug, open the issue yourself to ensure we have a record of it. Do not rely on someone else in the community to file the bug report for you. We have created an issue template, which saves time and helps us help everyone with their issues more quickly. Please fill it out as completely and as accurately as possible:
 
-When sharing YAML in playbooks, formatting can be preserved by using `code blocks  <https://help.github.com/articles/creating-and-highlighting-code-blocks/>`_.
+  * Include the Ansible version
+  * Include any relevant configuration
+  * Include the exact commands or tasks you are running
+  * Describe the behavior you expected
+  * Provide steps to reproduce the bug
+    * Use minimal well-reduced and well-commented examples, not your entire production playbook
+    * When sharing YAML in playbooks, preserve the formatting by using `code blocks  <https://help.github.com/articles/creating-and-highlighting-code-blocks/>`_.
+  * Document the behavior you got
+  * Include output where possible
+  * For multiple-file content, use gist.github.com, which is more durable than pastebin content
 
-For multiple-file content, we encourage use of gist.github.com.  Online pastebin content can expire, so it's nice to have things around for a longer term if they are referenced in a ticket.
+Bugs in collections
+-------------------
 
-If you are not sure if something is a bug yet, you are welcome to ask about something on the :ref:`mailing list or IRC first <communication>`.
+Many bugs only affect a single module or plugin. If you find a bug that affects a module or plugin hosted in a collection, file the bug in the repository of the :ref:`collection <collections>`:
 
-As we are a very high volume project, if you determine that you do have a bug, please be sure to open the issue yourself to ensure we have a record of it. Don't rely on someone else in the community to file the bug report for you.
+  #. Find the collection on Galaxy.
+  #. Click on the Issue Tracker link for that collection.
+  #. Follow the contributor guidelines or instructions in the collection repo.
+
+If you are not sure whether a bug is in ansible-base or in a collection, you can report the behavior on the :ref:`mailing list or IRC first <communication>`.
 
 .. _request_features:
 
 Requesting a feature
 ====================
 
-The best way to get a feature into Ansible is to :ref:`submit a pull request <community_pull_requests>`.
+The best way to get a feature into Ansible is to :ref:`submit a pull request <community_pull_requests>`, either against ansible-base or against a collection. See also :ref:`ansible_collection_merge_requirements`.

docs/docsite/rst/conf.py

@@ -147,7 +147,7 @@ html_context = {
     'current_version': version,
     'latest_version': '2.9',
     # list specifically out of order to make latest work
-    'available_versions': ('latest', '2.8', '2.7', 'devel'),
+    'available_versions': ('latest', '2.9_ja', '2.8', '2.7', 'devel'),
     'css_files': ('_static/ansible.css',  # overrides to the standard theme
                   ),
 }
@@ -277,6 +277,10 @@ autoclass_content = 'both'
 intersphinx_mapping = {'python': ('https://docs.python.org/2/', (None, '../python2.inv')),
                        'python3': ('https://docs.python.org/3/', (None, '../python3.inv')),
                        'jinja2': ('http://jinja.palletsprojects.com/', (None, '../jinja2.inv')),
+                       'collections': ('https://docs.ansible.com/collections/',
+                                       (None, '../collections.inv',
+                                        'http://docs.testing.ansible.com/collections/objects.inv',
+                                        '../_collections_build/html/objects.inv')),
                        'ansible_2_9': ('https://docs.ansible.com/ansible/2.9/', (None, '../ansible_2_9.inv')),
                        'ansible_2_8': ('https://docs.ansible.com/ansible/2.8/', (None, '../ansible_2_8.inv')),
                        'ansible_2_7': ('https://docs.ansible.com/ansible/2.7/', (None, '../ansible_2_7.inv')),

docs/docsite/rst/dev_guide/developing_api.rst

@@ -71,6 +71,18 @@ This example is a simple demonstration that shows how to minimally run a couple
     # variable manager takes care of merging all the different sources to give you a unified view of variables available in each context
     variable_manager = VariableManager(loader=loader, inventory=inventory)
 
+    # Instantiate task queue manager, which takes care of forking
+    # and setting up all objects to iterate over host list and tasks.
+    # IMPORTANT: This also adds library dirs paths to the module loader
+    # IMPORTANT: and so it must be initialized before calling `Play.load()`.
+    tqm = TaskQueueManager(
+        inventory=inventory,
+        variable_manager=variable_manager,
+        loader=loader,
+        passwords=passwords,
+        stdout_callback=results_callback,  # Use our custom callback instead of the ``default`` callback plugin, which prints to stdout
+    )
+
     # create data structure that represents our play, including tasks, this is basically what our YAML loader does internally.
     play_source =  dict(
             name = "Ansible Play",
@@ -86,16 +98,8 @@ This example is a simple demonstration that shows how to minimally run a couple
     # this will also automatically create the task objects from the info provided in play_source
     play = Play().load(play_source, variable_manager=variable_manager, loader=loader)
 
-    # Run it - instantiate task queue manager, which takes care of forking and setting up all objects to iterate over host list and tasks
-    tqm = None
+    # Actually run it
     try:
-        tqm = TaskQueueManager(
-                  inventory=inventory,
-                  variable_manager=variable_manager,
-                  loader=loader,
-                  passwords=passwords,
-                  stdout_callback=results_callback,  # Use our custom callback instead of the ``default`` callback plugin, which prints to stdout
-              )
         result = tqm.run(play) # most interesting data for a play is actually sent to the callback's methods
     finally:
         # we always need to cleanup child procs and the structures we use to communicate with them

docs/docsite/rst/dev_guide/developing_collections.rst

@@ -5,7 +5,6 @@
 Developing collections
 **********************
 
-
 Collections are a distribution format for Ansible content. You can use collections to package and distribute playbooks, roles, modules, and plugins.
 You can publish and use collections through `Ansible Galaxy <https://galaxy.ansible.com>`_.
 
@@ -613,6 +612,59 @@ Now you can use this checkout of ``community.general`` in playbooks and roles wi
 
 For collections hosted in the ``ansible_collections`` GitHub org, create a branch and commit your changes on the branch. When you are done (remember to add tests, see :ref:`testing_collections`), push your changes to your fork of the collection and create a Pull Request. For other collections, especially for collections not hosted on GitHub, check the ``README.md`` of the collection for information on contributing to it.
 
+.. _collection_changelogs:
+
+Generating changelogs for a collection
+======================================
+
+We recommend that you use the `antsibull-changelog <https://github.com/ansible-community/antsibull-changelog>`_ tool to generate Ansible-compatible changelogs for your collection. The Ansible changelog uses the output of this tool to collate all the collections included in an Ansible release into one combined changelog for the release.
+
+.. note::
+
+	Ansible here refers to the Ansible 2.10 or later release that includes a curated set of collections.
+
+If your collection is part of Ansible but you are not using this tool, your collection should include the properly formatted ``changelog.yaml`` file or your changelogs will not be part of the combined Ansible CHANGELOG.rst and Porting Guide at release.  See the `changlog.yaml format <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelog.yaml-format.md>`_ for details.
+
+Understanding antsibull-changelog
+---------------------------------
+
+The ``antsibull-changelog`` tool allows you to create and update changelogs for Ansible collections that are compatible with the combined Ansible changelogs. This is an update to the changelog generator used in prior Ansible releases. The tool adds three new changelog fragment categories: ``breaking_changes``, ``security_fixes`` and ``trivial``. The tool also generates the ``changelog.yaml`` file that Ansible uses to create the combined ``CHANGELOG.rst`` file and Porting Guide for the release.
+
+See :ref:`changelogs_how_to` and the `antsibull-changelog documentation <https://github.com/ansible-community/antsibull-changelog/tree/main/docs>`_ for complete details.
+
+.. note::
+
+	The collection maintainers set the changelog policy for their collections. See the individual collection contributing guidelines for complete details.
+
+Generating changelogs
+---------------------
+
+To initialize changelog generation:
+
+#. Install ``antsibull-changelog``: :code:`pip install antsibull-changelog`.
+#. Initialize changelogs for your repository: :code:`antsibull-changelog init <path/to/your/collection>`.
+#. Optionally, edit the ``changelogs/config.yaml`` file to customize the location of the generated changelog ``.rst`` file or other options. See `Bootstrapping changelogs for collections <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst#bootstrapping-changelogs-for-collections>`_ for details.
+
+To generate changelogs from the changelog fragments you created:
+
+#. Optionally, validate your changelog fragments: :code:`antsibull-changelog lint`.
+#. Generate the changelog for your release: :code:`antsibull-changelog release [--version version_number]`.
+
+.. note::
+
+  Add the  ``--reload-plugins`` option if you ran the ``antsibull-changelog release`` command previously and the version of the collection has not changed. ``antsibull-changelog`` caches the information on all plugins and does not update its cache until the collection version changes.
+
+
+Porting Guide entries
+----------------------
+
+The following changelog fragment categories are consumed by the Ansible changelog generator into the Ansible Porting Guide:
+
+* ``major_changes``
+* ``breaking_changes``
+* ``deprecated_features``
+* ``removed_features``
+
 
 .. seealso::
 

docs/docsite/rst/dev_guide/developing_locally.rst

@@ -43,8 +43,16 @@ Once you save your module file in one of these locations, Ansible will load it a
 
 To confirm that ``my_custom_module`` is available:
 
+* type ``ansible localhost -m my_custom_module``. You should see the output for that module.
+
+or 
+
 * type ``ansible-doc -t module my_custom_module``. You should see the documentation for that module.
 
+.. note::
+
+	Currently, ``ansible-doc`` command can only parse Python modules for the module documentation. If you have module written in a different programming language other than Python, please write a documentation in Python file adjacent to module file.
+
 To use a local module only in certain playbooks:
 
 * store it in a sub-directory called ``library`` in the directory that contains the playbook(s)

docs/docsite/rst/dev_guide/developing_modules_best_practices.rst

@@ -174,3 +174,4 @@ Module Security
 * If you must use the shell, you must pass ``use_unsafe_shell=True`` to ``module.run_command``.
 * If any variables in your module can come from user input with ``use_unsafe_shell=True``, you must wrap them with ``pipes.quote(x)``.
 * When fetching URLs, use ``fetch_url`` or ``open_url`` from ``ansible.module_utils.urls``. Do not use ``urllib2``, which does not natively verify TLS certificates and so is insecure for https.
+* Sensitive values marked with ``no_log=True`` will automatically have that value stripped from module return values. If your module could return these sensitive values as part of a dictionary key name, you should call the ``ansible.module_utils.basic.sanitize_keys()`` function to strip the values from the keys. See the ``uri`` module for an example.

docs/docsite/rst/dev_guide/developing_modules_documenting.rst

@@ -193,6 +193,7 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
 :seealso:
 
   * A list of references to other modules, documentation or Internet resources
+  * In Ansible 2.10 and later, references to modules must use  the FQCN or ``ansible.builtin`` for modules in ``ansible-base``.
   * A reference can be one of the following formats:
 
 
@@ -201,10 +202,10 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
         seealso:
 
         # Reference by module name
-        - module: aci_tenant
+        - module: cisco.aci.aci_tenant
 
         # Reference by module name, including description
-        - module: aci_tenant
+        - module: cisco.aci.aci_tenant
           description: ACI module to create tenants on a Cisco ACI fabric.
 
         # Reference by rST documentation anchor
@@ -232,18 +233,23 @@ You can link from your module documentation to other module docs, other resource
 * ``R()`` for cross-references with a heading (added in Ansible 2.10). For example: ``See R(Cisco IOS Platform Guide,ios_platform_options)``.  Use the RST anchor for the cross-reference. See :ref:`adding_anchors_rst` for details.
 * ``I()`` for option names. For example: ``Required if I(state=present).``
 * ``C()`` for files and option values. For example: ``If not set the environment variable C(ACME_PASSWORD) will be used.``
-* ``M()`` for module names. For example: ``See also M(win_copy) or M(win_template).``
+* ``M()`` for module names. For example: ``See also M(ansible.builtin.yum) or M(community.general.apt_rpm)``.
 
 .. note::
 
-  For links between modules and documentation within a collection, you can use any of the options above. For links outside of your collection, use ``R()`` if available. Otherwise, use ``U()`` or ``L()`` with full URLs (not relative links). If you are creating your own documentation site, you will need to use the `intersphinx extension <https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html>`_ to convert ``R()`` and ``M()`` to the correct links.
+  For links between modules and documentation within a collection, you can use any of the options above. For links outside of your collection, use ``R()`` if available. Otherwise, use ``U()`` or ``L()`` with full URLs (not relative links). For modules, use ``M()`` with the FQCN or ``ansible.builtin`` as shown in the example. If you are creating your own documentation site, you will need to use the `intersphinx extension <https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html>`_ to convert ``R()`` and ``M()`` to the correct links.
 
 
+.. note::
+    - To refer to a group of modules in a collection, use ``R()``.  When a collection is not the right granularity, use ``C(..)``:
+
+        -``Refer to the R(community.kubernetes collection, plugins_in_community.kubernetes) for information on managing kubernetes clusters.``
+        -``The C(win_*) modules (spread across several collections) allow you to manage various aspects of windows hosts.``
+
 
 .. note::
 
-    - To refer a group of modules, use ``C(..)``, e.g. ``Refer to the C(win_*) modules.``
-    - Because it stands out better, using ``seealso`` is preferred for general references over the use of notes or adding links to the description.
+   Because it stands out better, use ``seealso`` for general references over the use of notes or adding links to the description.
 
 .. _module_docs_fragments:
 

docs/docsite/rst/dev_guide/developing_modules_general.rst

@@ -51,7 +51,7 @@ Starting a new module
 
 To create a new module:
 
-1. Navigate to the correct directory for your new module: ``$ cd lib/ansible/modules/cloud/azure/``
+1. Navigate to the correct directory for your new module: ``$ cd lib/ansible/modules/``
 2. Create your new module file: ``$ touch my_test.py``
 3. Paste the content below into your new module file. It includes the :ref:`required Ansible format and documentation <developing_modules_documenting>` and some example code.
 4. Modify and extend the code to do what you want your new module to do. See the :ref:`programming tips <developing_modules_best_practices>` and :ref:`Python 3 compatibility <developing_python_3>` pages for pointers on writing clean, concise module code.
@@ -62,8 +62,10 @@ To create a new module:
 
     # Copyright: (c) 2018, Terry Jones <terry.jones@example.org>
     # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+    from __future__ import (absolute_import, division, print_function)
+    __metaclass__ = type
 
-    DOCUMENTATION = '''
+    DOCUMENTATION = r'''
     ---
     module: my_test
 
@@ -72,17 +74,19 @@ To create a new module:
     version_added: "2.4"
 
     description:
-        - "This is my longer description explaining my test module"
+        - "This is my longer description explaining my test module."
 
     options:
         name:
             description:
-                - This is the message to send to the test module
+                - This is the message to send to the test module.
             required: true
+            type: str
         new:
             description:
-                - Control to demo if the result of this module is changed or not
+                - Control to demo if the result of this module is changed or not.
             required: false
+            type: bool
 
     extends_documentation_fragment:
         - azure
@@ -91,7 +95,7 @@ To create a new module:
         - Your Name (@yourhandle)
     '''
 
-    EXAMPLES = '''
+    EXAMPLES = r'''
     # Pass in a message
     - name: Test with a message
       my_test:
@@ -109,7 +113,7 @@ To create a new module:
         name: fail me
     '''
 
-    RETURN = '''
+    RETURN = r'''
     original_message:
         description: The original name param that was passed in
         type: str
@@ -208,7 +212,7 @@ If your module does not need to target a remote host, you can quickly and easily
    development) activate it: ``$ . venv/bin/activate``
 -  Setup the environment for development: ``$ . hacking/env-setup``
 -  Run your test module locally and directly:
-   ``$ python -m ansible.modules.cloud.azure.my_test /tmp/args.json``
+   ``$ python -m ansible.modules.my_test /tmp/args.json``
 
 This should return output like this:
 

docs/docsite/rst/dev_guide/developing_modules_general_windows.rst

@@ -213,7 +213,7 @@ options set:
 - ``elements``: When ``type=list``, this sets the type of each list value, the values are the same as ``type``
 - ``no_log``: Will sanitise the input value before being returned in the ``module_invocation`` return value
 - ``removed_in_version``: States when a deprecated module option is to be removed, a warning is displayed to the end user if set
-- ``removed_at_date``: States the date when a deprecated module option will be removed, a warning is displayed to the end user if set
+- ``removed_at_date``: States the date (YYYY-MM-DD) when a deprecated module option will be removed, a warning is displayed to the end user if set
 - ``removed_from_collection``: States from which collection the deprecated module option will be removed; must be specified if one of ``removed_in_version`` and ``removed_at_date`` is specified
 - ``required``: Will fail when the module option is not set
 - ``type``: The type of the module option, if not set then it defaults to ``str``. The valid types are;

docs/docsite/rst/dev_guide/developing_modules_in_groups.rst

@@ -1,64 +1,49 @@
 .. _developing_modules_in_groups:
 
-*********************************************
-Information for submitting a group of modules
-*********************************************
+*************************
+Creating a new collection
+*************************
 
-.. contents:: Topics
+Starting with Ansible 2.10, related modules should be developed in a collection. The Ansible core team and community compiled these module development tips and tricks to help companies developing Ansible modules for their products and users developing Ansible modules for third-party products. See :ref:`developing_collections` for a more detailed description of the collections format and additional development guidelines.
+
+.. contents::
    :local:
 
-Submitting a group of modules
-=============================
-
-This section discusses how to get multiple related modules into Ansible.
-
-This document is intended for both companies wishing to add modules for their own products as well as users of 3rd party products wishing to add Ansible functionality.
-
-It's based on module development tips and tricks that the Ansible core team and community have accumulated.
-
 .. include:: shared_snippets/licensing.txt
 
 Before you start coding
 =======================
 
-Although it's tempting to get straight into coding, there are a few things to be aware of first. This list of prerequisites is designed to help ensure that you develop high-quality modules that flow easily through the review process and get into Ansible more quickly.
+This list of prerequisites is designed to help ensure that you develop high-quality modules that work well with ansible-base and provide a seamless user experience.
 
 * Read though all the pages linked off :ref:`developing_modules_general`; paying particular focus to the :ref:`developing_modules_checklist`.
-* New modules must be PEP 8 compliant. See :ref:`testing_pep8` for more information.
-* Starting with Ansible version 2.4, all new modules must :ref:`support Python 2.6+ and Python 3.5+ <developing_python_3>`. If this is an issue, please contact us (see the "Speak to us" section later in this document to learn how).
-* Have a look at the existing modules and how they've been named in the :ref:`all_modules`, especially in the same functional area (such as cloud, networking, databases).
-* Shared code can be placed into ``lib/ansible/module_utils/``
-* Shared documentation (for example describing common arguments) can be placed in ``lib/ansible/plugins/doc_fragments/``.
-* With great power comes great responsibility: Ansible module maintainers have a duty to help keep modules up to date. As with all successful community projects, module maintainers should keep a watchful eye for reported issues and contributions.
-* Although not required, unit and/or integration tests are strongly recommended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :ref:`developing_testing` and the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
-  * Starting with Ansible 2.4 all :ref:`network_modules` MUST have unit tests.
+* We encourage PEP 8 compliance. See :ref:`testing_pep8` for more information.
+* We encourage supporting :ref:`Python 2.6+ and Python 3.5+ <developing_python_3>`.
+* Look at Ansible Galaxy and review the naming conventions in your functional area (such as cloud, networking, databases).
+* With great power comes great responsibility: Ansible collection maintainers have a duty to help keep content up to date. As with all successful community projects, collection maintainers should keep a watchful eye for reported issues and contributions.
+* We strongly recommend unit and/or integration tests. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :ref:`developing_testing` and the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
 
 
-Naming convention
-=================
+Naming conventions
+==================
 
-As you may have noticed when looking under ``lib/ansible/modules/`` we support up to two directories deep (but no deeper), e.g. `databases/mysql`. This is used to group files on disk as well as group related modules into categories and topics the Module Index, for example: :ref:`database_modules`.
+Fully Qualified Collection Names (FQCNs) for plugins and modules include three elements:
 
-The directory name should represent the *product* or *OS* name, not the company name.
+  * the Galaxy namespace, which generally represents the company or group
+  * the collection name, which generally represents the product or OS
+  * the plugin or module name
+      * always in lower case
+      * words separated with an underscore (``_``) character
+      * singular, rather than plural, for example, ``command`` not ``commands``
 
-Each module should have the above (or similar) prefix; see existing :ref:`all_modules` for existing examples.
-
-**Note:**
-
-* File and directory names are always in lower case
-* Words are separated with an underscore (``_``) character
-* Module names should be in the singular, rather than plural, eg ``command`` not ``commands``
+For example, ``community.mongodb.mongodb_linux`` or ``cisco.meraki.meraki_device``.
 
+It is convenient if the organization and repository names on GitHub (or elsewhere) match your namespace and collection names on Ansible Galaxy, but it is not required. The plugin names you select, however, are always the same in your code repository and in your collection artifact on Galaxy.
 
 Speak to us
 ===========
 
-Circulating your ideas before coding is a good way to help you set off in the right direction.
-
-After reading the "Before you start coding" section you will hopefully have a reasonable idea of the structure of your modules.
-
-We've found that writing a list of your proposed module names and a one or two line description of what they will achieve and having that reviewed by Ansible is a great way to ensure the modules fit the way people have used Ansible Modules before, and therefore make them easier to use.
-
+Circulating your ideas before coding helps you adopt good practices and avoid common mistakes. After reading the "Before you start coding" section you should have a reasonable idea of the structure of your modules. Write a list of your proposed plugin and/or module names, with a short description of what each one does. Circulate that list on IRC or a mailing list so the Ansible community can review your ideas for consistency and familiarity. Names and functionality that are consistent, predictable, and familiar make your collection easier to use.
 
 Where to get support
 ====================
@@ -67,76 +52,23 @@ Ansible has a thriving and knowledgeable community of module developers that is
 
 In the :ref:`ansible_community_guide` you can find how to:
 
-* Subscribe to the Mailing Lists - We suggest "Ansible Development List" (for codefreeze info) and "Ansible Announce list"
-* ``#ansible-devel`` - We have found that IRC ``#ansible-devel`` on FreeNode's IRC network works best for module developers so we can have an interactive dialogue.
+* Subscribe to the Mailing Lists - We suggest "Ansible Development List" and "Ansible Announce list"
+* ``#ansible-devel`` - We have found that IRC ``#ansible-devel`` on FreeNode's IRC network works best for developers so we can have an interactive dialogue.
 * IRC meetings - Join the various weekly IRC meetings `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_
 
-
-Your first pull request
-=======================
-
-Now that you've reviewed this document, you should be ready to open your first pull request.
-
-The first PR is slightly different to the rest because it:
-
-* defines the namespace
-* provides a basis for detailed review that will help shape your future PRs
-* may include shared documentation (`doc_fragments`) that multiple modules require
-* may include shared code (`module_utils`) that multiple modules require
-
-
-The first PR should include the following files:
-
-* ``lib/ansible/modules/$category/$topic/__init__.py`` - An empty file to initialize namespace and allow Python to import the files. *Required new file*
-* ``lib/ansible/modules/$category/$topic/$yourfirstmodule.py`` - A single module. *Required new file*
-* ``lib/ansible/plugins/doc_fragments/$topic.py`` - Code documentation, such as details regarding common arguments. *Optional new file*
-* ``lib/ansible/module_utils/$topic.py`` - Code shared between more than one module, such as common arguments. *Optional new file*
-
-And that's it.
-
-Before pushing your PR to GitHub it's a good idea to review the :ref:`developing_modules_checklist` again.
-
-After publishing your PR to https://github.com/ansible/ansible, a Shippable CI test should run within a few minutes. Check the results (at the end of the PR page) to ensure that it's passing (green). If it's not passing, inspect each of the results. Most of the errors should be self-explanatory and are often related to badly formatted documentation (see :ref:`yaml_syntax`) or code that isn't valid Python 2.6  or valid Python 3.5 (see :ref:`developing_python_3`). If you aren't sure what a Shippable test message means, copy it into the PR along with a comment and we will review.
-
-If you need further advice, consider join the ``#ansible-devel`` IRC channel (see how in the "Where to get support").
-
-
-We have a ``ansibullbot`` helper that comments on GitHub Issues and PRs which should highlight important information.
-
-
-Subsequent PRs
+Required files
 ==============
 
-By this point you first PR that defined the module namespace should have been merged. You can take the lessons learned from the first PR and apply it to the rest of the modules.
+Your collection should include the following files to be usable:
 
-Raise exactly one PR per module for the remaining modules.
+* an ``__init__.py`` file - An empty file to initialize namespace and allow Python to import the files. *Required*
+* at least one plugin, for example, ``/plugins/modules/$your_first_module.py``. *Required*
+* if needed, one or more ``/plugins/doc_fragments/$topic.py`` files - Code documentation, such as details regarding common arguments. *Optional*
+* if needed, one or more ``/plugins/module_utils/$topic.py`` files - Code shared between more than one module, such as common arguments. *Optional*
 
-Over the years we've experimented with different sized module PRs, ranging from one module to many tens of modules, and during that time we've found the following:
+When you have these files ready, review the :ref:`developing_modules_checklist` again. If you are creating a new collection, you are responsible for all procedures related to your repository, including setting rules for contributions, finding reviewers, and testing and maintaining the code in your collection.
 
-* A PR with a single file gets a higher quality review
-* PRs with multiple modules are harder for the creator to ensure all feedback has been applied
-* PRs with many modules take a lot more work to review, and tend to get passed over for easier-to-review PRs.
-
-You can raise up to five PRs at once (5 PRs = 5 new modules) **after** your first PR has been merged. We've found this is a good batch size to keep the review process flowing.
-
-Maintaining your modules
-========================
-
-Now that your modules are integrated there are a few bits of housekeeping to be done.
-
-**Bot Meta**
-Update `Ansibullbot` so it knows who to notify if/when bugs or PRs are raised against your modules
-`BOTMETA.yml <https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml>`_.
-
-If there are multiple people that can be notified, please list them. That avoids waiting on a single person who may be unavailable for any reason. Note that in `BOTMETA.yml` you can take ownership of an entire directory.
-
-
-**Review Module web docs**
-Review the autogenerated module documentation for each of your modules, found in :ref:`Module Docs <modules_by_category>` to ensure they are correctly formatted. If there are any issues please fix by raising a single PR.
-
-If the module documentation hasn't been published live yet, please let a member of the Ansible Core Team know in the ``#ansible-devel`` IRC channel.
-
-.. note:: Consider adding a scenario guide to cover how to use your set of modules. Use the :ref:`sample scenario guide rst file <scenario_template>` to help you get started. For network modules, see :ref:`documenting_modules_network` for further documentation requirements.
+If you need help or advice, consider join the ``#ansible-devel`` IRC channel (see how in the "Where to get support").
 
 New to git or GitHub
 ====================
@@ -146,9 +78,3 @@ We realize this may be your first use of Git or GitHub. The following guides may
 * `How to create a fork of ansible/ansible <https://help.github.com/articles/fork-a-repo/>`_
 * `How to sync (update) your fork <https://help.github.com/articles/syncing-a-fork/>`_
 * `How to create a Pull Request (PR) <https://help.github.com/articles/about-pull-requests/>`_
-
-Please note that in the Ansible Git Repo the main branch is called ``devel`` rather than ``master``, which is used in the official GitHub documentation
-
-After your first PR has been merged ensure you "sync your fork" with ``ansible/ansible`` to ensure you've pulled in the directory structure and and shared code or documentation previously created.
-
-As stated in the GitHub documentation, always use feature branches for your PRs, never commit directly into ``devel``.

docs/docsite/rst/dev_guide/platforms/aws_guidelines.rst

@@ -638,7 +638,7 @@ available during the test run. Second putting the test in a test group causing i
 continuous integration build.
 
 Tests for new modules should be added to the same group as existing AWS tests. In general just copy
-an existing aliases file such as the `aws_s3 tests aliases file <https://github.com/ansible/ansible/blob/devel/test/integration/targets/aws_s3/aliases>`_.
+an existing aliases file such as the `aws_s3 tests aliases file <https://github.com/ansible-collections/amazon.aws/blob/master/tests/integration/targets/aws_s3/aliases>`_.
 
 AWS Credentials for Integration Tests
 -------------------------------------
@@ -678,13 +678,13 @@ AWS Permissions for Integration Tests
 -------------------------------------
 
 As explained in the :ref:`Integration Test guide <testing_integration>`
-there are defined IAM policies in ``hacking/aws_config/testing_policies/`` that contain the necessary permissions
-to run the AWS integration test. The permissions used by CI are more restrictive than those in ``hacking/aws_config/testing_policies``; for CI we want
-the most restrictive policy possible that still allows the given tests to pass.
+there are defined IAM policies in `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator>`_ that contain the necessary permissions
+to run the AWS integration test.
 
 If your module interacts with a new service or otherwise requires new permissions, tests will fail when you submit a pull request and the
 `Ansibullbot <https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md>`_ will tag your PR as needing revision.
-We do not automatically grant additional permissions to the roles used by the continuous integration builds. You must provide the minimum IAM permissions required to run your integration test.
+We do not automatically grant additional permissions to the roles used by the continuous integration builds.
+You will need to raise a Pull Request against `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator>`_ to add them.
 
 If your PR has test failures, check carefully to be certain the failure is only due to the missing permissions. If you've ruled out other sources of failure, add a comment with the `ready_for_review`
 tag and explain that it's due to missing permissions.
@@ -705,7 +705,7 @@ To start with the most permissive IAM policy:
 3) Modify your policy to allow only the actions your tests use. Restrict account, region, and prefix where possible. Wait a few minutes for your policy to update.
 4) Run the tests again with a user or role that allows only the new policy.
 5) If the tests fail, troubleshoot (see tips below), modify the policy, run the tests again, and repeat the process until the tests pass with a restrictive policy.
-6) Open a pull request proposing the minimum required policy to the `testing policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_.
+6) Open a pull request proposing the minimum required policy to the `CI policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_.
 
 To start from the least permissive IAM policy:
 
@@ -722,7 +722,7 @@ To start from the least permissive IAM policy:
 3) Add the action or resource that caused the failure to `an IAM policy <https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html#access_policies_create-start>`_. Wait a few minutes for your policy to update.
 4) Run the tests again with this policy attached to your user or role.
 5) If the tests still fail at the same place with the same error you will need to troubleshoot (see tips below). If the first test passes, repeat steps 2 and 3 for the next error. Repeat the process until the tests pass with a restrictive policy.
-6) Open a pull request proposing the minimum required policy to the `testing policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_.
+6) Open a pull request proposing the minimum required policy to the `CI policies <https://github.com/mattclay/aws-terminator/tree/master/aws/policy>`_.
 
 Troubleshooting IAM policies
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -737,7 +737,23 @@ Troubleshooting IAM policies
 - Use a search engine.
 - Ask in the Ansible IRC channel #ansible-aws (on freenode IRC).
 
+Unsupported Integration tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+There are a limited number of reasons why it may not be practical to run integration
+tests for a module within CI.  Where these apply you should add the keyword
+`unsupported` to the aliases file in `test/integration/targets/MODULE_NAME/aliases`.
+
 Some cases where tests should be marked as unsupported:
 1) The tests take longer than 10 or 15 minutes to complete
 2) The tests create expensive resources
 3) The tests create inline policies
+4) The tests require the existance of external resources
+5) The tests manage Account level security policies such as the password policy or AWS Organizations.
+
+Where one of these reasons apply you should open a pull request proposing the minimum required policy to the
+`unsupported test policies <https://github.com/mattclay/aws-terminator/tree/master/hacking/aws_config/test_policies>`_.
+
+Unsupported integration tests will not be automatically run by CI.  However, the
+necessary policies should be available so that the tests can be manually run by
+someone performing a PR review or writing a patch.

docs/docsite/rst/dev_guide/testing_integration.rst

@@ -190,10 +190,19 @@ Ansible needs fairly wide ranging powers to run the tests in an AWS account.  Th
 testing-policies
 ----------------
 
-``hacking/aws_config/testing_policies`` contains a set of policies that are required for all existing AWS module tests.
-The ``hacking/aws_config/setup_iam.yml`` playbook can be used to add all of those policies to an IAM group (using
-``-e iam_group=GROUP_NAME``. Once the group is created, you'll need to create a user and make the user a member of the
-group. The policies are designed to minimize the rights of that user.  Please note that while this policy does limit
+The GitHub repository `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator/>`_
+contains two sets of policies used for all existing AWS module integratoin tests.
+The `hacking/aws_config/setup_iam.yml` playbook can be used to setup two groups:
+
+  - `ansible-integration-ci` will have the policies applied necessary to run any
+    integration tests not marked as `unsupported` and are designed to mirror those
+    used by Ansible's CI.
+  - `ansible-integration-unsupported` will have the additional policies applied
+    necessary to run the integraion tests marked as `unsupported` including tests
+    for managing IAM roles, users and groups.
+
+Once the groups have been created, you'll need to create a user and make the user a member of these
+groups. The policies are designed to minimize the rights of that user.  Please note that while this policy does limit
 the user to one region, this does not fully restrict the user (primarily due to the limitations of the Amazon ARN
 notation). The user will still have wide privileges for viewing account definitions, and will also able to manage
 some resources that are not related to testing (for example, AWS lambdas with different names).  Tests should not

docs/docsite/rst/index.rst

@@ -74,7 +74,7 @@ Ansible releases a new major release of Ansible approximately three to four time
    :maxdepth: 1
    :caption: Reference & Appendices
 
-   ../modules/modules_by_category
+   collections/index
    reference_appendices/playbooks_keywords
    reference_appendices/common_return_values
    reference_appendices/config

docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst

@@ -511,9 +511,9 @@ For platforms that support ``connection: local`` *and* ``connection: network_cli
 * Name the :file:`targets/` directories after the module name.
 * The :file:`main.yaml` file should just reference the transport.
 
-The following example walks through the integration tests for the ``vyos.vyos.vyos_banner`` module in the `vyos.vyos <https://github.com/ansible-collections/vyos.vyos/tree/master/tests/integration>`_ collection:
+The following example walks through the integration tests for the ``vyos.vyos.vyos_l3_interfaces`` module in the `vyos.vyos <https://github.com/ansible-collections/vyos.vyos/tree/master/tests/integration>`_ collection:
 
-``test/integration/targets/vyos_banner/tasks/main.yaml``
+``test/integration/targets/vyos_l3_interfaces/tasks/main.yaml``
 
 .. code-block:: yaml
 
@@ -522,7 +522,7 @@ The following example walks through the integration tests for the ``vyos.vyos.vy
      tags:
        - cli
 
-``test/integration/targets/vyos_banner/tasks/cli.yaml``
+``test/integration/targets/vyos_l3_interfaces/tasks/cli.yaml``
 
 .. code-block:: yaml
 
@@ -549,46 +549,71 @@ The following example walks through the integration tests for the ``vyos.vyos.vy
      loop_control:
        loop_var: test_case_to_run
 
-``test/integration/targets/vyos_banner/tests/cli/basic-no-login.yaml``
+``test/integration/targets/vyos_l3_interfaces/tests/cli/overridden.yaml``
 
 .. code-block:: yaml
 
-   ---
-   - debug:
-       msg: "cli/basic-no-login.yaml on connection={{ ansible_connection }}"
+  ---
+  - debug:
+   msg: START vyos_l3_interfaces merged integration tests on connection={{ ansible_connection
+     }}
 
-   - name: Setup
-     vyos_banner:
-       banner: pre-login
-       text: |
-         Junk pre-login banner
-         over multiple lines
-       state: present
+  - include_tasks: _remove_config.yaml
 
-   - name: remove pre-login
-     vyos_banner:
-       banner: pre-login
-       state: absent
+  - block:
+
+   - include_tasks: _populate.yaml
+
+   - name: Overrides all device configuration with provided configuration
      register: result
+     vyos.vyos.vyos_l3_interfaces: &id001
+       config:
 
-   - debug:
-       msg: "{{ result }}"
+         - name: eth0
+           ipv4:
 
-   - assert:
+             - address: dhcp
+
+         - name: eth1
+           ipv4:
+
+             - address: 192.0.2.15/24
+       state: overridden
+
+   - name: Assert that before dicts were correctly generated
+     assert:
        that:
-         - "result.changed == true"
-         - "'delete system login banner pre-login' in result.commands"
+         - "{{ populate | symmetric_difference(result['before']) |length == 0 }}"
 
-   - name: remove pre-login (idempotent)
-     vyos_banner:
-       banner: pre-login
-       state: absent
+   - name: Assert that correct commands were generated
+     assert:
+       that:
+         - "{{ overridden['commands'] | symmetric_difference(result['commands'])\
+           \ |length == 0 }}"
+
+   - name: Assert that after dicts were correctly generated
+     assert:
+       that:
+         - "{{ overridden['after'] | symmetric_difference(result['after']) |length\
+           \ == 0 }}"
+
+   - name: Overrides all device configuration with provided configurations (IDEMPOTENT)
      register: result
+     vyos.vyos.vyos_l3_interfaces: *id001
 
-   - assert:
+   - name: Assert that the previous task was idempotent
+     assert:
        that:
-         - "result.changed == false"
-         - "result.commands | length == 0"
+         - result['changed'] == false
+
+   - name: Assert that before dicts were correctly generated
+     assert:
+       that:
+         - "{{ overridden['after'] | symmetric_difference(result['before']) |length\
+           \ == 0 }}"
+  always:
+
+   - include_tasks: _remove_config.yaml
 
 
 Detecting test resources at runtime
@@ -659,14 +684,14 @@ To run integration tests for a specific module:
 
 .. code-block:: bash
 
-  ansible-test network-integration --inventory  /path/to-collection-module/test/integration/inventory.networking vyos_vlan
+  ansible-test network-integration --inventory  /path/to-collection-module/test/integration/inventory.networking vyos_l3_interfaces
 
 To run a single test case on a specific module:
 
 .. code-block:: bash
 
-   # Only run vyos_vlan/tests/cli/basic.yaml
-   ansible-test network-integration --inventory  /path/to-collection-module/test/integration/inventory.networking vyos_vlan --testcase basic
+   # Only run vyos_l3_interfaces/tests/cli/gathered.yaml
+   ansible-test network-integration --inventory  /path/to-collection-module/test/integration/inventory.networking vyos_l3_interfaces --testcase gathered
 
 To run integration tests for a specific transport:
 

docs/docsite/rst/network/index.rst

@@ -10,7 +10,7 @@ Ansible Network modules extend the benefits of simple, powerful, agentless autom
 
 If you're new to Ansible, or new to using Ansible for network management, start with :ref:`network_getting_started`. If you are already familiar with network automation with Ansible, see :ref:`network_advanced`.
 
-For documentation on using a particular network module, consult the :ref:`list of all network modules<network_modules>`. Some network modules are maintained by the Ansible community - here's a list of :ref:`network modules maintained by the Ansible Network Team<network_supported>`.
+For documentation on using a particular network module, consult the :ref:`list of all network modules<network_modules>`. Network modules for various hardware are supported by different teams including the hardware vendors themselves, volunteers from the Ansible community, and the Ansible Network Team.
 
 .. toctree::
    :maxdepth: 3

docs/docsite/rst/network/user_guide/network_best_practices_2.5.rst

@@ -483,4 +483,4 @@ If you receive an connection error please double check the inventory and playboo
 
   * :ref:`network_guide`
   * :ref:`intro_inventory`
-  * :ref:`Vault best practices <best_practices_for_variables_and_vaults>`
+  * :ref:`Keeping vaulted variables visible <tip_for_variables_and_vaults>`

docs/docsite/rst/network/user_guide/platform_index.rst

@@ -59,35 +59,61 @@ Settings by Platform
     --------------------------------------------------------  ------------------------------------------
     Network OS                       ``ansible_network_os:``  network_cli  netconf  httpapi  local
     ===============================  =======================  ===========  =======  =======  ===========
-    Arista EOS `[†]`_                ``eos``                  ✓                     ✓        ✓
-    Cisco ASA                        ``asa``                  ✓                              ✓
-    Cisco IOS `[†]`_                 ``ios``                  ✓                              ✓
-    Cisco IOS XR `[†]`_              ``iosxr``                ✓                              ✓
-    Cisco NX-OS `[†]`_               ``nxos``                 ✓                     ✓        ✓
-    Cloudengine OS `[†]`_            ``ce``                   ✓            ✓                 ✓
-    Dell OS6                         ``dellos6``              ✓                              ✓
-    Dell OS9                         ``dellos9``              ✓                              ✓
-    Dell OS10                        ``dellos10``             ✓                              ✓
-    Ericsson ECCLI                   ``eric_eccli``           ✓                              ✓
-    Extreme EXOS                     ``exos``                 ✓                     ✓
-    Extreme IronWare                 ``ironware``             ✓                              ✓
-    Extreme NOS                      ``nos``                  ✓
-    Extreme SLX-OS                   ``slxos``                ✓
-    Extreme VOSS                     ``voss``                 ✓
-    F5 BIG-IP                                                                                ✓
-    F5 BIG-IQ                                                                                ✓
-    Junos OS `[†]`_                  ``junos``                ✓            ✓                 ✓
-    Lenovo CNOS                      ``cnos``                 ✓                              ✓
-    Lenovo ENOS                      ``enos``                 ✓                              ✓
-    Meraki                           ``meraki``                                              ✓
-    MikroTik RouterOS                ``routeros``             ✓
-    Nokia SR OS                      ``sros``                 ✓                              ✓
-    Pluribus Netvisor                ``netvisor``             ✓
-    Ruckus ICX `[†]`_                ``icx``                  ✓
-    VyOS `[†]`_                      ``vyos``                 ✓                              ✓
+    `Arista EOS`_ `[†]`_                ``eos``                  ✓                     ✓        ✓
+    `Cisco ASA`_ `[†]`_                      ``asa``                  ✓                              ✓
+    `Cisco IOS`_ `[†]`_                 ``ios``                  ✓                              ✓
+    `Cisco IOS XR`_ `[†]`_              ``iosxr``                ✓                              ✓
+    `Cisco NX-OS`_ `[†]`_               ``nxos``                 ✓                     ✓        ✓
+    `Cloudengine OS`_                   ``ce``                   ✓            ✓                 ✓
+    `Dell OS6`_                         ``dellos6``              ✓                              ✓
+    `Dell OS9`_                        ``dellos9``              ✓                              ✓
+    `Dell OS10`_                        ``dellos10``             ✓                              ✓
+    `Ericsson ECCLI`_                   ``eric_eccli``           ✓                              ✓
+    `Extreme EXOS`_                     ``exos``                 ✓                     ✓
+    `Extreme IronWare`_                 ``ironware``             ✓                              ✓
+    `Extreme NOS`_                     ``nos``                  ✓
+    `Extreme SLX-OS`_                   ``slxos``                ✓
+    `Extreme VOSS`_                     ``voss``                 ✓
+    `F5 BIG-IP`_                                                                                ✓
+    `F5 BIG-IQ`_                                                                                ✓
+    `Junos OS`_ `[†]`_                  ``junos``                ✓            ✓                 ✓
+    `Lenovo CNOS`_                      ``cnos``                 ✓                              ✓
+    `Lenovo ENOS`_                      ``enos``                 ✓                              ✓
+    `Meraki`_                           ``meraki``                                              ✓
+    `MikroTik RouterOS`_                ``routeros``             ✓
+    `Nokia SR OS`_                      ``sros``                 ✓                              ✓
+    `Pluribus Netvisor`_                ``netvisor``             ✓
+    `Ruckus ICX`_                       ``icx``                  ✓
+    `VyOS`_ `[†]`_                      ``vyos``                 ✓                              ✓
     OS that supports Netconf `[†]`_  ``<network-os>``                      ✓                 ✓
     ===============================  =======================  ===========  =======  =======  ===========
 
+.. _Arista EOS: https://galaxy.ansible.com/arista/eos
+.. _Cisco ASA: https://galaxy.ansible.com/cisco/asa
+.. _Cisco IOS: https://galaxy.ansible.com/cisco/ios
+.. _Cisco IOS XR: https://galaxy.ansible.com/cisco/iosxr
+.. _Cisco NX-OS: https://galaxy.ansible.com/cisco/nxos
+.. _Cloudengine OS: https://galaxy.ansible.com/community/network
+.. _Dell OS6: https://github.com/ansible-collections/dellemc.os6
+.. _Dell OS9: https://github.com/ansible-collections/dellemc.os9
+.. _Dell OS10: https://galaxy.ansible.com/dellemc_networking/os10
+.. _Ericsson ECCLI: https://galaxy.ansible.com/community/network
+.. _Extreme EXOS: https://galaxy.ansible.com/community/network
+.. _Extreme IronWare: https://galaxy.ansible.com/community/network
+.. _Extreme NOS: https://galaxy.ansible.com/community/network
+.. _Extreme SLX-OS: https://galaxy.ansible.com/community/network
+.. _Extreme VOSS: https://galaxy.ansible.com/community/network
+.. _F5 BIG-IP: https://galaxy.ansible.com/f5networks/f5_modules
+.. _F5 BIG-IQ: https://galaxy.ansible.com/f5networks/f5_modules
+.. _Junos OS: https://galaxy.ansible.com/junipernetworks/junos
+.. _Lenovo CNOS: https://galaxy.ansible.com/community/network
+.. _Lenovo ENOS: https://galaxy.ansible.com/community/network
+.. _Meraki: https://galaxy.ansible.com/cisco/meraki
+.. _MikroTik RouterOS: https://galaxy.ansible.com/community/network
+.. _Nokia SR OS: https://galaxy.ansible.com/community/network
+.. _Pluribus Netvisor: https://galaxy.ansible.com/community/network
+.. _Ruckus ICX: https://galaxy.ansible.com/community/network
+.. _VyOS: https://galaxy.ansible.com/vyos/vyos
 .. _`[†]`:
 
 **[†]** Maintained by Ansible Network Team

docs/docsite/rst/plugins/become.rst

@@ -47,11 +47,6 @@ Plugin List
 You can use ``ansible-doc -t become -l`` to see the list of available plugins.
 Use ``ansible-doc -t become <plugin name>`` to see specific documentation and examples.
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    become/*
-
 .. seealso::
 
    :ref:`about_playbooks`

docs/docsite/rst/plugins/cache.rst

@@ -118,11 +118,6 @@ Plugin List
 You can use ``ansible-doc -t cache -l`` to see the list of available plugins.
 Use ``ansible-doc -t cache <plugin name>`` to see specific documentation and examples.
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    cache/*
-
 .. seealso::
 
    :ref:`action_plugins`

docs/docsite/rst/plugins/callback.rst

@@ -79,12 +79,6 @@ Plugin list
 You can use ``ansible-doc -t callback -l`` to see the list of available plugins.
 Use ``ansible-doc -t callback <plugin name>`` to see specific documents and examples.
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    callback/*
-
-
 .. seealso::
 
    :ref:`action_plugins`

docs/docsite/rst/plugins/connection.rst

@@ -58,12 +58,6 @@ You can use ``ansible-doc -t connection -l`` to see the list of available plugin
 Use ``ansible-doc -t connection <plugin name>`` to see detailed documentation and examples.
 
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    connection/*
-
-
 .. seealso::
 
    :ref:`Working with Playbooks<working_with_playbooks>`

docs/docsite/rst/plugins/inventory.rst

@@ -162,11 +162,6 @@ Plugin List
 You can use ``ansible-doc -t inventory -l`` to see the list of available plugins.
 Use ``ansible-doc -t inventory <plugin name>`` to see plugin-specific documentation and examples.
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    inventory/*
-
 .. seealso::
 
    :ref:`about_playbooks`

docs/docsite/rst/plugins/lookup.rst

@@ -138,11 +138,6 @@ Plugin list
 You can use ``ansible-doc -t lookup -l`` to see the list of available plugins. Use ``ansible-doc -t lookup <plugin name>`` to see specific documents and examples.
 
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    lookup/*
-
 .. seealso::
 
    :ref:`about_playbooks`

docs/docsite/rst/plugins/shell.rst

@@ -33,11 +33,6 @@ In this case, you will also want to update the :ref:`ansible_shell_executable <a
 You can further control the settings for each plugin via other configuration options
 detailed in the plugin themselves (linked below).
 
-.. toctree:: :maxdepth: 1
-    :glob:
-
-    shell/*
-
 .. seealso::
 
    :ref:`about_playbooks`