Start of work folding threading and forking together

It's almost always preferable to clone by way of https:// rather than
using the bare git:// protocol. Not only does https:// provide
stronger guarantees it also plays nicer with corporate proxies, etc.

.github/BOTMETA.yml

@@ -28,17 +28,16 @@
 #           deprecated - this file is deprecated but probably not yet renamed
 #           keywords - used to identify this file based on the issue description
 #           support - used for files without internal metadata
-#           labels - list of GitHub labels to apply
+#           labels - list of GitHub labels to apply. Path components of 'file' parent key
+#                    which are valid GitHub labels are automatically added.
 #
 
-automerge: False
+automerge: True
 files:
   .github/BOTMETA.yml:
-    labels:
-    - botmeta
+    labels: botmeta
   $modules/cloud/amazon/:
-    notify:
-    - willthames
+    notify: willthames
   $modules/cloud/amazon/aws_api_gateway.py: mikedlr willthames
   $modules/cloud/amazon/aws_direct_connect_connection.py: s-hertel
   $modules/cloud/amazon/aws_direct_connect_link_aggregation_group.py: s-hertel
@@ -55,7 +54,7 @@ files:
   $modules/cloud/amazon/ec2_ami.py: Constantin07 gunzy83 scicoin-project willthames
   $modules/cloud/amazon/ec2_ami_copy.py: amir343
   $modules/cloud/amazon/ec2_ami_find.py: tombamford
-  $modules/cloud/amazon/ec2_asg.py: $team_ansible garethr
+  $modules/cloud/amazon/ec2_asg.py: $team_ansible garethr s-hertel ryansb
   $modules/cloud/amazon/ec2_asg_facts.py: wimnat
   $modules/cloud/amazon/ec2_customer_gateway.py: MichaelBaydoun michaelbaydoun
   $modules/cloud/amazon/ec2_eip.py: rickmendes
@@ -173,14 +172,18 @@ files:
   $modules/cloud/azure/azure_rm_virtualnetwork_facts.py: $team_azure
   $modules/cloud/centurylink/: clc-runner
   $modules/cloud/cloudscale/cloudscale_server.py: gaudenz
-  $modules/cloud/cloudstack/: resmo
-  $modules/cloud/digital_ocean/: bondanthony
+  $modules/cloud/cloudstack/:
+    labels: cloudstack
+    maintainers: resmo
+  $modules/cloud/digital_ocean/: BondAnthony
   $modules/cloud/digital_ocean/digital_ocean.py: alukovenko zbal
   $modules/cloud/digital_ocean/digital_ocean_domain.py: alukovenko mgregson
   $modules/cloud/digital_ocean/digital_ocean_sshkey.py: alukovenko mgregson
   $modules/cloud/digital_ocean/digital_ocean_tag.py: kontrafiktion
   $modules/cloud/dimensiondata/dimensiondata_network.py: aimonb tintoy
-  $modules/cloud/docker/docker_container.py: ThomasSteinbach $team_ansible cove dusdanig jctanner kassiansun softzilla zfil
+  $modules/cloud/docker/docker_container.py:
+    ignored: ThomasSteinbach
+    maintainers: $team_ansible cove dusdanig jctanner kassiansun softzilla zfil
   $modules/cloud/docker/docker_image.py: $team_ansible jctanner softzilla
   $modules/cloud/docker/docker_image_facts.py: $team_ansible jctanner
   $modules/cloud/docker/docker_login.py: jctanner olsaki
@@ -213,7 +216,7 @@ files:
   $modules/cloud/misc/helm.py: flaper87
   $modules/cloud/misc/ovirt.py:
     ignored: vincentvdk
-    maintainers: joshainglis karmab
+    maintainers: joshainglis karmab machacekondra mwperina
   $modules/cloud/misc/proxmox.py: UnderGreen
   $modules/cloud/misc/proxmox_kvm.py: helldorado
   $modules/cloud/misc/proxmox_template.py: UnderGreen
@@ -227,7 +230,7 @@ files:
   $modules/cloud/openstack/os_keystone_service.py: $team_openstack SamYaple
   $modules/cloud/openstack/os_project.py: $team_openstack agireud
   $modules/cloud/ovh/ovh_ip_loadbalancing_backend.py: pascalheraud
-  $modules/cloud/ovirt/: machacekondra
+  $modules/cloud/ovirt/: machacekondra mwperina
   $modules/cloud/packet/:
   $modules/cloud/profitbricks/: baldwinSPC
   $modules/cloud/pubnub/pubnub_blocks.py: parfeon pubnub
@@ -268,21 +271,21 @@ files:
   $modules/cloud/vmware/vmware_guest.py: $team_vmware
   $modules/cloud/vmware/vmware_guest_facts.py: nerzhul
   $modules/cloud/vmware/vmware_guest_find.py: $team_vmware
-  $modules/cloud/vmware/vmware_host.py: $team_ansible jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_host.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_local_user_manager.py: $team_ansible
   $modules/cloud/vmware/vmware_maintenancemode.py: $team_ansible
-  $modules/cloud/vmware/vmware_migrate_vmk.py: $team_ansible jcpowermac mtnbikenc
-  $modules/cloud/vmware/vmware_portgroup.py: $team_ansible jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_migrate_vmk.py: $team_ansible jcpowermac
+  $modules/cloud/vmware/vmware_portgroup.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_resource_pool.py: dav1x
   $modules/cloud/vmware/vmware_target_canonical_facts.py: $team_ansible
   $modules/cloud/vmware/vmware_vm_facts.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_vm_shell.py: chrrrles ritzk
   $modules/cloud/vmware/vmware_vm_vss_dvs_migrate.py: $team_ansible jcpowermac
-  $modules/cloud/vmware/vmware_vmkernel.py: $team_ansible jcpowermac mtnbikenc
-  $modules/cloud/vmware/vmware_vmkernel_ip_config.py: $team_ansible jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_vmkernel.py: $team_ansible jcpowermac
+  $modules/cloud/vmware/vmware_vmkernel_ip_config.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_vmotion.py: $team_ansible bedecarroll
-  $modules/cloud/vmware/vmware_vsan_cluster.py: $team_ansible mtnbikenc
-  $modules/cloud/vmware/vmware_vswitch.py: akasurde jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_vsan_cluster.py: $team_ansible
+  $modules/cloud/vmware/vmware_vswitch.py: akasurde jcpowermac
   $modules/cloud/vmware/vsphere_copy.py: dagwieers
   $modules/cloud/vmware/vsphere_guest.py: $team_ansible rhoop
   $modules/cloud/webfaction/: quentinsf
@@ -306,8 +309,7 @@ files:
   $modules/database/misc/riak.py: drewkerrigan jsmartin
   $modules/database/mongodb/mongodb_parameter.py: nerzhul
   $modules/database/mongodb/mongodb_user.py: lujeni
-  $modules/database/mssql/mssql_db.py: Jmainguy vedit
-  $modules/database/mysql/mysql_db.py: $team_ansible
+  $modules/database/mssql/mssql_db.py: Jmainguy vedit kenichi-ogawa-1988
   $modules/database/mysql/mysql_replication.py: banyek
   $modules/database/mysql/mysql_user.py: Jmainguy
   $modules/database/mysql/mysql_variables.py: banyek
@@ -334,21 +336,16 @@ files:
   $modules/files/synchronize.py: $team_ansible tima
   $modules/files/tempfile.py:
   $modules/files/template.py: $team_ansible
-  $modules/files/unarchive.py: dagwieers pileofrogs
+  $modules/files/unarchive.py:
+    labels: unarchive
+    maintainers: pileofrogs
   $modules/files/xattr.py: $team_ansible bcoca
   $modules/files/xml.py:
     ignored: magnus919
+    labels: xml
     maintainers: tbielawa dagwieers sm4rk0 cmprescott
-  $modules/identity/ipa/ipa_dnsrecord.py: Nosmoht akasurde
-  $modules/identity/ipa/ipa_group.py: Nosmoht
-  $modules/identity/ipa/ipa_hbacrule.py: Nosmoht
-  $modules/identity/ipa/ipa_host.py: Nosmoht
-  $modules/identity/ipa/ipa_hostgroup.py: Nosmoht
-  $modules/identity/ipa/ipa_role.py: Nosmoht
-  $modules/identity/ipa/ipa_sudocmd.py: Nosmoht
-  $modules/identity/ipa/ipa_sudocmdgroup.py: Nosmoht
-  $modules/identity/ipa/ipa_sudorule.py: Nosmoht
-  $modules/identity/ipa/ipa_user.py: Nosmoht
+  $modules/identity/ipa/:
+    maintainers: $team_ipa
   $modules/identity/opendj/opendj_backendprop.py: dj-wasabi
   $modules/inventory/add_host.py: $team_ansible
   $modules/inventory/group_by.py: $team_ansible jhoekx
@@ -410,12 +407,16 @@ files:
   $modules/net_tools/omapi_host.py: nerzhul
   $modules/net_tools/snmp_facts.py: ogenstad
   $modules/network/a10/: ericchou1 mischapeters
-  $modules/network/aci/: dagwieers schunduri jmcgill298
+  $modules/network/aci/:
+    labels: aci
+    maintainers: dagwieers schunduri jmcgill298
   $modules/network/aireos/: jmighion
   $modules/network/aos/: dgarros jeremyschulman
   $modules/network/aruba/: jmighion
   $modules/network/asa/: ogenstad
-  $modules/network/avi/: $team_avi
+  $modules/network/avi/:
+    labels: avi
+    maintainers: $team_avi
   $modules/network/bigswitch/: jayakody tedelhourani vuile
   $modules/network/citrix/netscaler.py: $team_ansible
   $modules/network/cloudengine/: QijunPan
@@ -424,7 +425,10 @@ files:
   $modules/network/dellos6/: abirami-n skg-net
   $modules/network/dellos9/: dhivyap skg-net
   $modules/network/eos/: privateip trishnaguha
-  $modules/network/f5/: caphrim007
+  $modules/network/f5/:
+    ignored: Etienne-Carriere mhite mryanlam perzizzle srvg wojtek0806
+    labels: f5
+    maintainers: caphrim007
   $modules/network/fortios/: bjolivot
   $modules/network/illumos/: xen0l
   $modules/network/interface/: $team_networking
@@ -433,7 +437,7 @@ files:
   $modules/network/junos/: Qalthos ganeshrn
   $modules/network/layer2/: $team_networking
   $modules/network/layer3/: $team_networking
-  $modules/network/lenovo/: dkasberg
+  $modules/network/lenovo/: dkasberg amuraleedhar
   $modules/network/netconf/netconf_config.py: ganeshrn lpenz userlerueda
   $modules/network/netscaler/: $team_netscaler
   $modules/network/netvisor/: $team_netvisor
@@ -510,13 +514,13 @@ files:
   $modules/packaging/os/pkgng.py:
     ignored: bleader
     maintainers: JoergFiedler
-  $modules/packaging/os/pkgutil.py: dermute
+  $modules/packaging/os/pkgutil.py: dermute scathatheworm
   $modules/packaging/os/portage.py:
     ignored: sayap
     maintainers: wltjr
   $modules/packaging/os/portinstall.py: berenddeboer
   $modules/packaging/os/pulp_repo.py: sysadmind
-  $modules/packaging/os/redhat_subscription.py: alikins barnabycourt
+  $modules/packaging/os/redhat_subscription.py: alikins kahowell
   $modules/packaging/os/rhn_channel.py: alikins vincentvdk
   $modules/packaging/os/rhn_register.py: $team_rhn
   $modules/packaging/os/rpm_key.py: hacosta
@@ -526,7 +530,7 @@ files:
   $modules/packaging/os/swdepot.py: melodous
   $modules/packaging/os/swupd.py: albertomurillo
   $modules/packaging/os/urpmi.py: pmakowski
-  $modules/packaging/os/yum.py: $team_ansible berenddeschouwer verm666
+  $modules/packaging/os/yum.py: $team_ansible berenddeschouwer verm666 mkrizek
   $modules/packaging/os/yum_repository.py: jtyr
   $modules/packaging/os/zypper.py:
     ignored: dirtyharrycallahan
@@ -540,7 +544,7 @@ files:
   $modules/remote_management/stacki/stacki_host.py: bbyhuy bsanders
   $modules/remote_management/wakeonlan.py: dagwieers
   $modules/source_control/bzr.py: andreparames
-  $modules/source_control/git.py: $team_ansible robinro
+  $modules/source_control/git.py: $team_ansible
   $modules/source_control/git_config.py: djmattyg007
   $modules/source_control/github_deploy_key.py: bincyber
   $modules/source_control/github_hooks.py: pcgentry
@@ -610,7 +614,9 @@ files:
   $modules/system/selinux_permissive.py: mscherer
   $modules/system/seport.py: dankeder
   $modules/system/service.py: $team_ansible
-  $modules/system/setup.py: $team_ansible
+  $modules/system/setup.py:
+    ignored: david_obrien
+    maintainers: $team_ansible
   $modules/system/solaris_zone.py: pmarkham
   $modules/system/svc.py: bcoca
   $modules/system/sysctl.py: akasurde davixx
@@ -639,7 +645,9 @@ files:
   $modules/web_infrastructure/ejabberd_user.py: privateip
   $modules/web_infrastructure/gunicorn.py: agmezr
   $modules/web_infrastructure/htpasswd.py: $team_ansible
-  $modules/web_infrastructure/jboss.py: jhoekx
+  $modules/web_infrastructure/jboss.py:
+    labels: jboss
+    maintainers: jhoekx
   $modules/web_infrastructure/jenkins_job.py: sermilrod
   $modules/web_infrastructure/jenkins_plugin.py: jtyr
   $modules/web_infrastructure/jenkins_script.py: hogarthj
@@ -809,15 +817,13 @@ files:
     - dynamic inventory script
     - dynamic inventory
     - inventory script
-    labels:
-    - "c:inventory/contrib_script"
+    labels: "c:inventory/contrib_script"
     support: community
   contrib/inventory/linode.py:
     keywords:
     - linode dynamic inventory script
     maintainers: intheclouddan rwaweber zbal
-    labels:
-    - cloud
+    labels: cloud
   contrib/inventory/openstack.py:
     keywords:
     - openstack dynamic inventory script
@@ -865,7 +871,9 @@ files:
     labels: networking
   $module_utils/aci.py:
     maintainers: dagwieers schunduri jmcgill298
-    labels: networking
+    labels:
+    - aci
+    - networking
   $module_utils/aireos.py:
     maintainers: jmighion
     labels: networking
@@ -880,7 +888,9 @@ files:
     labels: networking
   $module_utils/avi.py:
     maintainers: $team_avi
-    labels: networking
+    labels:
+    - avi
+    - networking
   $module_utils/bigswitch_utils.py:
     maintainers: jayakody tedelhourani vuile
     labels: networking
@@ -891,13 +901,13 @@ files:
     maintainers: QijunPan
     labels: networking
   $module_utils/cnos_devicerules.py:
-    maintainers: dkasberg
+    maintainers: dkasberg amuraleedhar
     labels: networking
   $module_utils/cnos_errorcodes.py:
-    maintainers: dkasberg
+    maintainers: dkasberg amuraleedhar
     labels: networking
   $module_utils/cnos.py:
-    maintainers: dkasberg
+    maintainers: dkasberg amuraleedhar
     labels: networking
   $module_utils/crypto.py:
     maintainers: Spredzy
@@ -906,7 +916,9 @@ files:
     labels: networking
   $module_utils/f5_utils.py:
     maintainers: caphrim007
-    labels: networking
+    labels:
+    - f5
+    - networking
   $module_utils/fortios.py:
     maintainers: bjolivot
     labels: networking
@@ -950,6 +962,8 @@ files:
   $module_utils/ordnance.py:
     maintainers: alexanderturner djh00t
     labels: networking
+  $module_utils/ovirt.py:
+    maintainers: joshainglis karmab machacekondra
   $module_utils/sros.py:
     maintainers: $team_networking
     labels: networking
@@ -1075,14 +1089,11 @@ files:
   test/sanity/validate-modules:
     keywords:
     - validate-modules
-  test/:
-    labels: test_pull_requests
   docs/:
     labels: docs_pull_request
-    notify:
-      - dharmabumstead
+    notify: dharmabumstead
   packaging/:
-    labels: packaging
+    # 'packaging' automatically added to labels
 macros:
   module_utils: lib/ansible/module_utils
   modules: lib/ansible/modules
@@ -1090,7 +1101,7 @@ macros:
   team_avi: ericsysmin grastogi23 khaltore
   team_azure: haroldwongms nitzmahone tstringer
   team_cumulus: isharacomix jrrivers privateip
-  team_manageiq: gtanzillo abellotti zgalor yaacov
+  team_manageiq: gtanzillo abellotti zgalor yaacov cben
   team_netapp: hulquest lmprice broncofan
   team_netscaler: chiradeep giorgos-nikolopoulos
   team_netvisor: Qalthos amitsi gundalow privateip
@@ -1098,8 +1109,9 @@ macros:
   team_nxos: GGabriele jedelman8 mikewiebe privateip rahushen rcarrillocruz trishnaguha
   team_openstack: emonty j2sol juliakreger rcarrillocruz shrews thingee dagnello
   team_openswitch: Qalthos gundalow privateip
-  team_rabbitmq: chrishoffman manuel-sousa romanek-adam hyperized
+  team_rabbitmq: chrishoffman manuel-sousa hyperized
   team_rhn: alikins barnabycourt flossware vritant
   team_tower: ghjm jlaska matburt wwitzel3
   team_vmware: dagwieers dav1x jctanner nerzhul akasurde
   team_windows: dagwieers jborean93 jhawkesworth
+  team_ipa: Nosmoht akasurde

.github/ISSUE_TEMPLATE.md

@@ -20,8 +20,11 @@ Also test if the latest release, and master branch are affected too.
 
 ##### CONFIGURATION
 <!---
-Mention any settings you have changed/added/removed in ansible.cfg
+If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
+
+Otherwise, mention any settings you have changed/added/removed in ansible.cfg
 (or using the ANSIBLE_* environment variables).
+
 -->
 
 ##### OS / ENVIRONMENT

.gitignore

@@ -27,27 +27,30 @@ docs/man/man3/*
 *.sublime-project
 *.sublime-workspace
 # docsite stuff...
+docs/api/_build/
+docs/api/rst/
+docs/docsite/*.html
+docs/docsite/_build
+docs/docsite/_static/*.gif
+docs/docsite/_static/*.png
+docs/docsite/_static/websupport.js
+docs/docsite/htmlout
+docs/docsite/searchindex.js
+docs/docsite/rst_warnings
+docs/docsite/rst/*_module.rst
+docs/docsite/rst/ansible.rst
+docs/docsite/rst/ansible-*.rst
 docs/docsite/rst/community_maintained.rst
 docs/docsite/rst/config.rst
 docs/docsite/rst/core_maintained.rst
 docs/docsite/rst/list_of_*.rst
-docs/docsite/rst/*_module.rst
 docs/docsite/rst/modules_by_category.rst
 docs/docsite/rst/network_maintained.rst
+docs/docsite/rst/plugins_by_category.rst
 docs/docsite/rst/partner_maintained.rst
-docs/docsite/rst/playbooks_keywords.rst
+docs/docsite/rst/playbook_keywords.rst
 docs/docsite/rst/playbooks_directives.rst
-docs/docsite/*.html
-docs/docsite/rst/ansible.rst
-docs/docsite/rst/ansible-*.rst
-docs/docsite/_static/*.gif
-docs/docsite/_static/*.png
-docs/docsite/_static/websupport.js
-docs/docsite/searchindex.js
-docs/docsite/htmlout
-docs/docsite/_build
-docs/api/rst/
-docs/api/_build/
+docs/docsite/rst/plugins/*/*.rst
 # deb building stuff...
 /debian/
 deb-build
@@ -64,7 +67,6 @@ results.xml
 coverage.xml
 /test/units/cover-html
 /test/integration/targets/*/backup/
-/test/cache/*
 # Development
 /test/develop
 venv
@@ -72,7 +74,8 @@ Vagrantfile
 .vagrant
 ansible.egg-info/
 /shippable/
-/test/integration/cloud-config-azure.yml
+/test/integration/cloud-config-*.*
+!/test/integration/cloud-config-*.*.template
 # Release directory
 packaging/release/ansible_release
 /.cache/

CHANGELOG.md

@@ -1,622 +1,35 @@
 Ansible Changes By Release
 ==========================
 
-<a id="2.4.7"></a>
+<a id="2.5"></a>
 
-## 2.4.7 "Dancing Days" - TBD
+## 2.5 "TBD" - ACTIVE DEVELOPMENT
 
-### Bugfixes
+### Major Changes
+* Removed the previously deprecated 'accelerate' mode and all associated keywords and code.
 
-* The fix for CVE-2018-10875 (https://access.redhat.com/security/cve/cve-2018-10875)
-  prints out a warning message about skipping a config file from a world
-  writable current working directory.  However, if the user explicitly
-  specifies that the config file should be used via the ANSIBLE_CONFIG
-  environment variable then Ansible would honor that but still print out the
-  warning message.  This has been fixed so that Ansible honors the user's
-  explicit wishes and does not print a warning message in that circumstance.
+### Deprecations
 
+#### Deprecated Modules (to be removed in 2.9):
 
-<a id="2.4.6"></a>
+#### Removed Modules (previously deprecated):
+* accelerate
 
-## 2.4.6 "Dancing Days" - 2018-07-05
+### Minor Changes
+* added a few new magic vars corresponding to configuration/command line options:
+  `ansible_diff_mode`, `ansible_inventory_sources`, `ansible_limit`, `ansible_run_tags` , `ansible_forks` and `ansible_skip_tags`
+* Updated the bundled copy of the six library to 1.11.0
 
-### Bugfixes
-* **Security Fix** - avoid loading host/group vars from cwd when not specifying
-  a playbook or playbook base dir (https://github.com/ansible/ansible/pull/42067)
-* **Security Fix** - avoid using ansible.cfg in a world writable dir
-  https://github.com/ansible/ansible/pull/42070
+### New Modules
 
+#### Windows
 
-<a id="2.4.5"></a>
+  * win_scheduled_task_stat
 
-## 2.4.5 "Dancing Days" - 2018-06-21
-
-### Bugfixes
-
-* **Security Fix** - Some connection exceptions would cause no_log specified on
-  a task to be ignored.  If this happened, the task information, including any
-  private information could have been displayed to stdout and (if enabled, not
-  the default) logged to a log file specified in ansible.cfg''s log_path.
-  Additionally, sites which redirected stdout from ansible runs to a log file
-  may have stored that private information onto disk that way as well.
-  (https://github.com/ansible/ansible/pull/41414)
-
-* Fix win_copy to preserve the global Ansible local tmp path instead of
-  deleting it when dealing with multiple files
-  (https://github.com/ansible/ansible/pull/37964)
-* Fix Windows setup.ps1 for slow performance in large domain environments
-  (https://github.com/ansible/ansible/pull/38646)
-* Fix eos_logging idempotency issue when trying to set both logging destination & facility
-  (https://github.com/ansible/ansible/pull/40604)
-* Fix ios_logging idempotency issue when trying to set both logging destination & facility
-  (https://github.com/ansible/ansible/pull/41076)
-
-
-
-<a id="2.4.4"></a>
-
-## 2.4.4 "Dancing Days" - 2018-04-04
-
-### Bugfixes
-* Fix python 3 dictionary runtime error in ios_confg and eos_config
-  (https://github.com/ansible/ansible/issues/36717)
-* Fix `win_script` to work with large arguments and removed uneeded function
-  that produces errors and was not needed
-  (https://github.com/ansible/ansible/pull/33855)
-* Fix timeout when using piped ssh transfer with become
-  https://github.com/ansible/ansible/issues/34523
-* Fix win_scheduled_task docs to correctly reflect what is required and when
-  (https://github.com/ansible/ansible/issues/35072)
-* Updated copy test to create recursive symlink during the test and not have it
-  located in the git repo
-  (https://github.com/ansible/ansible/pull/35073)
-* Fix Digital Ocean tags data type due to backend API changes no longer
-  acceping integers
-  (https://github.com/ansible/ansible/pull/33486)
-* Fix for nxos_vxlan_vtep_vni issues: https://github.com/ansible/ansible/pull/34946
-* Fixes for nxos_bgp: https://github.com/ansible/ansible/pull/34590
-* Enable nxapi nxos_banner test: https://github.com/ansible/ansible/pull/35033
-* fix vxlan idempotent issue in nxos_vxlan_vtep: https://github.com/ansible/ansible/pull/34750
-* Fix win_dns_client to allow setting dynamic IP from static IP
-  (https://github.com/ansible/ansible/pull/35149)
-* Fix azure_rm_subnet absent idempotency issues
-  (https://github.com/ansible/ansible/pull/35037)
-* Fix azure_rm_virtualmachine creating VM with vnet in another resource group
-  (https://github.com/ansible/ansible/pull/35038)
-* Fix nxos terminal plugin regex to support certain commands
-  (https://github.com/ansible/ansible/pull/35186)
-* Fix network os_config modules backward diff
-  (https://github.com/ansible/ansible/pull/35332)
-* Fix nxos_snmp_user removing encryption from user on subsequent runs of the task
-  (https://github.com/ansible/ansible/pull/35433)
-* Fix traceback in winrm module when the ipaddress module is not installed
-  https://github.com/ansible/ansible/pull/35723/files
-* Fix bug in `lineinfile` where the line would not be inserted when using `insertbefore` or `insertafter` if the pattern occured anywhere in the file. (https://github.com/ansible/ansible/issues/28721)
-* Fix connection local getting overridden by network_cli for transport nxapi,eapi for platform agnostic modules
-  (https://github.com/ansible/ansible/pull/35590)
-* Include dest i nthe results from file copy:
-  https://github.com/ansible/ansible/pull/35702/
-* Fix eos_config second-level indent idempotece
-  https://github.com/ansible/ansible/pull/35588
-* Fix the removed_in_version to 2.6 ios_config force option
-  https://github.com/ansible/ansible/pull/35853
-* Fix memory ballooning caused as a result of task caching changes
-  https://github.com/ansible/ansible/pull/35921
-* Fix nxos_igmp_interface for diff nxos versions
-  (https://github.com/ansible/ansible/pull/35959)
-* Fix recursion error with many flat includes
-  (https://github.com/ansible/ansible/pull/36075)
-* Fix win_uri to work with `creates` and `removed` option
-  (https://github.com/ansible/ansible/pull/36016)
-* Fix the oom_killer parameter to docker_container not being honored
-  https://github.com/ansible/ansible/pull/34130
-* Fix docker_service so a build is not triggered every time
-  https://github.com/ansible/ansible/issues/36145
-* Be more tolerant about spaces when gathering virtual facts (https://github.com/ansible/ansible/pull/36042)
-* validate add_host name parameter (https://github.com/ansible/ansible/pull/36055)
-* spelling fixes (https://github.com/ansible/ansible/pull/36007)
-* avoid needles vault prompt on ansible-console (https://github.com/ansible/ansible/pull/36244)
-* fix callback function signatures (https://github.com/ansible/ansible/pull/35664)
-* Clarify error message from convert_bool()
-  https://github.com/ansible/ansible/pull/36041
-* Fix EC2 C5 instance_type fact to be kvm:
-  https://github.com/ansible/ansible/pull/35063
-* Fix templating of loop_control properties:
-  https://github.com/ansible/ansible/pull/36124
-* Fix dependency in the deb package on Ubuntu-12.04:
-  https://github.com/ansible/ansible/pull/36407
-* Fix WinRM Python 3 encoding when getting Kerberos ticket
-  (https://github.com/ansible/ansible/issues/36255)
-* Always show custom prompt in pause module
-  (https://github.com/ansible/ansible/issues/36057)
-* Improve performance and recursion depth in include_role
-  (https://github.com/ansible/ansible/pull/36470)
-* Fix using ansible_*_interpreter on Python3 with non-new-style modules
-  (old-style ansible python modules, modules written in another language, etc)
-  https://github.com/ansible/ansible/pull/36541
-* Fix vyos_config IndexError in sanitize_config
-  (https://github.com/ansible/ansible/issues/36351)
-* Fix vyos_l3_interface multiple address assignment to interfaces
-  (https://github.com/ansible/ansible/pull/36721)
-* Protect from inventory plugins using verify incorrectly
-  https://github.com/ansible/ansible/pull/36591
-* loop control templating
-  https://github.com/ansible/ansible/pull/36124
-* fix debug output
-  https://github.com/ansible/ansible/pull/36307
-* Fix credentials for Ansible Tower modules to work with v1 and v2 of the API
-  (https://github.com/ansible/ansible/pull/36587)
-  (https://github.com/ansible/ansible/pull/36662)
-* Python3 fixes:
-  * Fix for the znode zookeeper module: https://github.com/ansible/ansible/pull/36999
-  * Fix for the maven_artifact module: https://github.com/ansible/ansible/pull/37035
-* Add changes to get docker_container, docker_common, and docker_network
-  working with Docker SDK 3.x: https://github.com/ansible/ansible/pull/36973
-* Ensure we install ansible-config and ansible-inventory with `pip install -e`
-  (https://github.com/ansible/ansible/pull/37151)
-* Fix for unarchive when users use the --strip-components extra_opt to tar
-  causing ansible to set permissions on the wrong directory.
-  https://github.com/ansible/ansible/pull/37048
-* Fix powershell plugin to handle special chars in environment keys as well as
-  int and bool values
-  (https://github.com/ansible/ansible/pull/37215)
-* Fix error messages to not be inappropriately templated:
-  https://github.com/ansible/ansible/pull/37329
-* Fix Python 3 error in the openssl_certificate module:
-  https://github.com/ansible/ansible/pull/35143
-* Fix traceback when creating or stopping ovirt vms
-  (https://github.com/ansible/ansible/pull/37249)
-* Connection error messages may contain characters that jinja2 would
-  interpret as a template.  Wrap the error string so this doesn't happen
-  (https://github.com/ansible/ansible/pull/37329)
-* Fix python3 compatibility bug in wait_for_connection
-  (https://github.com/ansible/ansible/pull/37646)
-* Fix output of the interfaces_file module:
-  https://github.com/ansible/ansible/pull/37818/files
-* Fix haproxy traceback on Python 3 https://github.com/ansible/ansible/pull/35176
-* Fix the csvfile lookup plugin for python3 tracebacks: https://github.com/ansible/ansible/pull/37665
-* Fix ec2 user_data parameter to properly convert to base64 on python3 (https://github.com/ansible/ansible/pull/37628)
-* ansible-pull - fixed a bug checking for changes when we've pulled from the
-  git repository on python3 https://github.com/ansible/ansible/issues/36962
-* Fix digital_ocean module unique_name parameter for a python3 bug:
-  https://github.com/ansible/ansible/issues/37114
-* Fix a Python3 bug in the vagrant dynamic inventory script: https://github.com/ansible/ansible/pull/37631
-* Fix python3 bug in the jira module: https://github.com/ansible/ansible/pull/33862
-* Fix consul module's state=absent: https://github.com/ansible/ansible/issues/34628
-* fix required args for nxos_snapshot: https://github.com/ansible/ansible/pull/37232
-* Properly check that nxos_snapshot parameters that are required in certain circumstances are present:
-  https://github.com/ansible/ansible/pull/37232
-* Fix error messages to not be inappropriately templated:
-  https://github.com/ansible/ansible/pull/37329
-* Fix Python 3 error in the openssl_certificate module:
-  https://github.com/ansible/ansible/pull/35143
-* Fix nxos enable mode
-  https://github.com/ansible/ansible/pull/39898
-
-
-<a id="2.4.3"></a>
-
-## 2.4.3 "Dancing Days" - 2018-01-31
-
-### Bugfixes
-* Fix `pamd` rule args regexp to match file paths (https://github.com/ansible/ansible/pull/33432)
-* Check if SELinux policy exists before setting (https://github.com/ansible/ansible/pull/31834)
-* Set locale to `C` in `letsencrypt` module to fix date parsing errors (https://github.com/ansible/ansible/pull/31339)
-* Fix include in loop when stategy=free
-  (https://github.com/ansible/ansible/pull/33094)
-* Fix save parameter in asa_config (https://github.com/ansible/ansible/pull/32761)
-* Fix --vault-id support in ansible-pull (https://github.com/ansible/ansible/pull/33629)
-* In nxos_interface_ospf, fail nicely if loopback is used with passive_interface:
-  (https://github.com/ansible/ansible/pull/33252)
-* Fix quote filter when given an integer to quote (https://github.com/ansible/ansible/issues/33272)
-* nxos_vrf_interface fix when validating the interface (https://github.com/ansible/ansible/issues/33227)
-* Fix for win_copy when sourcing files from an SMBv1 share
-  (https://github.com/ansible/ansible/pull/33576)
-* correctly report callback plugin file
-* restrict revaulting to vault cli
-  https://github.com/ansible/ansible/pull/33656
-* Fix python3 tracebacks in letsencrypt module (https://github.com/ansible/ansible/pull/32734)
-* Fix ansible_*_interpreter variables to be templated prior to being used:
-  https://github.com/ansible/ansible/pull/33698
-* Fix setting of environment in a task that uses a loop:
-  https://github.com/ansible/ansible/issues/32685
-* Fix fetch on Windows failing to fetch files or particular block size
-  (https://github.com/ansible/ansible/pull/33697)
-* preserve certain fields during no log. https://github.com/ansible/ansible/pull/33637
-* fix issue with order of declaration of sections in ini inventory https://github.com/ansible/ansible/pull/33781
-* Fix win_iis_webapppool to correctly stop a apppool
-  (https://github.com/ansible/ansible/pull/33777)
-* Fix CloudEngine host failed (https://github.com/ansible/ansible/pull/27876)
-* Fix ios_config save issue (https://github.com/ansible/ansible/pull/33791)
-* Handle vault filenames with nonascii chars when displaying messages (https://github.com/ansible/ansible/pull/33926)
-* Fix win_iis_webapppool to not return passwords
-  (https://github.com/ansible/ansible/pull/33931)
-* Fix extended file attributes detection and changing:
-  (https://github.com/ansible/ansible/pull/18731)
-* correctly ensure 'ungrouped' membership rules (https://github.com/ansible/ansible/pull/33878)
-* made warnings less noisy when empty/no inventory is supplied (https://github.com/ansible/ansible/pull/32806)
-* Fixes a failure which prevents to create servers in module cloudscale_server
-* Fix win_firewall_rule "Specified cast is invalid" error when modifying a rule with all of Domain/Public/Private profiles set (https://github.com/ansible/ansible/pull/34383)
-* Fix case for multilib when installing from a file in the yum module
-  (https://github.com/ansible/ansible/pull/32236)
-* Fix WinRM parsing/escaping of IPv6 addresses (https://github.com/ansible/ansible/pull/34072)
-* Fix win_package to detect MSI regardless of the extension case
-  (https://github.com/ansible/ansible/issues/34465)
-* Updated win_mapped_drive docs to clarify what it is used for
-  (https://github.com/ansible/ansible/pull/34478)
-* Fix file related modules run in check_mode when the file being operated on does not exist
-  (https://github.com/ansible/ansible/pull/33967)
-* Make eos_vlan idempotent
-  (https://github.com/ansible/ansible/pull/34443)
-* Fix win_iis_website to properly check attributes before setting
-  (https://github.com/ansible/ansible/pull/34501)
-* Fixed the removal date for ios_config save and force parameters
-  (https://github.com/ansible/ansible/pull/33885)
-* cloudstack: fix timeout from ini config file being ignored
-  https://github.com/ansible/ansible/pull/34854
-* fixes memory usage issues with many blocks/includes
-  https://github.com/ansible/ansible/issues/31673
-  https://github.com/ansible/ansible/pull/34461
-* Fixes maximum recursion depth exceeded with include_role
-  https://github.com/ansible/ansible/issues/23609
-* Fix to win_dns_client module to take ordering of DNS servers to resolve into account:
-  https://github.com/ansible/ansible/pull/34656
-* Fix for the nxos_banner module where some nxos images nest the output inside of an additional dict:
-  https://github.com/ansible/ansible/pull/34695
-* Fix failure message "got multiple values for keyword argument id" in the
-  azure_rm_securitygroup module (caused by changes to the azure python API):
-  https://github.com/ansible/ansible/pull/34810
-* Bump Azure storage client minimum to 1.5.0 to fix deserialization issues. This will break Azure Stack
-  until it receives storage API version 2017-10-01 or changes are made to support multiple versions.
-  (https://github.com/ansible/ansible/pull/34442)
-* Flush stdin when passing the become password.  Fixes some cases of timeout on
-  Python 3 with the ssh connection plugin: https://github.com/ansible/ansible/pull/35049
-
-
-<a id="2.4.2"></a>
-
-## 2.4.2 "Dancing Days" - 2017-11-29
-
-### Bugfixes
-
-* Fix formatting typo in panos_security_rule.py docs. (https://github.com/ansible/ansible/commit/c0fc797a06451d2fe1ac4fc077fc64f3a1666447)
-* Fix rpm spec file to build on RHEL6 without EPEL packages (https://github.com/ansible/ansible/pull/31653)
-* Keep hosts in play vars if inside of a rescue task (https://github.com/ansible/ansible/pull/31710)
-* Fix wait_for module to treat broken connections as unready so that the connection continues to be retried:
-  https://github.com/ansible/ansible/pull/28839
-* Python3 fixes:
-  * windows_azure, clc_firewall_policy, and ce_template modules fixed for
-    imports of urllib which changed between Python2 and Python3 lookup plugin
-    for consul_kv.py fixed for imports of urllib
-    (https://github.com/ansible/ansible/issues/31240)
-  * Make internal hashing of hostvars use bytes on both python2 and python3
-    (https://github.com/ansible/ansible/pull/31788)
-* Fix logging inside of KubernetesAnsibleModule() to not use
-  self.helper.logging.  the Ansible builtin log() method will strip out
-  parameters marked no_log and will not log if no_log was set in the playbook.
-  self.helper.log() circumvents that (https://github.com/ansible/ansible/pull/31789)
-* Correct task results display so that it more closely matches what was present
-  in 2.3.x and previous.
-* Warn when a group has a bad key (Should be one of vars, children, or hosts)
-  https://github.com/ansible/ansible/pull/31495
-* Use controller configured ansible_shell_executable to run commands in the module
-  (https://github.com/ansible/ansible/pull/31361)
-* Add documentation about writing unittests for Ansible
-* Fix bugs in get_url/uri's SNI and TLS version handling when used on systems
-  that have Python-2.7.9+ and urllib3 installed.
-* Have ansible-pull process inventory in its own way.  Fixes issues with
-  ansible-pull not using the correct inventory,  especially for localhost
-  (https://github.com/ansible/ansible/pull/32135)
-* Fix for implicit localhost receiving too many variables from the all group
-  (https://github.com/ansible/ansible/pull/31959)
-* Fix the service module to correctly detect which type of init system is
-  present on the host. (https://github.com/ansible/ansible/pull/32086)
-* Fix inventory patterns to convert to strings before processing:
-  (https://github.com/ansible/ansible/issues/31978)
-* Fix traceback in firewalld module instead of a nice error message:
-  (https://github.com/ansible/ansible/pull/31949)
-* Fix for entering privileged mode using eos network modules:
-  (https://github.com/ansible/ansible/issues/30802)
-* Validate that the destination for ansible-pull is a valid.directory:
-  (https://github.com/ansible/ansible/pull/31499)
-* Document how to preserve strings of digits as strings in the ini inventory:
-  (https://github.com/ansible/ansible/pull/32047)
-* Make sure we return ansible_distribution_major_version to macOS:
-  (https://github.com/ansible/ansible/pull/31708)
-* Fix to ansible-doc -l to list custom inventory plugins:
-  (https://github.com/ansible/ansible/pull/31996)
-* Fix win_chocolatey to respect case sensitivity in URLs:
-  (https://github.com/ansible/ansible/pull/31983)
-* Fix config_format json in the junos_facts module:
-  (https://github.com/ansible/ansible/pull/31818)
-* Allow the apt module's autoremove parameter to take effect in upgrades:
-  (https://github.com/ansible/ansible/pull/30747)
-* When creating a new use via eos_user, create the user before setting the
-  user's privilege level: (https://github.com/ansible/ansible/pull/32162)
-* Fixes nxos_portchannel idempotence failure on N1 images:
-  (https://github.com/ansible/ansible/pull/31057)
-* Remove provider from prepare_ios_tests integration test:
-  (https://github.com/ansible/ansible/pull/31038)
-* Fix nxos_acl change ports to non well known ports and drop time_range for N1:
-  (https://github.com/ansible/ansible/pull/31261)
-* Fix nxos_banner removal idempotence issue in N1 images:
-  (https://github.com/ansible/ansible/pull/31259)
-* Return error message back to the module
-  (https://github.com/ansible/ansible/pull/31035)
-* Fix nxos_igmp_snooping idempotence:
-  (https://github.com/ansible/ansible/pull/31688)
-* NXOS integration test nxos_file_copy, nxos_igmp, nxos_igmp_interface
-  nxos_igmp_snooping, nxos_ntp_auth, nxos_ntp_options:
-  (https://github.com/ansible/ansible/pull/29030)
-* Fix elb_target_group module traceback when ports were specified inside of the targets parameter:
-  (https://github.com/ansible/ansible/pull/32202)
-* Fix creation of empty virtual directories in aws_s3 module:
-  (https://github.com/ansible/ansible/pull/32169)
-* Enable echo for `pause` module: (https://github.com/ansible/ansible/issues/14160)
-* Fix for `hashi_vault` lookup to return all keys at a given path when no key is specified (https://github.com/ansible/ansible/pull/32182)
-* Fix for `win_package` to allow TLS 1.1 and 1.2 on web requests:
-  (https://github.com/ansible/ansible/pull/32184)
-* Remove provider from ios integration test:
-  (https://github.com/ansible/ansible/pull/31037)
-* Fix eos_user tests
-  (https://github.com/ansible/ansible/pull/32261)
-* Fix ansible-galaxy --force with installed roles:
-  (https://github.com/ansible/ansible/pull/32282)
-* ios_interface testfix:
-  (https://github.com/ansible/ansible/pull/32335)
-* Fix ios integration tests:
-  (https://github.com/ansible/ansible/pull/32342)
-* Ensure there is always a basdir so we always pickup group/host_vars
-  https://github.com/ansible/ansible/pull/32269
-* Fix vars placement in ansible-inventory
-  https://github.com/ansible/ansible/pull/32276
-* Correct options for luseradd in user module
-  https://github.com/ansible/ansible/pull/32262
-* Clarified package docs on 'latest' state
-  https://github.com/ansible/ansible/pull/32397
-* Fix issue with user module when local is true
-  (https://github.com/ansible/ansible/pull/32262 and https://github.com/ansible/ansible/pull/32411)
-* Fix for max_fail_percentage being inaccurate:
-  (https://github.com/ansible/ansible/issues/32255)
-* Fix check mode when deleting ACS instance in azure_rm_acs module:
-  (https://github.com/ansible/ansible/pull/32063)
-* Fix ios_logging smaller issues and make default size for buffered work:
-  (https://github.com/ansible/ansible/pull/32321)
-* Fix ios_logging module issue where facility is being deleted along with host:
-  (https://github.com/ansible/ansible/pull/32234)
-* Fix wrong prompt issue for network modules (https://github.com/ansible/ansible/pull/32426)
-* Fix eos_eapi to enable non-default vrfs if the default vrf is already configured (https://github.com/ansible/ansible/pull/32112)
-* Fix network parse_cli filter in case of single match is not caught when using start_block and end_block
-  (https://github.com/ansible/ansible/pull/31092)
-* Fix win_find failing on files it can't access, change behaviour to be more
-  like the find module (https://github.com/ansible/ansible/issues/31898)
-* Amended tracking of 'changed'
-  https://github.com/ansible/ansible/pull/31812
-* Fix label assignment in ovirt_host_networks
-  (https://github.com/ansible/ansible/pull/31973)
-* Fix fencing and kuma usage in ovirt_cluster module
-  (https://github.com/ansible/ansible/pull/32190)
-* Fix failure during upgrade due to NON_RESPONSIVE state for ovirt_hosts module
-  (https://github.com/ansible/ansible/pull/32192)
-* ini inventory format now correclty handles group creation w/o need for specific orders
-  https://github.com/ansible/ansible/pull/32471
-* Fix for quoted paths in win_service
-  (https://github.com/ansible/ansible/issues/32368)
-* Fix tracebacks for non-ascii paths when parsing inventory
-  (https://github.com/ansible/ansible/pull/32511)
-* Fix git archive when update is set to no
-  (https://github.com/ansible/ansible/pull/31829)
-* Fix locale when screen scraping in the yum module
-  (https://github.com/ansible/ansible/pull/32203)
-* Fix for validating proxy results on Python3 for modules making http requests:
-  (https://github.com/ansible/ansible/pull/32596)
-* Fix unreferenced variable in SNS topic module
-  (https://github.com/ansible/ansible/pull/29117)
-* Handle ignore_errors in loops
-  (https://github.com/ansible/ansible/pull/32546)
-* Fix running with closed stdin on python 3
-  (https://github.com/ansible/ansible/pull/31695)
-* Fix undefined variable in script inventory plugin
-  (https://github.com/ansible/ansible/pull/31381)
-* Fix win_copy on Python 2.x to support files greater than 4GB
-  (https://github.com/ansible/ansible/pull/32682)
-* Add extra error handling for wmare connect to correctly detect scenarios where
-  username does not have the required logon permissions
-  (https://github.com/ansible/ansible/pull/32613)
-* Fix ios_config file prompt issue while using save_when
-  (https://github.com/ansible/ansible/pull/32744)
-* Prevent host_group_vars plugin load errors when using 'path as inventory hostname'
-  https://github.com/ansible/ansible/issues/32764
-* Better errors when loading malformed vault envelopes
-  (https://github.com/ansible/ansible/issues/28038)
-* nxos_interface error handling
-  (https://github.com/ansible/ansible/pull/32846)
-* Fix snmp bugs on Nexus 3500 platform
-  (https://github.com/ansible/ansible/pull/32773)
-* nxos_config and nxos_facts - fixes for N35 platform
-  (https://github.com/ansible/ansible/pull/32762)
-* fix dci failure nxos
-  (https://github.com/ansible/ansible/pull/32877)
-* Do not execute `script` tasks is check mode (https://github.com/ansible/ansible/issues/30676)
-* Keep newlines when reading LXC container config file (https://github.com/ansible/ansible/pull/32219)
-* Fix a traceback in os_floating_ip when required instance is already present in the cloud:
-  https://github.com/ansible/ansible/pull/32887
-* Fix for modifying existing application load balancers using certificates (https://github.com/ansible/ansible/pull/28217)
-* Fix --ask-vault-pass with no tty and password from stdin
-  (https://github.com/ansible/ansible/issues/30993)
-* Fix for IIS windows modules to use hashtables instead of PSCustomObject
-  (https://github.com/ansible/ansible/pull/32710)
-* Fix nxos_snmp_host bug
-  (https://github.com/ansible/ansible/pull/32916)
-* Make IOS devices consistent ios_logging
-  (https://github.com/ansible/ansible/pull/33100)
-* restore error on orphan group:vars delcaration for ini inventories
-  https://github.com/ansible/ansible/pull/32866
-* restore host/group_vars merge order
-  https://github.com/ansible/ansible/pull/32963
-* use correct loop var when delegating
-  https://github.com/ansible/ansible/pull/32986
-* Handle sets and datetime objects in inventory sources fixing tracebacks
-  https://github.com/ansible/ansible/pull/32990
-* Fix for breaking change to Azure Python SDK DNS RecordSet constructor in azure-mgmt-dns==1.2.0
-  https://github.com/ansible/ansible/pull/33165
-* Fix for breaking change to Azure Python SDK that prevented some members from being returned in facts modules
-  https://github.com/ansible/ansible/pull/33169
-* restored glob/regex host pattern matching to traverse groups and hosts and not return after first found
-  https://github.com/ansible/ansible/pull/33158
-* change nxos_interface module to use "show interface" to support more platforms
-  https://github.com/ansible/ansible/pull/33037
-
-
-<a id="2.4.1"></a>
-
-## 2.4.1 "Dancing Days" - 2017-10-25
-
-### Bugfixes
-
-* Security fix for CVE-2017-7550 the jenkins_plugin module was logging the jenkins
-  server password if the url_password was passed via the params field:
-  https://github.com/ansible/ansible/pull/30875
-* Update openssl\* module documentation to show openssl-0.16 is the minimum version
-* Fix openssl_certificate's csr handling
-* Python-3 fixes
-  * Fix openssl_certificate parameter assertion on Python3
-  * Fix for python3 and nonascii strings in inventory plugins (https://github.com/ansible/ansible/pull/30666)
-  * Fix missing urllib in iam_policy
-  * Fix crypttab module for bytes<=>text string mismatch ( https://github.com/ansible/ansible/pull/30457 )
-  * Fix lxc_container module combining bytes with text ( https://github.com/ansible/ansible/pull/30572 )
-  * Fix map doesn't return a list on python3 in ec2_snapshot_facts module (https://github.com/ansible/ansible/pull/30606)
-  * Fix uri (and other url retrieving) modules when used with a proxy. (https://github.com/ansible/ansible/issues/31109)
-  * Fix azure_rm dynamic inventory script ConfigParser usage.
-* Fix for win_file to respect check mode when deleting directories
-* Fix for Ansible.ModuleUtils.Legacy.psm1 to return list params correctly
-* Fix for a proper logout in the module ovirt_vms
-* Fixed docs for 'password' lookup
-* Corrected and added missing feature and porting docs for 2.4
-* Fix for Ansible.ModuleUtils.CamelConversion to handle empty lists and lists with one entry
-* Fix nxos terminal regex to parse username correctly.
-* Fix colors for selective callback
-* Fix for 'New password' prompt on 'ansible-vault edit' (https://github.com/ansible/ansible/issues/30491)
-* Fix for 'ansible-vault encrypt' with vault_password_file in config and --ask-vault-pass cli (https://github.com/ansible/ansible/pull/30514#pullrequestreview-63395903)
-* updated porting guide with notes for callbacks and config
-* Added backwards compatiblity shim for callbacks that do not inherit from CallbackBase
-* Corrected issue with configuration and multiple ini entries being overwriten even when not set.
-* backported fix for doc generation (plugin_formatter)
-* Fix ec2_lc module for an unknown parameter name (https://github.com/ansible/ansible/pull/30573)
-* Change configuration of defaults to use standard jinja2 instead of custom
-  eval() for using variables in the default field of config (https://github.com/ansible/ansible/pull/30650)
-* added missing entry in chlog deprecation
-* Fixed precedence and values for become flags and executable settings
-* Fix for win_domain_membership to throw more helpful error messages and check/fix when calling WMI function after changing workgroup
-* Fix for win_power_plan to compare the OS version's correctly and work on Windows 10/Server 2016
-* Fix module doc for typo in telnet command option
-* Fix OpenBSD pkg_mgr fact (https://github.com/ansible/ansible/issues/30623)
-* Fix encoding error when there are nonascii values in the path to the ssh binary
-* removed YAML inventory group name validation, broke existing setups and should be global in any case, and configurable
-* performance improvment for inventory, had slown down considerably from 2.3
-* Fix cpu facts on sparc64 (https://github.com/ansible/ansible/pull/30261)
-* Fix ansible_distribution fact for Arch linux (https://github.com/ansible/ansible/issues/30600)
-* remove print statements from play_context/become
-* Fix vault errors after 'ansible-vault edit' (https://github.com/ansible/ansible/issues/30575)
-* updated api doc example to match api changes
-* corrected issues with slack callback plugin
-* it is import_playbook .. not import_plays .. docs now reflect this
-* fixed typo and missed include/import conversion in import_tasks docs
-* updated porting docs with note about inventory_dir
-* removed extension requirement for yaml inventory plugin to restore previous behaviour
-* fixed ansible-pull to now correctly deal with inventory
-* corrected dig lookup docs
-* fix type handling for sensu_silence so the module works
-* added fix for win_iis_webapppool to correctly handle array elements
-* Fix bugs caused by lack of collector ordering like service_mgr being incorrect (https://github.com/ansible/ansible/issues/30753)
-* Fix os_image when the id parameter is not set in the task. ( https://github.com/ansible/ansible/pull/29147 )
-* Fix for the winrm connection to use proper task vars
-* removed typo from dig lookup docs
-* Updated win_chocolatey example to be clearer around what should be used with become
-* Fix for copy module when permissions are changed but the file contents are not ( https://github.com/ansible/ansible/issues/30556 )
-* corrected YAML_FILENAME_EXTENSIONS ini setter as key/section were swapped
-* Better error message when a yaml inventory is invalid
-* avoid include_Xs conflating vars with options
-* Fix aws_s3 module handling `encrypt` option (https://github.com/ansible/ansible/pull/31203)
-* Fix for win_msg to document and show error when message is greater than 255 characters
-* Fix for win_dotnet_ngen to work after recent regression
-* fixed backwards compat method for config
-* removed docs for prematurely added ssh specific pipelining settings
-* fixed redis cache typo
-* Fix AttributeError during inventory group deserialization (https://github.com/ansible/ansible/issues/30903)
-* Fix 'ansible-vault encrypt --output=-' (https://github.com/ansible/ansible/issues/30550)
-* restore pre 2.4 pipeline configuration options (env and ini)
-* Fix win_copy regression: handling of vault-encrypted source files (https://github.com/ansible/ansible/pull/31084)
-* Updated return values for win_reg_stat to correctly show what is being returned (https://github.com/ansible/ansible/pull/31252)
-* reduced normal error redundancy and verbosity, display on increased and when needed
-* Give an informative error instead of a traceback if include_vars dir is file instead of directory (https://github.com/ansible/ansible/pull/31157)
-* Fix monit module's version check for color support (https://github.com/ansible/ansible/pull/31212)
-* Make `elasticsearch_plugin` module work with both 2.x and 5.x (https://github.com/ansible/ansible/issues/21989)
-* Fix for become on Windows to handle ignored errors (https://github.com/ansible/ansible/issues/30468)
-* Fix removal of newlines when writing SELinux config (https://github.com/ansible/ansible/issues/30618)
-* clarified extension requirement for constructed inv plugin
-* really turn off inventory caching, toggle will be added in 2.5
-* for inventory sources, dont follow symlinks to calculate base directory, used for group/host_vars
-* Port the uptime.py example script to the new inventory API.
-* inventory_file variable again returns full path, not just basename
-* added info about cwd group/host vars to porting guide
-* Fix name parsing out of envra in the yum module
-* give user friendly error on badly formatted yaml inventory source
-* Fix any_errors_fatal setting in playbooks.
-* Fix setting of ssh-extra-args from the cli (https://github.com/ansible/ansible/pull/31326)
-* Change SELinux fact behavior to always return a dictionary. (https://github.com/ansible/ansible/issues/18692)
-* Revert a fix for using non /bin/sh shells for modules' running commands as
-  this was causing output from commands to change, thus breaking playbooks.
-  See the original bug for details and links to the eventual fix:
-  https://github.com/ansible/ansible/issues/24169
-* Do not log data field in `docker_secrets` module (https://github.com/ansible/ansible/pull/31366)
-* Fix rpm_key taking the wrong 8 chars from the keyid (https://github.com/ansible/ansible/pull/31045)
-* chown errors now more informative
-* Fix for win_copy to copy a source file that has invalid windows characters in the filename, the dest still must be have valid windows characters (https://github.com/ansible/ansible/issues/31336#issuecomment-334649927)
-* Fix systemd module to not run daemon-reload in check mode.
-* fixed some parsing and selection issues with inventory manager, fixed minor bugs in yaml and constructed plugins
-* Fix the ping module documentation to reference win_ping instead of itself: https://github.com/ansible/ansible/pull/31444
-* Fix for ec2_win_password to allow blank key_passphrase again (https://github.com/ansible/ansible/pull/28791)
-* added toggle for vars_plugin behaviour to execute relative to playbook, set default to revert to previous way.
-* Fix for win_copy to not remove destination file on change when in check mode (https://github.com/ansible/ansible/pull/31469)
-* Fix include_role usage of role_name (https://github.com/ansible/ansible/pull/31463)
-* Fix service and package forcing a second run of the setup module to function (https://github.com/ansible/ansible/issues/31485)
-* Better error message when attempting to use include or import with /usr/bin/ansible (https://github.com/ansible/ansible/pull/31492/)
-* Fix `sysctl` module to remove etries when `state=absent` (https://github.com/ansible/ansible/issues/29920)
-* Fix for ec2_group to avoid trying to iterate over None (https://github.com/ansible/ansible/pull/31531)
-* Fix for ec2_group for a possible KeyError bug (https://github.com/ansible/ansible/pull/31540)
-* Fix for the rpm_key module when importing the first gpg key on a system (https://github.com/ansible/ansible/pull/31514)
-* Fix for aws_s3 metadata to use the correct parameters when uploading a file (https://github.com/ansible/ansible/issues/31232)
-* Fix for the yum module when installing from file/url crashes (https://github.com/ansible/ansible/pull/31529)
-* Improved error messaging for Windows become/runas when username is bogus (https://github.com/ansible/ansible/pull/31551)
-* Fix rollback feature in junos_config to now allow configuration rollback on device (https://github.com/ansible/ansible/pull/31424)
-* Remove command executed log from ansible-connection (https://github.com/ansible/ansible/pull/31581)
-* Fix relative paths to be relative to config file when there is no playbook available (https://github.com/ansible/ansible/issues/31533)
-* Fix Inventory plugins to use the configured inventory plugin path (https://github.com/ansible/ansible/issues/31605)
-* Fix include task to be dynamic (https://github.com/ansible/ansible/issues/31593)
-* A couple fixes to the test process to account for new testing resources in
-  our ci system and an upstream cryptography update that didn't work with
-  pip-8.x
-* Document backup_path in a few dellos modules and vyos_config (https://github.com/ansible/ansible/issues/31844)
-* Fix for vmware_vm_facts with dangling inaccessible VM which don't have MAC addresses (https://github.com/ansible/ansible/pull/31629)
-* Fix for win_regedit sending extra data that could confuse ansible's result parsing (https://github.com/ansible/ansible/pull/31813)
-* Fix git module to correctly cleanup temporary dirs (https://github.com/ansible/ansible/pull/31541)
-* Fix for modules which use atomic_move() to rename files raising an exception
-  if a file could not be opened.  Fix will return a nice error message instead:
-  https://github.com/ansible/ansible/issues/31786
-* Fix ansible-doc and ansible-console module-path option (https://github.com/ansible/ansible/pull/31744)
-* Fix for hostname module on RHEL 7.5 (https://github.com/ansible/ansible/issues/31811)
-* Fix provider password leak in logs for asa modules (https://github.com/ansible/ansible/issues/32343)
-* Fix tagging for dynamodb_table if region is not explicitly passed to the module (https://github.com/ansible/ansible/pull/32557)
-* Fix Python 3 decode error in `cloudflare_dns` (https://github.com/ansible/ansible/pull/32065)
-
-### Known Bugs
-* Implicit localhost is getting ansible_connection from all:vars instead of
-  from the implicit localhost definition (https://github.com/ansible/ansible/issues/31420)
 
 <a id="2.4"></a>
 
-## 2.4 "Dancing Days" - 2017-09-19
+## 2.4 "Dancing Days" - 2017/09/18
 
 ### Major Changes
 
@@ -648,8 +61,6 @@ Ansible Changes By Release
 * Windows modules now support the use of multiple shared module_utils files in the form of Powershell modules (.psm1), via `#Requires -Module Ansible.ModuleUtils.Whatever.psm1`
 * Python module argument_spec now supports custom validation logic by accepting a callable as the `type` argument.
 * Windows become_method: runas now works across all authtypes and will auto-elevate under UAC if WinRM user has "Act as part of the operating system" privilege
-* Do not escape backslashes in the template lookup plugin to mirror what the template module does
-  https://github.com/ansible/ansible/issues/26397
 
 ### Deprecations
 * The behaviour when specifying `--tags` (or `--skip-tags`) multiple times on the command line
@@ -664,8 +75,7 @@ Ansible Changes By Release
   moved to `ansible.utils.unsafe_proxy` to avoid a circular import.
 * The win_get_url module has the dictionary 'win_get_url' in its results deprecated,
   its content is now also available directly in the resulting output, like other modules.
-* previouslly deprecated 'hostfile' config settings have been 're-deprecated' as before the code did not warn about deprecated configuration settings
-, but it does now.
+* previouslly deprecated 'hostfile' config settings have been 're-deprecated' as before the code did not warn about deprecated configuration settings, but it does now.
 
 #### Deprecated Modules (to be removed in 2.8):
 * azure: use M(azure_rm_virtualmachine) instead
@@ -679,7 +89,7 @@ Ansible Changes By Release
 * s3: replaced by aws_s3
 * win_msi: use M(win_package) instead
 
-#### Removed Modules (previously deprecated):
+#### Removed Modules (previouslly deprecated):
 * eos_template: use eos_config instead
 * ios_template: use ios_config instead
 * iosxr_template: use iosxr_config instead
@@ -693,7 +103,7 @@ Ansible Changes By Release
 * Now deprecated configuration options issue warnings when set.
 * Removed unused and deprecated config option `pattern`
 * Updated the copy of six bundled for modules to use from 1.4.1 to 1.10.0
-* The `inventory_dir` var is not a global anymore, as we now allow multiple inventory sources, it is now host dependant.
+* The `include_dir` var is not a global anymore, as we now allow multiple inventory sources, it is now host dependant.
   This means it cannot be used wherever host vars are not permitted, for example in task/handler names.
 * Fixed a cornercase with ini inventory vars.  Previously, if an inventory var
   was a quoted string with hash marks ("#") in it then the parsed string
@@ -759,40 +169,15 @@ Ansible Changes By Release
 - profile_roles
 - stderr
 
-#### New Connection plugins:
-- buildah
-- saltstack
-
 #### New Filters:
-- ipaddr filter gained several new suboptions
-  - first_usable
-  - ip/prefix
-  - ip_netmask
-  - last_usable
-  - next_usable
-  - network_id
-  - network/prefix
-  - network_netmask
-  - network_wildcard
-  - previous_usable
-  - range_usable
-  - size_usable
-  - wildcard
-- next_nth_usable
-- network_in_network
-- network_in_usable
-- previous_nth_usable
 - parse_cli
 - parse_cli_textfsm
-- strftime
-- urlsplit
 
 #### New Inventory Plugins:
 - advanced_host_list
 - constructed
 - host_list
 - ini
-- openstack
 - script
 - virtualbox
 - yaml
@@ -800,22 +185,11 @@ Ansible Changes By Release
 #### New Inventory scripts:
 - lxd
 
-#### New Lookups:
-- chef_databag
-- cyberarkpassword
-- hiera
-
-#### New Tests:
+#### New: Tests
 - any : true if any element is true
 - all: true if all elements are true
 
 ### Module Notes
-- By mistake, an early version of elb_classic_lb, elb_instance, and elb_classic_lb_facts modules
-  were released and marked as stableinterface.  These are now marked as preview in 2.4.1 and their
-  parameters and return values may change in 2.5.0.  Part of this mistake included deprecating the
-  ec2_elb_lb, ec2_lb, and ec2_elb_facts modules prematurely.  These modules won't be deprecated
-  until the replacements above have a stableinterface and the erroneous deprecation has been fixed
-  in 2.4.1.
 - The docker_container module has gained a new option, `working_dir` which allows
   specifying the working directory for the command being run in the image.
 - The ec2_win_password module now requires the cryptography python module be installed to run
@@ -1244,6 +618,7 @@ Ansible Changes By Release
 ### Bugfixes
 * Fix alternatives module handlling of non existing options
 * Fix synchronize traceback with the docker connection plugin
+* Do not escape backslashes in the template lookup plugin to mirror what the template module does
 * Fix the expires option of the postgresq_user module
 * Fix for win_acl when settings permissions on registry objects that use `ALL APPLICATION PACKAGES` and `ALL RESTRICTED APPLICATION PACKAGES`
 * Python3 fixes

MODULE_GUIDELINES.md

@@ -47,9 +47,9 @@ A detailed explanation of the PR workflow can be seen here: https://github.com/a
 
 ***
 
-# Extras maintainers list
+# Maintainers list
 
-The full list of maintainers for modules is located here: https://github.com/ansible/ansibullbot/blob/master/MAINTAINERS.txt
+The full list of maintainers for modules is located here: https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml
 
 ## Changing Maintainership
 

Makefile

@@ -186,7 +186,7 @@ clean:
 	rm -rf logs/
 	rm -rf .cache/
 	rm -f test/units/.coverage*
-	rm -f test/results/*/*
+	rm -rf test/results/*/*
 	find test/ -type f -name '*.retry' -delete
 	@echo "Cleaning up RPM building stuff"
 	rm -rf MANIFEST rpm-build

README.md

@@ -9,9 +9,7 @@ Ansible is a radically simple IT automation system.  It handles configuration-ma
 
 Read the documentation and more at https://ansible.com/
 
-Many users run straight from the development branch (it's generally fine to do so), but you might also wish to consume a release.
-
-You can find instructions [here](https://docs.ansible.com/ansible/intro_installation.html) for a variety of platforms.
+You can find installation instructions [here](https://docs.ansible.com/ansible/intro_installation.html) for a variety of platforms. Most users should probably install a released version of Ansible from `pip`, a package manager or our [release repository](https://releases.ansible.com/ansible/). [Officially supported](https://www.ansible.com/ansible-engine) builds of Ansible are also available. Some power users run directly from the development branch - while significant efforts are made to ensure that `devel` is reasonably stable, you're more likely to encounter breaking changes when running Ansible this way.
 
 Design Principles
 =================
@@ -41,7 +39,6 @@ Branch Info
 
    * Releases are named after Led Zeppelin songs. (Releases prior to 2.0 were named after Van Halen songs.)
    * The devel branch corresponds to the release actively under development.
-   * For releases 1.8 - 2.2, modules are kept in different repos, you'll want to follow [core](https://github.com/ansible/ansible-modules-core) and [extras](https://github.com/ansible/ansible-modules-extras)
    * Various release-X.Y branches exist for previous releases.
    * We'd love to have your contributions, read [Community Information](https://docs.ansible.com/community.html) for notes on how to get started.
 
@@ -52,9 +49,8 @@ Ansible was created by [Michael DeHaan](https://github.com/mpdehaan) (michael.de
 
 Ansible is sponsored by [Ansible, Inc](https://ansible.com)
 
-Licence
+License
 =======
-GNU
-Click on the [Link](COPYING) to see the full text.
-
+GNU General Public License v3.0
 
+See [COPYING](COPYING) to see the full text.

RELEASES.txt

@@ -4,93 +4,80 @@ Ansible Releases at a Glance
 VERSION  RELEASE     CODE NAME
 ++++++++++++++++++++++++++++++
 
-2.6.0    TBD         "HeartBreaker"
-2.5.5    14-06-2018  "Kashmir"
-2.5.4    31-05-2018  "Kashmir"
-2.5.3    17-05-2018  "Kashmir"
-2.5.2    26-04-2018  "Kashmir"
-2.5.1    19-04-2018  "Kashmir"
-2.5.0    23-03-2018  "Kashmir"
-2.4.6    07-05-2018  "Dancing Days"
-2.4.5    06-21-2018  "Dancing Days"
-2.4.4    04-04-2018  "Dancing Days"
-2.4.3    01-31.2018  "Dancing Days"
-2.4.2    11-29-2017  "Dancing Days"
-2.4.1    10-25-2017  "Dancing Days"
-2.4.0    09-19-2017  "Dancing Days"
-2.3.2    08-04-2017  "Ramble On"
-2.3.1    06-01-2017  "Ramble On"
-2.3.0    04-12-2017  "Ramble On"
-2.2.3    05-09-2017  "The Battle of Evermore"
-2.2.2    03-27-2017  "The Battle of Evermore"
-2.2.1    01-16-2017  "The Battle of Evermore"
-2.2.0    11-01-2016  "The Battle of Evermore"
-2.1.6    06-01-2017  "The Song Remains the Same"
-2.1.5    03-27-2017  "The Song Remains the Same"
-2.1.4    01-16-2017  "The Song Remains the Same"
-2.1.3    11-04-2016  "The Song Remains the Same"
-2.1.2    09-29-2016  "The Song Remains the Same"
-2.1.1    07-28-2016  "The Song Remains the Same"
-2.1.0    05-25-2016  "The Song Remains the Same"
-2.0.2    04-19-2016  "Over the Hills and Far Away"
-2.0.1    02-24-2016  "Over the Hills and Far Away"
-2.0.0    01-12-2016  "Over the Hills and Far Away"
-1.9.6    04-15-2016  "Dancing In the Streets"
-1.9.5    03-21-2016  "Dancing In the Streets"
-1.9.4    10-09-2015  "Dancing In the Streets"
-1.9.3    09-03-2015  "Dancing In the Streets"
-1.9.2    06-24-2015  "Dancing In the Streets"
-1.9.1    04-27-2015  "Dancing In the Streets"
-1.9.0    03-25-2015  "Dancing In the Streets"
-1.8.4    02-19-2015  "You Really Got Me"
-1.8.3    02-17-2015  "You Really Got Me"
-1.8.2    12-04-2014  "You Really Got Me"
-1.8.1    11-26-2014  "You Really Got Me"
-1.8.0    11-25-2014  "You Really Got Me"
-1.7.2    09-24-2014  "Summer Nights"
-1.7.1    08-14-2014  "Summer Nights"
-1.7.0    08-06-2014  "Summer Nights"
-1.6.10   07-25-2014  "The Cradle Will Rock"
-1.6.9    07-24-2014  "The Cradle Will Rock"
-1.6.8    07-22-2014  "The Cradle Will Rock"
-1.6.7    07-21-2014  "The Cradle Will Rock"
-1.6.6    07-01-2014  "The Cradle Will Rock"
-1.6.5    06-25-2014  "The Cradle Will Rock"
-1.6.4    06-25-2014  "The Cradle Will Rock"
-1.6.3    06-09-2014  "The Cradle Will Rock"
-1.6.2    05-23-2014  "The Cradle Will Rock"
-1.6.1    05-07-2014  "The Cradle Will Rock"
-1.6.0    05-05-2014  "The Cradle Will Rock"
-1.5.5    04-18-2014  "Love Walks In"
-1.5.4    04-01-2014  "Love Walks In"
-1.5.3    03-13-2014  "Love Walks In"
-1.5.2    03-11-2014  "Love Walks In"
-1.5.1    03-10-2014  "Love Walks In"
-1.5.0    02-28-2014  "Love Walks In"
-1.4.5    02-12-2014  "Could This Be Magic?"
-1.4.4    01-06-2014  "Could This Be Magic?"
-1.4.3    12-20-2013  "Could This Be Magic?"
-1.4.2    12-18-2013  "Could This Be Magic?"
-1.4.1    11-27-2013  "Could This Be Magic?"
-1.4.0    11-21-2013  "Could This Be Magic?"
-1.3.4    10-29-2013  "Top of the World"
-1.3.3    10-09-2013  "Top of the World"
-1.3.2    09-19-2013  "Top of the World"
-1.3.1    09-16-2013  "Top of the World"
-1.3.0    09-13-2013  "Top of the World"
-1.2.3    08-21-2013  "Right Now"
-1.2.2    07-05-2013  "Right Now"
-1.2.1    07-04-2013  "Right Now"
-1.2.0    06-10-2013  "Right Now"
-1.1.0    04-02-2013  "Mean Street"
-1.0.0    02-01-2013  "Eruption"
-0.9.0    11-30-2012  "Dreams"
-0.8.0    10-19-2012  "Cathedral"
-0.7.0    09-06-2012  "Panama"
-0.6.0    08-06-2012  "Cabo"
-0.5.0    07-04-2012  "Amsterdam"
-0.4.0    05-23-2012  "Unchained"
-0.3.0    04-23-2012  "Baluchitherium"
+2.4.0    TBD         "Dancing Days"
+2.3.2    2017-08-04  "Ramble On"
+2.3.1    2017-06-01  "Ramble On"
+2.3.0    2017-04-12  "Ramble On"
+2.2.3    2017-05-09  "The Battle of Evermore"
+2.2.2    2017-03-27  "The Battle of Evermore"
+2.2.1    2017-01-16  "The Battle of Evermore"
+2.2.0    2016-11-01  "The Battle of Evermore"
+2.1.6    2017-06-01  "The Song Remains the Same"
+2.1.5    2017-03-27  "The Song Remains the Same"
+2.1.4    2017-01-16  "The Song Remains the Same"
+2.1.3    2016-11-04  "The Song Remains the Same"
+2.1.2    2016-09-29  "The Song Remains the Same"
+2.1.1    2016-07-28  "The Song Remains the Same"
+2.1.0    2016-05-25  "The Song Remains the Same"
+2.0.2    2016-04-19  "Over the Hills and Far Away"
+2.0.1    2016-02-24  "Over the Hills and Far Away"
+2.0.0    2016-01-12  "Over the Hills and Far Away"
+1.9.6    2016-04-15  "Dancing In the Streets"
+1.9.5    2016-03-21  "Dancing In the Streets"
+1.9.4    2015-10-09  "Dancing In the Streets"
+1.9.3    2015-09-03  "Dancing In the Streets"
+1.9.2    2015-06-24  "Dancing In the Streets"
+1.9.1    2015-04-27  "Dancing In the Streets"
+1.9.0    2015-03-25  "Dancing In the Streets"
+1.8.4    2015-02-19  "You Really Got Me"
+1.8.3    2015-02-17  "You Really Got Me"
+1.8.2    2014-12-04  "You Really Got Me"
+1.8.1    2014-11-26  "You Really Got Me"
+1.8.0    2014-11-25  "You Really Got Me"
+1.7.2    2014-09-24  "Summer Nights"
+1.7.1    2014-08-14  "Summer Nights"
+1.7.0    2014-08-06  "Summer Nights"
+1.6.10   2014-07-25  "The Cradle Will Rock"
+1.6.9    2014-07-24  "The Cradle Will Rock"
+1.6.8    2014-07-22  "The Cradle Will Rock"
+1.6.7    2014-07-21  "The Cradle Will Rock"
+1.6.6    2014-07-01  "The Cradle Will Rock"
+1.6.5    2014-06-25  "The Cradle Will Rock"
+1.6.4    2014-06-25  "The Cradle Will Rock"
+1.6.3    2014-06-09  "The Cradle Will Rock"
+1.6.2    2014-05-23  "The Cradle Will Rock"
+1.6.1    2014-05-07  "The Cradle Will Rock"
+1.6.0    2014-05-05  "The Cradle Will Rock"
+1.5.5    2014-04-18  "Love Walks In"
+1.5.4    2014-04-01  "Love Walks In"
+1.5.3    2014-03-13  "Love Walks In"
+1.5.2    2014-03-11  "Love Walks In"
+1.5.1    2014-03-10  "Love Walks In"
+1.5.0    2014-02-28  "Love Walks In"
+1.4.5    2014-02-12  "Could This Be Magic?"
+1.4.4    2014-01-06  "Could This Be Magic?"
+1.4.3    2013-12-20  "Could This Be Magic?"
+1.4.2    2013-12-18  "Could This Be Magic?"
+1.4.1    2013-11-27  "Could This Be Magic?"
+1.4.0    2013-11-21  "Could This Be Magic?"
+1.3.4    2013-10-29  "Top of the World"
+1.3.3    2013-10-09  "Top of the World"
+1.3.2    2013-09-19  "Top of the World"
+1.3.1    2013-09-16  "Top of the World"
+1.3.0    2013-09-13  "Top of the World"
+1.2.3    2013-08-21  "Right Now"
+1.2.2    2013-07-05  "Right Now"
+1.2.1    2013-07-04  "Right Now"
+1.2.0    2013-06-10  "Right Now"
+1.1.0    2013-04-02  "Mean Street"
+1.0.0    2013-02-01  "Eruption"
+0.9.0    2012-11-30  "Dreams"
+0.8.0    2012-10-19  "Cathedral"
+0.7.0    2012-09-06  "Panama"
+0.6.0    2012-08-06  "Cabo"
+0.5.0    2012-07-04  "Amsterdam"
+0.4.0    2012-05-23  "Unchained"
+0.3.0    2012-04-23  "Baluchitherium"
 0.2.0    ?           "Untitled"
 0.1.0    ?           "Untitled"
 0.0.2    ?           "Untitled"

VERSION

@@ -1 +1 @@
-2.4.6.0 1
+2.5.0 0.0.devel

bin/ansible

@@ -37,27 +37,30 @@ import shutil
 import sys
 import traceback
 
-import ansible.constants as C
 from ansible.errors import AnsibleError, AnsibleOptionsError, AnsibleParserError
-from ansible.utils.display import Display
 from ansible.module_utils._text import to_text
 
 
-########################################
-# OUTPUT OF LAST RESORT
 class LastResort(object):
-    def display(self, msg):
+    # OUTPUT OF LAST RESORT
+    def display(self, msg, log_only=None):
         print(msg, file=sys.stderr)
 
     def error(self, msg, wrap_text=None):
         print(msg, file=sys.stderr)
 
 
-########################################
-
 if __name__ == '__main__':
 
     display = LastResort()
+
+    try:  # bad ANSIBLE_CONFIG or config options can force ugly stacktrace
+        import ansible.constants as C
+        from ansible.utils.display import Display
+    except AnsibleOptionsError as e:
+        display.error(to_text(e), wrap_text=False)
+        sys.exit(5)
+
     cli = None
     me = os.path.basename(sys.argv[0])
 

bin/ansible-connection

@@ -214,6 +214,7 @@ class Server():
 
     def do_EXEC(self, data):
         cmd = data.split(b'EXEC: ')[1]
+        display.display('Command executed: %s' % cmd, log_only=True)
         return self.connection.exec_command(cmd)
 
     def do_PUT(self, data):

contrib/inventory/apstra_aos.py

@@ -466,35 +466,30 @@ class AosInventory(object):
         except:
             if 'AOS_SERVER' in os.environ.keys():
                 self.aos_server = os.environ['AOS_SERVER']
-            pass
 
         try:
             self.aos_server_port = config.get('aos', 'port')
         except:
             if 'AOS_PORT' in os.environ.keys():
                 self.aos_server_port = os.environ['AOS_PORT']
-            pass
 
         try:
             self.aos_username = config.get('aos', 'username')
         except:
             if 'AOS_USERNAME' in os.environ.keys():
                 self.aos_username = os.environ['AOS_USERNAME']
-            pass
 
         try:
             self.aos_password = config.get('aos', 'password')
         except:
             if 'AOS_PASSWORD' in os.environ.keys():
                 self.aos_password = os.environ['AOS_PASSWORD']
-            pass
 
         try:
             self.aos_blueprint = config.get('aos', 'blueprint')
         except:
             if 'AOS_BLUEPRINT' in os.environ.keys():
                 self.aos_blueprint = os.environ['AOS_BLUEPRINT']
-            pass
 
         try:
             if config.get('aos', 'blueprint_interface') in ['false', 'no']:

contrib/inventory/azure_rm.py

@@ -187,18 +187,14 @@ Version: 1.0.0
 '''
 
 import argparse
+import ConfigParser
 import json
 import os
 import re
 import sys
 import inspect
+import traceback
 
-try:
-    # python2
-    import ConfigParser as cp
-except ImportError:
-    # python3
-    import configparser as cp
 
 from packaging.version import Version
 
@@ -330,7 +326,7 @@ class AzureRM(object):
         path = expanduser("~")
         path += "/.azure/credentials"
         try:
-            config = cp.ConfigParser()
+            config = ConfigParser.ConfigParser()
             config.read(path)
         except Exception as exc:
             self.fail("Failed to access {0}. Check that the file exists and you have read "
@@ -799,7 +795,7 @@ class AzureInventory(object):
         config = None
         settings = None
         try:
-            config = cp.ConfigParser()
+            config = ConfigParser.ConfigParser()
             config.read(path)
         except:
             pass

contrib/inventory/collins.py

@@ -169,8 +169,9 @@ class CollinsInventory(object):
         print(data_to_print)
         return successful
 
-    def find_assets(self, attributes={}, operation='AND'):
+    def find_assets(self, attributes=None, operation='AND'):
         """ Obtains Collins assets matching the provided attributes. """
+        attributes = {} if attributes is None else attributes
 
         # Formats asset search query to locate assets matching attributes, using
         # the CQL search feature as described here:
@@ -211,7 +212,7 @@ class CollinsInventory(object):
                 cur_page += 1
                 num_retries = 0
             except:
-                self.log.error("Error while communicating with Collins, retrying:\n%s" % traceback.format_exc())
+                self.log.error("Error while communicating with Collins, retrying:\n%s", traceback.format_exc())
                 num_retries += 1
         return assets
 
@@ -280,7 +281,7 @@ class CollinsInventory(object):
         try:
             server_assets = self.find_assets()
         except:
-            self.log.error("Error while locating assets from Collins:\n%s" % traceback.format_exc())
+            self.log.error("Error while locating assets from Collins:\n%s", traceback.format_exc())
             return False
 
         for asset in server_assets:
@@ -304,7 +305,7 @@ class CollinsInventory(object):
             if self.prefer_hostnames and self._asset_has_attribute(asset, 'HOSTNAME'):
                 asset_identifier = self._asset_get_attribute(asset, 'HOSTNAME')
             elif 'ADDRESSES' not in asset:
-                self.log.warning("No IP addresses found for asset '%s', skipping" % asset)
+                self.log.warning("No IP addresses found for asset '%s', skipping", asset)
                 continue
             elif len(asset['ADDRESSES']) < ip_index + 1:
                 self.log.warning(

contrib/inventory/consul_io.py

@@ -473,7 +473,10 @@ class ConsulConfig(dict):
         scheme = 'http'
 
         if hasattr(self, 'url'):
-            from ansible.module_utils.six.moves.urllib.parse import urlparse
+            try:
+                from urlparse import urlparse
+            except ImportError:
+                from urllib.parse import urlparse
             o = urlparse(self.url)
             if o.hostname:
                 host = o.hostname

contrib/inventory/ec2.ini

@@ -159,7 +159,9 @@ group_by_elasticache_replication_group = True
 # inventory. For the full list of possible filters, please read the EC2 API
 # docs: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html#query-DescribeInstances-filters
 # Filters are key/value pairs separated by '=', to list multiple filters use
-# a list separated by commas. See examples below.
+# a list separated by commas. To "AND" criteria together, use "&". Note that
+# the "AND" is not useful along with stack_filters and so such usage is not allowed.
+# See examples below.
 
 # If you want to apply multiple filters simultaneously, set stack_filters to
 # True. Default behaviour is to combine the results of all filters. Stacking
@@ -181,6 +183,13 @@ stack_filters = False
 # (ex. webservers15, webservers1a, webservers123 etc)
 # instance_filters = tag:Name=webservers1*
 
+# Retrieve only instances of type t1.micro that also have tag env=stage
+# instance_filters = instance-type=t1.micro&tag:env=stage
+
+# Retrieve instances of type t1.micro AND tag env=stage, as well as any instance
+# that are of type m3.large, regardless of env tag
+# instance_filters = instance-type=t1.micro&tag:env=stage,instance-type=m3.large
+
 # An IAM role can be assumed, so all requests are run as that role.
 # This can be useful for connecting across different accounts, or to limit user
 # access

contrib/inventory/ec2.py

@@ -139,7 +139,7 @@ from ansible.module_utils import ec2 as ec2_utils
 
 HAS_BOTO3 = False
 try:
-    import boto3
+    import boto3  # noqa
     HAS_BOTO3 = True
 except ImportError:
     pass
@@ -152,6 +152,60 @@ try:
 except ImportError:
     import simplejson as json
 
+DEFAULTS = {
+    'all_elasticache_clusters': 'False',
+    'all_elasticache_nodes': 'False',
+    'all_elasticache_replication_groups': 'False',
+    'all_instances': 'False',
+    'all_rds_instances': 'False',
+    'aws_access_key_id': None,
+    'aws_secret_access_key': None,
+    'aws_security_token': None,
+    'boto_profile': None,
+    'cache_max_age': '300',
+    'cache_path': '~/.ansible/tmp',
+    'destination_variable': 'public_dns_name',
+    'elasticache': 'True',
+    'eucalyptus': 'False',
+    'eucalyptus_host': None,
+    'expand_csv_tags': 'False',
+    'group_by_ami_id': 'True',
+    'group_by_availability_zone': 'True',
+    'group_by_aws_account': 'False',
+    'group_by_elasticache_cluster': 'True',
+    'group_by_elasticache_engine': 'True',
+    'group_by_elasticache_parameter_group': 'True',
+    'group_by_elasticache_replication_group': 'True',
+    'group_by_instance_id': 'True',
+    'group_by_instance_state': 'False',
+    'group_by_instance_type': 'True',
+    'group_by_key_pair': 'True',
+    'group_by_platform': 'True',
+    'group_by_rds_engine': 'True',
+    'group_by_rds_parameter_group': 'True',
+    'group_by_region': 'True',
+    'group_by_route53_names': 'True',
+    'group_by_security_group': 'True',
+    'group_by_tag_keys': 'True',
+    'group_by_tag_none': 'True',
+    'group_by_vpc_id': 'True',
+    'hostname_variable': None,
+    'iam_role': None,
+    'include_rds_clusters': 'False',
+    'nested_groups': 'False',
+    'pattern_exclude': None,
+    'pattern_include': None,
+    'rds': 'False',
+    'regions': 'all',
+    'regions_exclude': 'us-gov-west-1, cn-north-1',
+    'replace_dash_in_groups': 'True',
+    'route53': 'False',
+    'route53_excluded_zones': '',
+    'route53_hostnames': None,
+    'stack_filters': 'False',
+    'vpc_destination_variable': 'ip_address'
+}
+
 
 class Ec2Inventory(object):
 
@@ -232,24 +286,32 @@ class Ec2Inventory(object):
         }
 
         if six.PY3:
-            config = configparser.ConfigParser()
+            config = configparser.ConfigParser(DEFAULTS)
         else:
-            config = configparser.SafeConfigParser()
+            config = configparser.SafeConfigParser(DEFAULTS)
         ec2_ini_path = os.environ.get('EC2_INI_PATH', defaults['ec2']['ini_path'])
         ec2_ini_path = os.path.expanduser(os.path.expandvars(ec2_ini_path))
 
         if not os.path.isfile(ec2_ini_path):
             ec2_ini_path = os.path.expanduser(defaults['ec2']['ini_fallback'])
 
-        config.read(ec2_ini_path)
+        if os.path.isfile(ec2_ini_path):
+            config.read(ec2_ini_path)
+
+        # Add empty sections if they don't exist
+        try:
+            config.add_section('ec2')
+        except configparser.DuplicateSectionError:
+            pass
+
+        try:
+            config.add_section('credentials')
+        except configparser.DuplicateSectionError:
+            pass
 
         # is eucalyptus?
-        self.eucalyptus_host = None
-        self.eucalyptus = False
-        if config.has_option('ec2', 'eucalyptus'):
-            self.eucalyptus = config.getboolean('ec2', 'eucalyptus')
-        if self.eucalyptus and config.has_option('ec2', 'eucalyptus_host'):
-            self.eucalyptus_host = config.get('ec2', 'eucalyptus_host')
+        self.eucalyptus = config.getboolean('ec2', 'eucalyptus')
+        self.eucalyptus_host = config.get('ec2', 'eucalyptus_host')
 
         # Regions
         self.regions = []
@@ -259,6 +321,7 @@ class Ec2Inventory(object):
                 self.regions.append(boto.connect_euca(host=self.eucalyptus_host).region.name, **self.credentials)
             else:
                 configRegions_exclude = config.get('ec2', 'regions_exclude')
+
                 for regionInfo in ec2.regions():
                     if regionInfo.name not in configRegions_exclude:
                         self.regions.append(regionInfo.name)
@@ -273,11 +336,7 @@ class Ec2Inventory(object):
         # Destination addresses
         self.destination_variable = config.get('ec2', 'destination_variable')
         self.vpc_destination_variable = config.get('ec2', 'vpc_destination_variable')
-
-        if config.has_option('ec2', 'hostname_variable'):
-            self.hostname_variable = config.get('ec2', 'hostname_variable')
-        else:
-            self.hostname_variable = None
+        self.hostname_variable = config.get('ec2', 'hostname_variable')
 
         if config.has_option('ec2', 'destination_format') and \
            config.has_option('ec2', 'destination_format_tags'):
@@ -289,36 +348,22 @@ class Ec2Inventory(object):
 
         # Route53
         self.route53_enabled = config.getboolean('ec2', 'route53')
-        if config.has_option('ec2', 'route53_hostnames'):
-            self.route53_hostnames = config.get('ec2', 'route53_hostnames')
-        else:
-            self.route53_hostnames = None
+        self.route53_hostnames = config.get('ec2', 'route53_hostnames')
+
         self.route53_excluded_zones = []
-        if config.has_option('ec2', 'route53_excluded_zones'):
-            self.route53_excluded_zones.extend(
-                config.get('ec2', 'route53_excluded_zones', '').split(','))
+        self.route53_excluded_zones = [a for a in config.get('ec2', 'route53_excluded_zones').split(',') if a]
 
         # Include RDS instances?
-        self.rds_enabled = True
-        if config.has_option('ec2', 'rds'):
-            self.rds_enabled = config.getboolean('ec2', 'rds')
+        self.rds_enabled = config.getboolean('ec2', 'rds')
 
         # Include RDS cluster instances?
-        if config.has_option('ec2', 'include_rds_clusters'):
-            self.include_rds_clusters = config.getboolean('ec2', 'include_rds_clusters')
-        else:
-            self.include_rds_clusters = False
+        self.include_rds_clusters = config.getboolean('ec2', 'include_rds_clusters')
 
         # Include ElastiCache instances?
-        self.elasticache_enabled = True
-        if config.has_option('ec2', 'elasticache'):
-            self.elasticache_enabled = config.getboolean('ec2', 'elasticache')
+        self.elasticache_enabled = config.getboolean('ec2', 'elasticache')
 
         # Return all EC2 instances?
-        if config.has_option('ec2', 'all_instances'):
-            self.all_instances = config.getboolean('ec2', 'all_instances')
-        else:
-            self.all_instances = False
+        self.all_instances = config.getboolean('ec2', 'all_instances')
 
         # Instance states to be gathered in inventory. Default is 'running'.
         # Setting 'all_instances' to 'yes' overrides this option.
@@ -343,49 +388,30 @@ class Ec2Inventory(object):
             self.ec2_instance_states = ['running']
 
         # Return all RDS instances? (if RDS is enabled)
-        if config.has_option('ec2', 'all_rds_instances') and self.rds_enabled:
-            self.all_rds_instances = config.getboolean('ec2', 'all_rds_instances')
-        else:
-            self.all_rds_instances = False
+        self.all_rds_instances = config.getboolean('ec2', 'all_rds_instances')
 
         # Return all ElastiCache replication groups? (if ElastiCache is enabled)
-        if config.has_option('ec2', 'all_elasticache_replication_groups') and self.elasticache_enabled:
-            self.all_elasticache_replication_groups = config.getboolean('ec2', 'all_elasticache_replication_groups')
-        else:
-            self.all_elasticache_replication_groups = False
+        self.all_elasticache_replication_groups = config.getboolean('ec2', 'all_elasticache_replication_groups')
 
         # Return all ElastiCache clusters? (if ElastiCache is enabled)
-        if config.has_option('ec2', 'all_elasticache_clusters') and self.elasticache_enabled:
-            self.all_elasticache_clusters = config.getboolean('ec2', 'all_elasticache_clusters')
-        else:
-            self.all_elasticache_clusters = False
+        self.all_elasticache_clusters = config.getboolean('ec2', 'all_elasticache_clusters')
 
         # Return all ElastiCache nodes? (if ElastiCache is enabled)
-        if config.has_option('ec2', 'all_elasticache_nodes') and self.elasticache_enabled:
-            self.all_elasticache_nodes = config.getboolean('ec2', 'all_elasticache_nodes')
-        else:
-            self.all_elasticache_nodes = False
+        self.all_elasticache_nodes = config.getboolean('ec2', 'all_elasticache_nodes')
 
         # boto configuration profile (prefer CLI argument then environment variables then config file)
-        self.boto_profile = self.args.boto_profile or os.environ.get('AWS_PROFILE')
-        if config.has_option('ec2', 'boto_profile') and not self.boto_profile:
-            self.boto_profile = config.get('ec2', 'boto_profile')
+        self.boto_profile = self.args.boto_profile or \
+            os.environ.get('AWS_PROFILE') or \
+            config.get('ec2', 'boto_profile')
 
         # AWS credentials (prefer environment variables)
         if not (self.boto_profile or os.environ.get('AWS_ACCESS_KEY_ID') or
                 os.environ.get('AWS_PROFILE')):
-            if config.has_option('credentials', 'aws_access_key_id'):
-                aws_access_key_id = config.get('credentials', 'aws_access_key_id')
-            else:
-                aws_access_key_id = None
-            if config.has_option('credentials', 'aws_secret_access_key'):
-                aws_secret_access_key = config.get('credentials', 'aws_secret_access_key')
-            else:
-                aws_secret_access_key = None
-            if config.has_option('credentials', 'aws_security_token'):
-                aws_security_token = config.get('credentials', 'aws_security_token')
-            else:
-                aws_security_token = None
+
+            aws_access_key_id = config.get('credentials', 'aws_access_key_id')
+            aws_secret_access_key = config.get('credentials', 'aws_secret_access_key')
+            aws_security_token = config.get('credentials', 'aws_security_token')
+
             if aws_access_key_id:
                 self.credentials = {
                     'aws_access_key_id': aws_access_key_id,
@@ -409,98 +435,59 @@ class Ec2Inventory(object):
         self.cache_path_index = os.path.join(cache_dir, "%s.index" % cache_name)
         self.cache_max_age = config.getint('ec2', 'cache_max_age')
 
-        if config.has_option('ec2', 'expand_csv_tags'):
-            self.expand_csv_tags = config.getboolean('ec2', 'expand_csv_tags')
-        else:
-            self.expand_csv_tags = False
+        self.expand_csv_tags = config.getboolean('ec2', 'expand_csv_tags')
 
         # Configure nested groups instead of flat namespace.
-        if config.has_option('ec2', 'nested_groups'):
-            self.nested_groups = config.getboolean('ec2', 'nested_groups')
-        else:
-            self.nested_groups = False
+        self.nested_groups = config.getboolean('ec2', 'nested_groups')
 
         # Replace dash or not in group names
-        if config.has_option('ec2', 'replace_dash_in_groups'):
-            self.replace_dash_in_groups = config.getboolean('ec2', 'replace_dash_in_groups')
-        else:
-            self.replace_dash_in_groups = True
+        self.replace_dash_in_groups = config.getboolean('ec2', 'replace_dash_in_groups')
 
         # IAM role to assume for connection
-        if config.has_option('ec2', 'iam_role'):
-            self.iam_role = config.get('ec2', 'iam_role')
-        else:
-            self.iam_role = None
+        self.iam_role = config.get('ec2', 'iam_role')
 
         # Configure which groups should be created.
-        group_by_options = [
-            'group_by_instance_id',
-            'group_by_region',
-            'group_by_availability_zone',
-            'group_by_ami_id',
-            'group_by_instance_type',
-            'group_by_instance_state',
-            'group_by_platform',
-            'group_by_key_pair',
-            'group_by_vpc_id',
-            'group_by_security_group',
-            'group_by_tag_keys',
-            'group_by_tag_none',
-            'group_by_route53_names',
-            'group_by_rds_engine',
-            'group_by_rds_parameter_group',
-            'group_by_elasticache_engine',
-            'group_by_elasticache_cluster',
-            'group_by_elasticache_parameter_group',
-            'group_by_elasticache_replication_group',
-            'group_by_aws_account',
-        ]
+
+        group_by_options = [a for a in DEFAULTS if a.startswith('group_by')]
         for option in group_by_options:
-            if config.has_option('ec2', option):
-                setattr(self, option, config.getboolean('ec2', option))
-            else:
-                setattr(self, option, True)
+            setattr(self, option, config.getboolean('ec2', option))
 
         # Do we need to just include hosts that match a pattern?
-        try:
-            pattern_include = config.get('ec2', 'pattern_include')
-            if pattern_include and len(pattern_include) > 0:
-                self.pattern_include = re.compile(pattern_include)
-            else:
-                self.pattern_include = None
-        except configparser.NoOptionError:
-            self.pattern_include = None
+        self.pattern_include = config.get('ec2', 'pattern_include')
+        if self.pattern_include:
+            self.pattern_include = re.compile(self.pattern_include)
 
         # Do we need to exclude hosts that match a pattern?
-        try:
-            pattern_exclude = config.get('ec2', 'pattern_exclude')
-            if pattern_exclude and len(pattern_exclude) > 0:
-                self.pattern_exclude = re.compile(pattern_exclude)
-            else:
-                self.pattern_exclude = None
-        except configparser.NoOptionError:
-            self.pattern_exclude = None
+        self.pattern_exclude = config.get('ec2', 'pattern_exclude')
+        if self.pattern_exclude:
+            self.pattern_exclude = re.compile(self.pattern_exclude)
 
         # Do we want to stack multiple filters?
-        if config.has_option('ec2', 'stack_filters'):
-            self.stack_filters = config.getboolean('ec2', 'stack_filters')
-        else:
-            self.stack_filters = False
+        self.stack_filters = config.getboolean('ec2', 'stack_filters')
 
         # Instance filters (see boto and EC2 API docs). Ignore invalid filters.
-        self.ec2_instance_filters = defaultdict(list)
+        self.ec2_instance_filters = []
+
         if config.has_option('ec2', 'instance_filters'):
+            filters = config.get('ec2', 'instance_filters')
 
-            filters = [f for f in config.get('ec2', 'instance_filters').split(',') if f]
+            if self.stack_filters and '&' in filters:
+                self.fail_with_error("AND filters along with stack_filter enabled is not supported.\n")
 
-            for instance_filter in filters:
-                instance_filter = instance_filter.strip()
-                if not instance_filter or '=' not in instance_filter:
-                    continue
-                filter_key, filter_value = [x.strip() for x in instance_filter.split('=', 1)]
-                if not filter_key:
-                    continue
-                self.ec2_instance_filters[filter_key].append(filter_value)
+            filter_sets = [f for f in filters.split(',') if f]
+
+            for filter_set in filter_sets:
+                filters = {}
+                filter_set = filter_set.strip()
+                for instance_filter in filter_set.split("&"):
+                    instance_filter = instance_filter.strip()
+                    if not instance_filter or '=' not in instance_filter:
+                        continue
+                    filter_key, filter_value = [x.strip() for x in instance_filter.split('=', 1)]
+                    if not filter_key:
+                        continue
+                    filters[filter_key] = filter_value
+                self.ec2_instance_filters.append(filters.copy())
 
     def parse_cli_args(self):
         ''' Command line argument processing '''
@@ -582,12 +569,12 @@ class Ec2Inventory(object):
             if self.ec2_instance_filters:
                 if self.stack_filters:
                     filters_dict = {}
-                    for filter_key, filter_values in self.ec2_instance_filters.items():
-                        filters_dict[filter_key] = filter_values
+                    for filters in self.ec2_instance_filters:
+                        filters_dict.update(filters)
                     reservations.extend(conn.get_all_instances(filters=filters_dict))
                 else:
-                    for filter_key, filter_values in self.ec2_instance_filters.items():
-                        reservations.extend(conn.get_all_instances(filters={filter_key: filter_values}))
+                    for filters in self.ec2_instance_filters:
+                        reservations.extend(conn.get_all_instances(filters=filters))
             else:
                 reservations = conn.get_all_instances()
 
@@ -623,6 +610,33 @@ class Ec2Inventory(object):
                 error = "Error connecting to %s backend.\n%s" % (backend, e.message)
             self.fail_with_error(error, 'getting EC2 instances')
 
+    def tags_match_filters(self, tags):
+        ''' return True if given tags match configured filters '''
+        if not self.ec2_instance_filters:
+            return True
+
+        for filters in self.ec2_instance_filters:
+            for filter_name, filter_value in filters.items():
+                if filter_name[:4] != 'tag:':
+                    continue
+                filter_name = filter_name[4:]
+                if filter_name not in tags:
+                    if self.stack_filters:
+                        return False
+                    continue
+                if isinstance(filter_value, list):
+                    if self.stack_filters and tags[filter_name] not in filter_value:
+                        return False
+                    if not self.stack_filters and tags[filter_name] in filter_value:
+                        return True
+                if isinstance(filter_value, six.string_types):
+                    if self.stack_filters and tags[filter_name] != filter_value:
+                        return False
+                    if not self.stack_filters and tags[filter_name] == filter_value:
+                        return True
+
+        return self.stack_filters
+
     def get_rds_instances_by_region(self, region):
         ''' Makes an AWS API call to the list of RDS instances in a particular
         region '''
@@ -648,8 +662,8 @@ class Ec2Inventory(object):
                         instance.tags = {}
                         for tag in tags:
                             instance.tags[tag['Key']] = tag['Value']
-
-                        self.add_rds_instance(instance, region)
+                        if self.tags_match_filters(instance.tags):
+                            self.add_rds_instance(instance, region)
                     if not marker:
                         break
         except boto.exception.BotoServerError as e:
@@ -688,7 +702,7 @@ class Ec2Inventory(object):
             if 'LatestRestorableTime' in c:
                 del c['LatestRestorableTime']
 
-            if self.ec2_instance_filters == {}:
+            if not self.ec2_instance_filters:
                 matches_filter = True
             else:
                 matches_filter = False
@@ -700,14 +714,18 @@ class Ec2Inventory(object):
                 c['Tags'] = tags['TagList']
 
                 if self.ec2_instance_filters:
-                    for filter_key, filter_values in self.ec2_instance_filters.items():
-                        # get AWS tag key e.g. tag:env will be 'env'
-                        tag_name = filter_key.split(":", 1)[1]
-                        # Filter values is a list (if you put multiple values for the same tag name)
-                        matches_filter = any(d['Key'] == tag_name and d['Value'] in filter_values for d in c['Tags'])
+                    for filters in self.ec2_instance_filters:
+                        for filter_key, filter_values in filters.items():
+                            # get AWS tag key e.g. tag:env will be 'env'
+                            tag_name = filter_key.split(":", 1)[1]
+                            # Filter values is a list (if you put multiple values for the same tag name)
+                            matches_filter = any(d['Key'] == tag_name and d['Value'] in filter_values for d in c['Tags'])
+
+                            if matches_filter:
+                                # it matches a filter, so stop looking for further matches
+                                break
 
                         if matches_filter:
-                            # it matches a filter, so stop looking for further matches
                             break
 
             except Exception as e:

contrib/inventory/proxmox.py

@@ -53,7 +53,9 @@ class ProxmoxVM(dict):
 
 
 class ProxmoxVMList(list):
-    def __init__(self, data=[]):
+    def __init__(self, data=None):
+        data = [] if data is None else data
+
         for item in data:
             self.append(ProxmoxVM(item))
 

contrib/inventory/rax.py

@@ -233,13 +233,17 @@ def _list_into_cache(regions):
                               'RAX_ACCESS_NETWORK', 'public', value_type='list')
     except TypeError:
         # Ansible 2.2.x and below
+        # pylint: disable=unexpected-keyword-arg
         networks = get_config(p, 'rax', 'access_network',
                               'RAX_ACCESS_NETWORK', 'public', islist=True)
     try:
         try:
+            # Ansible 2.3+
             ip_versions = map(int, get_config(p, 'rax', 'access_ip_version',
                                               'RAX_ACCESS_IP_VERSION', 4, value_type='list'))
         except TypeError:
+            # Ansible 2.2.x and below
+            # pylint: disable=unexpected-keyword-arg
             ip_versions = map(int, get_config(p, 'rax', 'access_ip_version',
                                               'RAX_ACCESS_IP_VERSION', 4, islist=True))
     except:
@@ -438,6 +442,7 @@ def setup():
                                      value_type='list')
         except TypeError:
             # Ansible 2.2.x and below
+            # pylint: disable=unexpected-keyword-arg
             region_list = get_config(p, 'rax', 'regions', 'RAX_REGION', 'all',
                                      islist=True)
 

contrib/inventory/rudder.py

@@ -56,8 +56,12 @@ import argparse
 import six
 import httplib2 as http
 from time import time
-from ansible.module_utils.six.moves import configparser
-from ansible.module_utils.six.moves.urllib.parse import urlparse
+from six.moves import configparser
+
+try:
+    from urlparse import urlparse
+except ImportError:
+    from urllib.parse import urlparse
 
 try:
     import json

contrib/inventory/vagrant.py

@@ -38,17 +38,14 @@ import os.path
 import subprocess
 import re
 from paramiko import SSHConfig
+from cStringIO import StringIO
 from optparse import OptionParser
 from collections import defaultdict
 try:
     import json
-except Exception:
+except:
     import simplejson as json
 
-from ansible.module_utils._text import to_text
-from ansible.module_utils.six.moves import StringIO
-
-
 _group = 'vagrant'  # a default group
 _ssh_to_ansible = [('user', 'ansible_ssh_user'),
                    ('hostname', 'ansible_ssh_host'),
@@ -77,8 +74,7 @@ def get_ssh_config():
 
 # list all the running boxes
 def list_running_boxes():
-
-    output = to_text(subprocess.check_output(["vagrant", "status"]), errors='surrogate_or_strict').split('\n')
+    output = subprocess.check_output(["vagrant", "status"]).split('\n')
 
     boxes = []
 
@@ -94,7 +90,7 @@ def list_running_boxes():
 def get_a_ssh_config(box_name):
     """Gives back a map of all the machine's ssh configurations"""
 
-    output = to_text(subprocess.check_output(["vagrant", "ssh-config", box_name]), errors='surrogate_or_strict')
+    output = subprocess.check_output(["vagrant", "ssh-config", box_name])
     config = SSHConfig()
     config.parse(StringIO(output))
     host_config = config.lookup(box_name)
@@ -108,7 +104,6 @@ def get_a_ssh_config(box_name):
 
     return dict((v, host_config[k]) for k, v in _ssh_to_ansible)
 
-
 # List out servers that vagrant has running
 # ------------------------------
 if options.list:

contrib/inventory/vmware_inventory.py

@@ -41,6 +41,7 @@ from __future__ import print_function
 import atexit
 import datetime
 import getpass
+import itertools
 import json
 import os
 import re
@@ -84,6 +85,14 @@ def regex_match(s, pattern):
         return False
 
 
+def select_chain_match(inlist, key, pattern):
+    '''Get a key from a list of dicts, squash values to a single list, then filter'''
+    outlist = [x[key] for x in inlist]
+    outlist = list(itertools.chain(*outlist))
+    outlist = [x for x in outlist if regex_match(x, pattern)]
+    return outlist
+
+
 class VMwareMissingHostException(Exception):
     pass
 
@@ -127,6 +136,7 @@ class VMWareInventory(object):
     # use jinja environments to allow for custom filters
     env = Environment()
     env.filters['regex_match'] = regex_match
+    env.filters['select_chain_match'] = select_chain_match
 
     # translation table for attributes to fetch for known vim types
 

contrib/inventory/windows_azure.py

@@ -39,7 +39,7 @@ import re
 import sys
 import argparse
 import os
-from ansible.module_utils.six.moves.urllib.parse import urlparse
+from urlparse import urlparse
 from time import time
 try:
     import json

docs/bin/dump_keywords.py

@@ -1,11 +1,8 @@
 #!/usr/bin/env python
 
 import optparse
-import re
-from distutils.version import LooseVersion
-
-import jinja2
 import yaml
+
 from jinja2 import Environment, FileSystemLoader
 
 from ansible.playbook import Play
@@ -21,7 +18,7 @@ class_list = [Play, Role, Block, Task]
 p = optparse.OptionParser(
     version='%prog 1.0',
     usage='usage: %prog [options]',
-    description='Generate playbook keyword documentation from code and descriptions',
+    description='Generate module documentation from metadata',
 )
 p.add_option("-T", "--template-dir", action="store", dest="template_dir", default="../templates", help="directory containing Jinja2 templates")
 p.add_option("-o", "--output-dir", action="store", dest="output_dir", default='/tmp/', help="Output directory for rst files")
@@ -68,10 +65,5 @@ template = env.get_template(template_file)
 outputname = options.output_dir + template_file.replace('.j2', '')
 tempvars = {'oblist': oblist, 'clist': clist}
 
-keyword_page = template.render(tempvars)
-if LooseVersion(jinja2.__version__) < LooseVersion('2.10'):
-    # jinja2 < 2.10's indent filter indents blank lines.  Cleanup
-    keyword_page = re.sub(' +\n', '\n', keyword_page)
-
 with open(outputname, 'w') as f:
-    f.write(keyword_page)
+    f.write(template.render(tempvars))

docs/bin/generate_man.py

@@ -31,14 +31,14 @@ def trim_docstring(docstring):
     # and split into a list of lines:
     lines = docstring.expandtabs().splitlines()
     # Determine minimum indentation (first line doesn't count):
-    indent = sys.maxint
+    indent = sys.maxsize
     for line in lines[1:]:
         stripped = line.lstrip()
         if stripped:
             indent = min(indent, len(line) - len(stripped))
     # Remove indentation (first line is special):
     trimmed = [lines[0].strip()]
-    if indent < sys.maxint:
+    if indent < sys.maxsize:
         for line in lines[1:]:
             trimmed.append(line[indent:].rstrip())
     # Strip off trailing and leading blank lines:

docs/bin/plugin_formatter.py

@@ -26,13 +26,11 @@ import datetime
 import glob
 import optparse
 import os
+import pprint
 import re
 import sys
 import warnings
 from collections import defaultdict
-from distutils.version import LooseVersion
-from pprint import PrettyPrinter
-
 try:
     from html import escape as html_escape
 except ImportError:
@@ -42,10 +40,9 @@ except ImportError:
     def html_escape(text, quote=True):
         return cgi.escape(text, quote)
 
-import jinja2
 import yaml
 from jinja2 import Environment, FileSystemLoader
-from six import iteritems
+from six import iteritems, string_types
 
 from ansible.errors import AnsibleError
 from ansible.module_utils._text import to_bytes
@@ -159,6 +156,7 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
             least one key, '_modules' which contains a list of module names in
             that category.  Any other keys in the dict are subcategories with
             the same structure.
+
     '''
 
     categories = dict()
@@ -181,7 +179,7 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
 
         # Do not list blacklisted modules
         module = os.path.splitext(os.path.basename(module_path))[0]
-        if module in plugin_docs.BLACKLIST['MODULE']:
+        if module in plugin_docs.BLACKLIST['MODULE'] or module == 'base':
             continue
 
         # If requested, limit module documentation building only to passed-in
@@ -214,15 +212,21 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
         # Start at the second directory because we don't want the "vendor"
         mod_path_only = os.path.dirname(module_path[len(module_dir):])
 
+        module_categories = []
         # build up the categories that this module belongs to
         for new_cat in mod_path_only.split('/')[1:]:
             if new_cat not in category:
                 category[new_cat] = dict()
                 category[new_cat]['_modules'] = []
+            module_categories.append(new_cat)
             category = category[new_cat]
 
         category['_modules'].append(module)
 
+        # the category we will use in links (so list_of_all_plugins can point to plugins/action_plugins/*'
+        if module_categories:
+            primary_category = module_categories[0]
+
         # use ansible core library to parse out doc metadata YAML and plaintext examples
         doc, examples, returndocs, metadata = plugin_docs.get_docstring(module_path, verbose=verbose)
 
@@ -234,6 +238,8 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
                                'doc': doc,
                                'examples': examples,
                                'returndocs': returndocs,
+                               'categories': module_categories,
+                               'primary_category': primary_category,
                                }
 
     # keep module tests out of becoming module docs
@@ -254,6 +260,7 @@ def generate_parser():
 
     p.add_option("-A", "--ansible-version", action="store", dest="ansible_version", default="unknown", help="Ansible version number")
     p.add_option("-M", "--module-dir", action="store", dest="module_dir", default=MODULEDIR, help="Ansible library path")
+    p.add_option("-P", "--plugin-type", action="store", dest="plugin_type", default='modules', help="The type of plugin (plugins, modules)")
     p.add_option("-T", "--template-dir", action="store", dest="template_dir", default="hacking/templates", help="directory containing Jinja2 templates")
     p.add_option("-t", "--type", action='store', dest='type', choices=['rst'], default='rst', help="Document type")
     p.add_option("-v", "--verbose", action='store_true', default=False, help="Verbose")
@@ -265,7 +272,7 @@ def generate_parser():
     return p
 
 
-def jinja2_environment(template_dir, typ):
+def jinja2_environment(template_dir, typ, plugin_type):
 
     env = Environment(loader=FileSystemLoader(template_dir),
                       variable_start_string="@{",
@@ -280,14 +287,13 @@ def jinja2_environment(template_dir, typ):
         env.filters['fmt'] = rst_fmt
         env.filters['xline'] = rst_xline
         templates['plugin'] = env.get_template('plugin.rst.j2')
-        templates['category_list'] = env.get_template('modules_by_category.rst.j2')
-        templates['support_list'] = env.get_template('modules_by_support.rst.j2')
-        templates['list_of_CATEGORY_modules'] = env.get_template('list_of_CATEGORY_modules.rst.j2')
-        outputname = "%s_module.rst"
+        templates['category_list'] = env.get_template('%s_by_category.rst.j2' % plugin_type)
+        templates['support_list'] = env.get_template('%s_by_support.rst.j2' % plugin_type)
+        templates['list_of_CATEGORY_modules'] = env.get_template('list_of_CATEGORY_%s.rst.j2' % plugin_type)
     else:
         raise Exception("unknown module format type: %s" % typ)
 
-    return templates, outputname
+    return templates
 
 
 def too_old(added):
@@ -303,24 +309,54 @@ def too_old(added):
     return added_float < TO_OLD_TO_BE_NOTABLE
 
 
-def process_modules(module_map, templates, outputname, output_dir, ansible_version):
+def process_modules(module_map, templates, outputname, output_dir, ansible_version, plugin_type):
     for module in module_map:
-        print("rendering: %s" % module)
+        # print("rendering: %s" % module)
+
+        # pprint.pprint(('process_modules module:', module))
 
         fname = module_map[module]['path']
 
+        # pprint.pprint(('process_modules module_info: ', module_map[module]))
+
+        module_categories = module_map[module].get('categories', [])
+
         # crash if module is missing documentation and not explicitly hidden from docs index
         if module_map[module]['doc'] is None:
-            sys.exit("*** ERROR: MODULE MISSING DOCUMENTATION: %s, %s ***\n" % (fname, module))
+            print("%s: ERROR: MODULE MISSING DOCUMENTATION" % (fname,))
+            _doc = {'module': module,
+                    'version_added': '2.4',
+                    'filename': fname}
+            module_map[module]['doc'] = _doc
+            # continue
 
         # Going to reference this heavily so make a short name to reference it by
         doc = module_map[module]['doc']
 
+        # pprint.pprint(('process_modules doc: ', doc))
+
+        # add some defaults for plugins that dont have most of the info
+        doc['module'] = doc.get('module', module)
+        doc['version_added'] = doc.get('version_added', 'historical')
+
+        doc['plugin_type'] = plugin_type
+
         if module_map[module]['deprecated'] and 'deprecated' not in doc:
-            sys.exit("*** ERROR: DEPRECATED MODULE MISSING 'deprecated' DOCUMENTATION: %s, %s ***\n" % (fname, module))
+            print("%s: WARNING: MODULE MISSING DEPRECATION DOCUMENTATION: %s" % (fname, 'deprecated'))
+
+        required_fields = ('short_description',)
+        for field in required_fields:
+            if field not in doc:
+                print("%s: WARNING: MODULE MISSING field '%s'" % (fname, field))
+
+        not_nullable_fields = ('short_description',)
+        for field in not_nullable_fields:
+            if field in doc and doc[field] in (None, ''):
+                print("%s: WARNING: MODULE field '%s' DOCUMENTATION is null/empty value=%s" % (fname, field, doc[field]))
 
         if 'version_added' not in doc:
-            sys.exit("*** ERROR: missing version_added in: %s ***\n" % module)
+            pprint.pprint(doc)
+            # sys.exit("*** ERROR: missing version_added in: %s ***\n" % module)
 
         #
         # The present template gets everything from doc so we spend most of this
@@ -373,28 +409,54 @@ def process_modules(module_map, templates, outputname, output_dir, ansible_versi
         doc['docuri'] = doc['module'].replace('_', '-')
         doc['now_date'] = datetime.date.today().strftime('%Y-%m-%d')
         doc['ansible_version'] = ansible_version
-        doc['plainexamples'] = module_map[module]['examples']  # plain text
+
+        # check the 'deprecated' field in doc. We expect a dict potentially with 'why', 'version', and 'alternative' fields
+        # examples = module_map[module]['examples']
+        # print('\n\n%s: type of examples: %s\n' % (module, type(examples)))
+        # if examples and not isinstance(examples, (str, unicode, list)):
+        #    raise TypeError('module %s examples is wrong type (%s): %s' % (module, type(examples), examples))
+
+        # use 'examples' for 'plainexamples' if 'examples' is a string
+        if isinstance(module_map[module]['examples'], string_types):
+            doc['plainexamples'] = module_map[module]['examples']  # plain text
+        else:
+            doc['plainexamples'] = ''
+
         doc['metadata'] = module_map[module]['metadata']
 
+        # pprint.pprint(module_map[module]
         if module_map[module]['returndocs']:
             try:
                 doc['returndocs'] = yaml.safe_load(module_map[module]['returndocs'])
-            except:
-                print("could not load yaml: %s" % module_map[module]['returndocs'])
-                raise
+            except Exception as e:
+                print("%s:%s:yaml error:%s:returndocs=%s" % (fname, module, e, module_map[module]['returndocs']))
+                doc['returndocs'] = None
         else:
             doc['returndocs'] = None
 
+        doc['author'] = doc.get('author', ['UNKNOWN'])
+        if isinstance(doc['author'], string_types):
+            doc['author'] = [doc['author']]
+
+        # print('about to template')
+        # pprint.pprint(doc)
         text = templates['plugin'].render(doc)
-        if LooseVersion(jinja2.__version__) < LooseVersion('2.10'):
-            # jinja2 < 2.10's indent filter indents blank lines.  Cleanup
-            text = re.sub(' +\n', '\n', text)
 
-        write_data(text, output_dir, outputname, module)
+        # plugins get namespace dirs but modules do not
+        if plugin_type == 'plugins':
+            for module_category in module_categories:
+                category_output_dir = os.path.join(output_dir, 'plugins', '%s' % module_category)
+                write_data(text, category_output_dir, outputname, module)
+        else:
+                write_data(text, output_dir, outputname, module)
 
 
-def process_categories(mod_info, categories, templates, output_dir, output_name):
+def process_categories(mod_info, categories, templates,
+                       output_dir, output_name, plugin_type):
     for category in sorted(categories.keys()):
+        if (plugin_type, category) == ('plugins', ''):
+            print('skipping unknown cat: %s' % category)
+            continue
         module_map = categories[category]
         category_filename = output_name % category
 
@@ -402,21 +464,23 @@ def process_categories(mod_info, categories, templates, output_dir, output_name)
 
         # start a new category file
 
-        category = category.replace("_", " ")
-        category = category.title()
+        category_name = category.replace("_", " ")
+        category_title = category_name.title()
 
         subcategories = dict((k, v) for k, v in module_map.items() if k != '_modules')
-        template_data = {'title': category,
+        template_data = {'title': category_title,
+                         'category_name': category_name,
                          'category': module_map,
                          'subcategories': subcategories,
                          'module_info': mod_info,
+                         'plugin_type': plugin_type
                          }
 
         text = templates['list_of_CATEGORY_modules'].render(template_data)
         write_data(text, output_dir, category_filename)
 
 
-def process_support_levels(mod_info, templates, output_dir):
+def process_support_levels(mod_info, templates, output_dir, plugin_type):
     supported_by = {'Ansible Core Team': {'slug': 'core_supported',
                                           'modules': [],
                                           'output': 'core_maintained.rst',
@@ -460,8 +524,14 @@ These modules are currently shipped with Ansible, but will most likely be shippe
                                           """},
                     }
 
+    # only gen support pages for modules for now, need to split and namespace templates and generated docs
+    if plugin_type == 'plugins':
+        return
     # Separate the modules by support_level
     for module, info in mod_info.items():
+        if not info.get('metadata', None):
+            print('no metadata for %s' % module)
+            continue
         if info['metadata']['supported_by'] == 'core':
             supported_by['Ansible Core Team']['modules'].append(module)
         elif info['metadata']['supported_by'] == 'network':
@@ -502,34 +572,57 @@ def main():
     (options, args) = p.parse_args()
     validate_options(options)
 
-    templates, outputname = jinja2_environment(options.template_dir, options.type)
+    plugin_type = options.plugin_type
+
+    if plugin_type == 'modules':
+        templates = jinja2_environment(options.template_dir, options.type, plugin_type)
+        output_dir = options.output_dir
+        # trim trailing s off of plugin_type for plugin_type=='modules'. ie 'copy_module.rst'
+        outputname = '%s_' + '%s.rst' % plugin_type[:-1]
+    else:
+        templates = jinja2_environment(options.template_dir, options.type, 'plugins')
+        # for plugins, just use 'ssh.rst' vs 'ssh_module.rst'
+        outputname = '%s.rst'
+        output_dir = '%s/plugins/%s' % (options.output_dir, plugin_type)
 
     # Convert passed-in limit_to_modules to None or list of modules.
     if options.limit_to_modules is not None:
         options.limit_to_modules = [s.lower() for s in options.limit_to_modules.split(",")]
 
-    mod_info, categories = get_module_info(options.module_dir, limit_to_modules=options.limit_to_modules,
-                                           verbose=options.verbose)
+    mod_info, categories = get_module_info(options.module_dir, limit_to_modules=options.limit_to_modules, verbose=options.verbose)
 
     categories['all'] = {'_modules': mod_info.keys()}
 
+    # pprint.pprint(categories)
+    # pprint.pprint(mod_info)
+    # pprint.pprint(dict(mod_info))
     # Transform the data
     if options.type == 'rst':
-        for record in mod_info.values():
-            record['doc']['short_description'] = rst_ify(record['doc']['short_description'])
+        for key, record in mod_info.items():
+            # pprint.pprint(('record', record))
+            if record.get('doc', None):
+                short_desc = record['doc']['short_description']
+                if short_desc is None:
+                    print('WARNING: short_description for %s is None' % key)
+                    short_desc = ''
+                record['doc']['short_description'] = rst_ify(short_desc)
 
-    # Write master category list
-    category_list_text = templates['category_list'].render(categories=sorted(categories.keys()))
-    write_data(category_list_text, options.output_dir, 'modules_by_category.rst')
+    if plugin_type == 'modules':
+        # Write master category list
+        category_list_text = templates['category_list'].render(categories=sorted(categories.keys()))
+        category_index_name = '%s_by_category.rst' % plugin_type
+        write_data(category_list_text, output_dir, category_index_name)
 
     # Render all the individual module pages
-    process_modules(mod_info, templates, outputname, options.output_dir, options.ansible_version)
+    process_modules(mod_info, templates, outputname, output_dir, options.ansible_version, plugin_type)
 
     # Render all the categories for modules
-    process_categories(mod_info, categories, templates, options.output_dir, "list_of_%s_modules.rst")
+    if plugin_type == 'modules':
+        category_list_name_template = 'list_of_%s_' + '%s.rst' % plugin_type
+        process_categories(mod_info, categories, templates, output_dir, category_list_name_template, plugin_type)
 
-    # Render all the categories for modules
-    process_support_levels(mod_info, templates, options.output_dir)
+        # Render all the categories for modules
+        process_support_levels(mod_info, templates, output_dir, plugin_type)
 
 
 if __name__ == '__main__':

docs/docsite/Makefile

@@ -11,6 +11,18 @@ else
 CPUS ?= $(shell nproc)
 endif
 
+MODULE_ARGS=
+ifdef MODULES
+	MODULE_ARGS = -l $(MODULES)
+endif
+
+PLUGIN_ARGS=
+ifdef PLUGINS
+	PLUGIN_ARGS = -l $(PLUGINS)
+endif
+
+DOC_PLUGINS ?= cache callback connection inventory lookup strategy vars
+
 assertrst:
 ifndef rst
 	$(error specify document or pattern with rst=somefile.rst)
@@ -20,10 +32,14 @@ all: docs
 
 docs: clean htmldocs
 
-htmldocs: testing keywords modules staticmin cli config
+generate_rst: testing keywords modules plugins staticmin cli config
 
+htmldocs: generate_rst
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx html
 
+singlehtmldocs: generate_rst
+	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx singlehtml
+
 webdocs: docs
 
 #TODO: leaving htmlout removal for those having older versions, should eventually be removed also
@@ -44,10 +60,14 @@ clean:
 	-rm rst/list_of_*.rst
 	-rm rst/*_by_category.rst
 	-rm rst/*_module.rst
+	-rm rst/*_plugin.rst
 	-rm rst/*_maintained.rst
 	-rm rst/playbooks_directives.rst
 	-rm rst/playbooks_keywords.rst
-#	-rm rst/cli/ansible*.rst
+	-rm rst/plugins/*/*.rst
+	-rm rst/config.rst
+	-rm rst/ansible-*.rst
+	-rm rst/ansible.rst
 
 .PHONEY: docs clean
 
@@ -62,12 +82,14 @@ config:
 	PYTHONPATH=../../lib $(CONFIG_DUMPER) --template-file=../templates/config.rst.j2 --output-dir=rst/ -d ../../lib/ansible/config/base.yml
 
 modules: $(FORMATTER) ../templates/plugin.rst.j2
-# Limit building of module documentation if requested.
-ifdef MODULES
-	PYTHONPATH=../../lib $(FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/ -l $(MODULES)
-else
-	PYTHONPATH=../../lib $(FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/
-endif
+	PYTHONPATH=../../lib $(FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/ $(MODULE_ARGS)
+
+plugins: $(FORMATTER) ../templates/plugin.rst.j2
+	@echo "looping over doc plugins"
+	for plugin in $(DOC_PLUGINS); \
+	do \
+		PYTHONPATH=../../lib $(FORMATTER) -t rst --plugin-type $$plugin --template-dir=../templates --module-dir=../../lib/ansible/plugins/$$plugin -o rst/ $(PLUGIN_ARGS); \
+	done
 
 testing:
 	$(TESTING_FORMATTER)

docs/docsite/Makefile.sphinx

@@ -2,7 +2,7 @@
 #
 
 # You can set these variables from the command line.
-SPHINXOPTS    = -j $(CPUS)
+SPHINXOPTS    = -j $(CPUS) -n -w rst_warnings
 SPHINXBUILD   = sphinx-build
 SPHINXPROJ    = sdfsdf
 SOURCEDIR     = rst
@@ -17,4 +17,4 @@ help:
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile.sphinx
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/docsite/_themes/srtd/ansible_eol_banner.html

@@ -1,4 +0,0 @@
-{# Creates a banner at the top of the page for EOL versions. #}
-<div id='banner' class='Admonition caution'>
-  <p>You are reading an unmaintained version of the Ansible documentation. Unmaintained Ansible versions can contain unfixed security vulnerabilities (CVE). Please upgrade to a maintained version. See <a href="https://docs.ansible.com/ansible/latest/index.html">the latest Ansible documentation</a>.</p>
-</div>

docs/docsite/_themes/srtd/breadcrumbs.html

@@ -1,5 +1,11 @@
 <ul class="wy-breadcrumbs">
   <li><a href="{{ pathto(master_doc) }}">Docs</a> &raquo;</li>
   <li><a href="">{{ title }}</a></li>
+  {% if not pagename.endswith('_module') and (not 'list_of' in pagename) and (not 'category' in pagename) %}
+    <li class="wy-breadcrumbs-aside">
+      <a href="https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/{{ pagename }}.rst" class="icon icon-github"> Edit on GitHub</a>
+    </li>
+  {% endif %}
 </ul>
 <hr/>
+

docs/docsite/_themes/srtd/layout.html

@@ -33,7 +33,6 @@
   {% if favicon %}
     <link rel="shortcut icon" href="{{ pathto('_static/' + favicon, 1) }}"/>
   {% endif %}
-  <link rel="canonical" href="https://docs.ansible.com/ansible/latest/{{ pagename }}.html"/>
 
   {# CSS #}
   <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'>
@@ -159,14 +158,14 @@
         <a class="DocSiteProduct-header DocSiteProduct-header--core" href="/">
             <div class="DocSiteProduct-productName">
                 <div class="DocSiteProduct-logoText">
-                    Ansible
+                    Ansible  
                     <div class="DocSiteProduct-CurrentVersion" align="right">
-                      v2.4
+                      devel
                   </div>
                 </div>
             </div>
         </a>
-
+        
 
           <div class="DocSiteProduct-CheckVersionPara">For previous versions, see the  <a class="DocSiteProduct-versionheader" href="/#coreversionselect">documentation archive.</a></div>
       </div>
@@ -229,7 +228,6 @@
         </div> -->
 
           {% include "breadcrumbs.html" %}
-          {% include "ansible_eol_banner.html" %}
           <div id="page-content">
           {% block body %}{% endblock %}
           </div>

docs/docsite/keyword_desc.yml

@@ -1,68 +1,51 @@
-accelerate: "*DEPRECATED*, set to True to use accelerate connection plugin."
-accelerate_ipv6: "*DEPRECATED*, set to True to force accelerate plugin to use ipv6 for its connection."
-accelerate_port: "*DEPRECATED*, set to override default port use for accelerate connection."
+accelerate: DEPRECATED, set to True to use accelerate connection plugin.
+accelerate_ipv6: "DEPRECATED, set to True to force accelerate plugin to use ipv6 for it's connection."
+accelerate_port: DEPRECATED, set to override default port use for accelerate connection.
 action: "The 'action' to execute for a task, it normally translates into a C(module) or action plugin."
-args: "*DEPRECATED*, A secondary way to add arguments into a task. Takes a dictionary in which keys map to options and values."
+args: DEPRECATED, A secondary way to add arguments into a task, it takes a dictionary in which keys map to options and values .. well you get it.
 always: List of tasks, in a block, that execute no matter if there is an error in the block or not.
-always_run: "*DEPRECATED*, forces a task to run even in check mode. Use :term:`check_mode` directive instead."
+always_run: DEPRECATED, forces a task to run even in check mode, use :term:`check_mode` directive instead.
 any_errors_fatal: Force any un-handled task errors on any host to propagate to all hosts and end the play.
 async: Run a task asyncronouslly if the C(action) supports this.
 become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution.
 become_flags: A string of flag(s) to pass to the privilege escalation program when :term:`become` is True.
-become_method: Which method of privilege escalation to use (such as sudo or su).
-become_user: "User that you 'become' after using privilege escalation. The remote/login user must have permissions to become this user."
+become_method: Which method of privilege escalation to use. i.e. sudo/su/etc.
+become_user: "User that you 'become' after using privilege escalation, the remote/login user must have permissions to become this user."
 block: List of tasks in a block.
 changed_when: "Conditional expression that overrides the task's normal 'changed' status."
-check_mode: |
-    A boolean that controls if a task is executed in 'check' mode
-
-    .. seealso:: :ref:`check_mode_dry`
-
-connection: |
-    Allows you to change the connection plugin used for tasks to execute on the target.
-
-    .. seealso:: :ref:`using_connection`
-
-delay: Number of seconds to delay between retries. This setting is only used in combination with :term:`until`.
-delegate_facts: Boolean that allows you to apply facts to a delegated host instead of inventory_hostname.
-delegate_to: Host to execute task instead of the target (inventory_hostname). Connection vars from the delegated host will also be used for the task.
+check_mode: "A boolean that controls if a task is executed in 'check' mode"
+connection: Allows you to change the connection plugin used for tasks to execute on the target.
+delay: Number of seconds to delay between retries, this setting is only used in combination with :term:`until`.
+delegate_facts: Boolean that allows you to apply facts to delegated host instead of inventory_hostname.
+delegate_to: Host to execute task instead of the target (inventory_hostname), connection vars from the delegated host will also be used for the task.
 diff: "Toggle to make tasks return 'diff' information or not."
 environment: A dictionary that gets converted into environment vars to be provided for the task upon execution.
 fact_path: Set the fact path option for the fact gathering plugin controlled by :term:`gather_facts`.
 failed_when: "Conditional expression that overrides the task's normal 'failed' status."
-force_handlers: Will force notified handler execution for hosts even if they failed during the play. Will not trigger if the play itself fails.
+force_handlers: Will force notified handler execution for hosts even if they failed during the play, it will not trigger if the play itself fails.
 gather_facts: "A boolean that controls if the play will automatically run the 'setup' task to gather facts for the hosts."
 gather_subset: Allows you to pass subset options to the  fact gathering plugin controlled by :term:`gather_facts`.
 gather_timeout: Allows you to set the timeout for the fact gathering plugin controlled by :term:`gather_facts`.
-handlers: "A section with tasks that are treated as handlers, these won't get executed normally, only when notified after each section of tasks is complete."
+handlers: "A section with tasks that are treated as handlers, these won't get executed normally, only when notified. After each section of tasks is complete."
 hosts: "A list of groups, hosts or host pattern that translates into a list of hosts that are the play's target."
 ignore_errors: Boolean that allows you to ignore task failures and continue with play. It does not affect connection errors.
-loop: "Takes a list for the task to iterate over, saving each list element into the ``item`` variable (configurable via loop_control)"
-loop_control: |
-    Several keys here allow you to modify/set loop behaviour in a task.
-
-    .. seealso:: :ref:`loop_control`
-
+loop_control: "Several keys here allow you to modify/set loop behaviour in a task see http://docs.ansible.com/ansible/latest/playbooks_loops.html#loop-control for details."
 max_fail_percentage: can be used to abort the run after a given percentage of hosts in the current batch has failed.
-name: "Identifier. Can be used for documentation, in or tasks/handlers."
+name: "It's a name, works mostly for documentation, in the case of tasks/handlers it can be an identifier."
 no_log: Boolean that controls information disclosure.
-notify: "List of handlers to notify when the task returns a 'changed=True' status."
+notify: "list of handlers to notify when the task returns a 'changed=True' status."
 order: Controls the sorting of hosts as they are used for executing the play. Possible values are inventory (default), sorted, reverse_sorted, reverse_inventory and shuffle.
 poll: Sets the polling interval in seconds for async tasks (default 10s).
 port: Used to override the default port used in a connection.
 post_tasks: A list of tasks to execute after the :term:`tasks` section.
 pre_tasks: A list of tasks to execute before :term:`roles`.
-remote_user: User used to log into the target via the connection plugin.
+remote_user: User used to log into the target via the connection plugin. AKA login user.
 register: Name of variable that will contain task status and module return data.
 rescue: List of tasks in a :term:`block` that run if there is a task error in the main :term:`block` list.
 retries: "Number of retries before giving up in a :term:`until` loop. This setting is only used in combination with :term:`until`."
 roles: List of roles to be imported into the play
 run_once: Boolean that will bypass the host loop, forcing the task to execute on the first host available and will also apply any facts to all active hosts.
-serial: |
-    Explicitly define how Ansible batches the execution of the current play on the play's target
-
-    .. seealso:: :ref:`rolling_update_batch_size`
-
+serial: Defines the 'batch' of hosts to execute the current play until the end.
 strategy: Allows you to choose the connection plugin to use for the play.
 tags: Tags applied to the task or included tasks, this allows selecting subsets of tasks from the command line.
 tasks: Main list of tasks to execute in the play, they run after :term:`roles` and before :term:`post_tasks`.

docs/docsite/rst/become.rst

@@ -8,7 +8,7 @@ Ansible can use existing privilege escalation systems to allow a user to execute
 Become
 ``````
 Ansible allows you to 'become' another user, different from the user that logged into the machine (remote user). This is done using existing
-privilege escalation tools, which you probably already use or have configured, like `sudo`, `su`, `pfexec`, `doas`, `pbrun`, `dzdo`, `ksu`, and others.
+privilege escalation tools, which you probably already use or have configured, like `sudo`, `su`, `pfexec`, `doas`, `pbrun`, `dzdo`, `ksu` and others.
 
 
 .. note:: Before 1.9 Ansible mostly allowed the use of `sudo` and a limited use of `su` to allow a login/remote user to become a different user

docs/docsite/rst/command_line_tools.rst

@@ -1,15 +1,11 @@
 Command Line Tools
 ==================
 
+Most users are familiar with `ansible` and `ansible-playbook`, but those are not the only utilities Ansible provides.
+Below you is a complete list of Ansible utilities. Each page contains a description of the utility and a listing of supported parameters.	
 
 .. toctree:: :maxdepth: 1
+    :glob:
 
     ansible
-    ansible-playbook
-    ansible-vault
-    ansible-galaxy
-    ansible-console
-    ansible-config
-    ansible-doc
-    ansible-inventory
-    ansible-pull
+    ansible-*

docs/docsite/rst/committer_guidelines.rst

@@ -60,7 +60,7 @@ Individuals with direct commit access to ansible/ansible are entrusted with powe
   - Discuss with other committers, specially when you are unsure of something.
   - Document! If your PR is a new feature or a change to behavior, make sure you've updated all associated documentation or have notified the right people to do so. It also helps to add the version of Core against which this documentation is compatible (to avoid confusion with stable versus devel docs, for backwards compatibility, etc.).
   - Consider scope, sometimes a fix can be generalized
-  - Keep it simple, then things are maintainable, debuggable, and intelligible.
+  - Keep it simple, then things are maintainable, debuggable and intelligible.
 
 Committers are expected to continue to follow the same community and contribution guidelines followed by the rest of the Ansible community. 
 

docs/docsite/rst/community.rst

@@ -45,7 +45,7 @@ I'd Like To Help Share and Promote Ansible
 You can help share Ansible with others by telling friends and colleagues, writing a blog post,
 or presenting at user groups (like DevOps groups or the local LUG).
 
-You are also welcome to share slides on speakerdeck, sign up for a free account, and tag it "Ansible". On Twitter,
+You are also welcome to share slides on speakerdeck, sign up for a free account and tag it "Ansible". On Twitter,
 you can also share things with #ansible and may wish to `follow us <https://twitter.com/ansible>`_.
 
 I'd Like To Help Ansible Move Faster
@@ -171,7 +171,7 @@ Contributions can be for new features like modules, or to fix bugs you or others
 are interested in writing new modules to be included in the core Ansible distribution, please refer
 to the `module development documentation <http://docs.ansible.com/developing_modules.html>`_.
 
-Ansible's aesthetic encourages simple, readable code, and consistent,
+Ansible's aesthetic encourages simple, readable code and consistent,
 conservatively extending, backwards-compatible improvements. When contributing code to Ansible,
 please observe the following guidelines:
 
@@ -182,7 +182,7 @@ please observe the following guidelines:
 - We are not strictly compliant with PEP8. See :doc:`dev_guide/testing_pep8` for more information.
 
 You can also contribute by testing and revising other requests, especially if it is one you are interested
-in using. Please keep your comments clear, to the point, courteous, and constructive, tickets are not a
+in using. Please keep your comments clear and to the point, courteous and constructive, tickets are not a
 good place to start discussions (ansible-devel and IRC exist for this).
 
 Tip: To easily run from a checkout, source ``./hacking/env-setup`` and that's it -- no install
@@ -194,7 +194,7 @@ Other Topics
 Ansible Staff
 -------------
 
-Ansible, Inc. is a company supporting Ansible and building additional solutions based on
+Ansible, Inc is a company supporting Ansible and building additional solutions based on
 Ansible.  We also do services and support for those that are interested. We also offer an
 enterprise web front end to Ansible (see Tower below).
 
@@ -310,7 +310,7 @@ All Ansible events and participants therein are governed by this Code of Conduct
 anti-harassment policy. We expect organizers to enforce these guidelines throughout all events,
 and we expect attendees, speakers, sponsors, and volunteers to help ensure a safe
 environment for our whole community. Specifically, this Code of Conduct covers
-participation in all Ansible-related forums, mailing lists, code, documentation
+participation in all Ansible-related forums and mailing lists, code and documentation
 contributions, public IRC channels, private correspondence, and public meetings.
 
 Ansible community members are...
@@ -341,7 +341,7 @@ rudeness, hostility, threatening behavior, abuse (verbal or physical), or person
 **Kind**
 
 Everyone should feel welcome in the Ansible community, regardless of their background.
-Please be courteous, respectful, and polite to fellow community members. Do not make or
+Please be courteous, respectful and polite to fellow community members. Do not make or
 post offensive comments related to skill level, gender, gender identity or expression,
 sexual orientation, disability, physical appearance, body size, race, or religion.
 Sexualized images or imagery, real or implied violence, intimidation, oppression,

docs/docsite/rst/community/maintainers.rst

@@ -15,13 +15,13 @@ In addition to the information below, module maintainers should be familiar with
 Maintainer Responsibilities
 ===========================
 
-When you contribute a new module to the `ansible/ansible <https://github.com/ansible/ansible>`_ repository, you become the maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
+When you contribute a new module to the [ansible/ansible](https://github.com/ansible/ansible) repository, you become the maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
 
 Maintainers of Ansible modules are expected to provide feedback, responses, or actions on pull requests or issues to the module(s) they maintain in a reasonably timely manner.
 
-It is also recommended that you occasionally revisit the `contribution guidelines <https://github.com/ansible/ansible/blob/devel/CONTRIBUTING.md>`_, as they are continually refined. Occasionally, you may be requested to update your module to move it closer to the general accepted standard requirements. We hope for this to be infrequent, and will always be a request with a fair amount of lead time (ie: not by tomorrow!).
+It is also recommended that you occasionally revisit the [contribution guidelines](https://github.com/ansible/ansible/blob/devel/CONTRIBUTING.md), as they are continually refined. Occasionally, you may be requested to update your module to move it closer to the general accepted standard requirements. We hope for this to be infrequent, and will always be a request with a fair amount of lead time (ie: not by tomorrow!).
 
-Finally, following the `ansible-devel <https://groups.google.com/forum/#!forum/ansible-devel>`_ mailing list can be a great way to participate in the broader Ansible community, and a place where you can influence the overall direction, quality, and goals of Ansible and its modules. If you're not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
+Finally, following the [ansible-devel](https://groups.google.com/forum/#!forum/ansible-devel) mailing list can be a great way to participate in the broader Ansible community, and a place where you can influence the overall direction, quality, and goals of Ansible and its modules. If you're not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
 
 The Ansible community hopes that you will find that maintaining your module is as rewarding for you as having the module is for the wider community.
 
@@ -31,21 +31,21 @@ Pull Requests, Issues, and Workflow
 Pull Requests
 -------------
 
-Module pull requests are located in the `main Ansible repository <https://github.com/ansible/ansible/pulls>`_.
+Module pull requests are located in the [main Ansible repository](https://github.com/ansible/ansible/pulls).
 
 Because of the high volume of pull requests, notification of PRs to specific modules are routed by an automated bot to the appropriate maintainer for handling. It is recommended that you set an appropriate notification process to receive notifications which mention your GitHub ID.
 
 Issues
 ------
 
-Issues for modules, including bug reports, documentation bug reports, and feature requests, are tracked in the `ansible repository <https://github.com/ansible/ansible/issues>`_.
+Issues for modules, including bug reports, documentation bug reports, and feature requests, are tracked in the [ansible repository](https://github.com/ansible/ansible/issues).
 
 Issues for modules are routed to their maintainers via an automated process. This process is still being refined, and currently depends upon the issue creator to provide adequate details (specifically, providing the proper module name) in order to route it correctly. If you are a maintainer of a specific module, it is recommended that you periodically search module issues for issues which mention your module's name (or some variation on that name), as well as setting an appropriate notification process for receiving notification of mentions of your GitHub ID.
 
 PR Workflow
 -----------
 
-Automated routing of pull requests is handled by a tool called `Ansibot <https://github.com/ansible/ansibullbot>`_.
+Automated routing of pull requests is handled by a tool called [Ansibot](https://github.com/ansible/ansibullbot). 
 
 Being moderately familiar with how the workflow behind the bot operates can be helpful to you, and -- should things go awry -- your feedback can be helpful to the folks that continually help Ansibullbot to evolve.
 

docs/docsite/rst/community/other_tools_and_programs.rst

@@ -16,3 +16,5 @@ of some of the most popular of these tools.
 - `Ansigenome <https://github.com/nickjj/ansigenome>`_ is a command line tool designed to help you manage your Ansible roles.
 
 - `Awesome Ansible <https://github.com/jdauphant/awesome-ansible>`_ is a collaboratively curated list of awesome Ansible resources.
+
+- `Ansible Inventory Grapher <http://github.com/willthames/ansible-inventory-grapher>_` can be used to visually display inventory inheritance hierarchies and at what level a variable is defined in inventory.

docs/docsite/rst/dev_guide/developing_modules_documenting.rst

@@ -41,7 +41,8 @@ code.
 .. code-block:: python
 
     #!/usr/bin/python
-    # Copyright (c) 2017 Ansible Project
+    
+    # Copyright: (c) 2017, Ansible Project
     # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 Every file should have a copyright line with the original copyright holder.
@@ -61,8 +62,9 @@ add the newer line above the older one, like so:
 .. code-block:: python
 
     #!/usr/bin/python
-    # Copyright (c) 2017 [New Contributor(s)]
-    # Copyright (c) 2015 [Original Contributor(s)]
+    
+    # Copyright: (c) 2017, [New Contributor(s)]
+    # Copyright: (c) 2015, [Original Contributor(s)]
     # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 
@@ -84,11 +86,11 @@ For new modules, the following block can be simply added into your module
    * ``metadata_version`` is the version of the ``ANSIBLE_METADATA`` schema, *not* the version of the module.
    * Promoting a module's ``status`` or ``supported_by`` status should only be done by members of the Ansible Core Team.
 
-.. note:: Pre-released metdata version
+.. note:: Pre-released metadata version
 
     During development of Ansible-2.3, modules had an initial version of the
     metadata.  This version was modified slightly after release to fix some
-    points of confusion.  You may occassionally see PRs for modules where the
+    points of confusion.  You may occasionally see PRs for modules where the
     ANSIBLE_METADATA doesn't look quite right because of this.  Module
     metadata should be fixed before checking it into the repository.
 
@@ -335,8 +337,30 @@ Example::
         returned: when supported
         type: string
         sample: 2a5aeecc61dc98c4d780b14b330e3282
-    ...
+    '''
 
+    RETURN = '''
+    packages:
+        description: Information about package requirements
+        returned: On success
+        type: complex
+        contains:
+            missing:
+                description: Packages that are missing from the system
+                returned: success
+                type: list
+                sample:
+                    - libmysqlclient-dev
+                    - libxml2-dev
+            badversion:
+                description: Packages that are installed but at bad versions.
+                returned: success
+                type: list
+                sample:
+                    - package: libxml2-dev
+                      version: 2.9.4+dfsg1-2
+                      constraint: ">= 3.0"
+    '''
 
 .. note::
 

docs/docsite/rst/dev_guide/developing_modules_general.rst

@@ -312,5 +312,5 @@ use the ``#ansible`` channel.
 Credit
 ======
 
-Thank you to Thomas Stringer (`@tstring <https://github.com/tstringer>`_) for contributing source
+Thank you to Thomas Stringer (`@tstringer <https://github.com/tstringer>`_) for contributing source
 material for this topic.

docs/docsite/rst/dev_guide/developing_modules_general_windows.rst

@@ -38,52 +38,17 @@ When creating a new module there are a few things to keep in mind:
 - Use the full cmdlet name instead of aliases, e.g. ``Remove-Item`` over ``rm``
 - Use named parameters with cmdlets, e.g. ``Remove-Item -Path C:\temp`` over ``Remove-Item C:\temp``
 
-A very basic powershell module template can be found found below:
+A very basic powershell module `win_environment <https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/windows/win_environment.ps1>`_ is included below. It demonstrates how to implement check-mode and diff-support, and also shows a warning to the user when a specific condition is met.
 
-.. code-block:: powershell
+.. .. include:: ../../../../lib/ansible/modules/windows/win_environment.ps1
+..    :code: powershell
 
-    #!powershell
-    # This file is part of Ansible
+.. literalinclude:: ../../../../lib/ansible/modules/windows/win_environment.ps1
+   :language: powershell
 
-    # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+A slightly more advanced module is `win_uri <https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/windows/win_uri.ps1>`_ which additionally shows how to use different parameter types (bool, str, int, list, dict, path) and a selection of choices for parameters, how to fail a module and how to handle exceptions.
 
-    #Requires -Module Ansible.ModuleUtils.Legacy.psm1
-
-    $ErrorActionPreference = 'Stop'
-
-    $params = Parse-Args -arguments $args -supports_check_mode $true
-    $check_mode = Get-AnsibleParam -obj $params -name "_ansible_check_mode" -type "bool" -default $false
-    $diff_mode = Get-AnsibleParam -obj $params -name "_ansible_diff" -type "bool" -default $false
-
-    # these are your module parameters, there are various types which can be
-    # used to format your parameters. You can also set mandatory parameters
-    # with -failifempty, set defaults with -default and set choices with
-    # -validateset.
-    $string = Get-AnsibleParam -obj $params -name "string" -type "str" -failifempty $true
-    $bool = Get-AnsibleParam -obj $params -name "bool" -type "bool" -default $false
-    $int = Get-AnsibleParam -obj $params -name "int" -type "int"
-    $path = Get-AnsibleParam -obj $params -name "path" -type "path"
-    $list = Get-AnsibleParam -obj $params -name "list" -type "list"
-    $choices = Get-AnsibleParam -obj $params -name "choices" -type "str" -default "present" -validateset "absent","present"
-
-    $result = @{
-        changed = $false
-    }
-
-    if ($diff_mode) {
-        $result.diff = @{}
-    }
-
-    # code goes here
-
-    # you can add/set new result objects with
-    $result.changed = $true
-    $result.new_result = "Hi"
-
-    Exit-Json -obj $result
-
-
-When in doubt, look at some of the core modules and see how things have been
+When in doubt, look at some of the other core modules and see how things have been
 implemented there.
 
 Sometimes there are multiple ways that Windows offers to complete a task; this

docs/docsite/rst/dev_guide/developing_plugins.rst

@@ -3,23 +3,50 @@ Developing Plugins
 
 .. contents:: Topics
 
-Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own.
+Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own. This section describes the various types of Ansible  plugins and how to implement them.
 
-The following types of plugins are available:
+.. _plugin_guidelines:
 
-- *Action* plugins are front ends to modules and can execute actions on the controller before calling the modules themselves.
-- *Cache* plugins are used to keep a cache of 'facts' to avoid costly fact-gathering operations.
-- *Callback* plugins enable you to hook into Ansible events for display or logging purposes.
-- *Connection* plugins define how to communicate with inventory hosts.
-- *Filters* plugins allow you to manipulate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra filter plugins.
-- *Lookup* plugins are used to pull data from an external source. These are implemented using a custom Jinja2 function.
-- *Strategy* plugins control the flow of a play and execution logic.
-- *Shell* plugins deal with low-level commands and formatting for the different shells Ansible can encounter on remote hosts.
-- *Test* plugins allow you to validate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra test plugins.
-- *Vars* plugins inject additional variable data into Ansible runs that did not come from an inventory, playbook, or the command line.
+General Guidelines
+------------------
 
-This section describes the various types of plugins and how to implement them.
+This section lists some things that should apply to any type of plugin you develop.
 
+Raising Errors
+``````````````
+
+In general, errors encountered during execution should be returned by raising AnsibleError() or similar class with a message describing the error. When wrapping other exceptions into error messages, you should always use the `to_text` Ansible function to ensure proper string compatibility across Python versions:
+
+.. code-block:: python
+
+    from ansible.module_utils._text import to_native
+
+    try:
+        cause_an_exception()
+    except Exception as e:
+        AnsibleError('Something happened, this was original exception: %s' % to_native(e))
+
+Check the different AnsibleError objects and see which one applies the best to your situation.
+
+String Encoding
+```````````````
+Any strings returned by your plugin that could ever contain non-ASCII characters must be converted into Python's unicode type because the strings will be run through jinja2.  To do this, you can use:
+
+.. code-block:: python
+
+    from ansible.module_utils._text import to_text
+    result_string = to_text(result_string)
+
+Plugin Configuration
+````````````````````
+
+Starting with Ansible version 2.4, we are unifying how each plugin type is configured and how they get those settings.  Plugins will be able to declare their requirements and have Ansible provide them with a resolved'configuration. Starting with Ansible 2.4 both callback and connection type plugins can use this system. 
+
+Most plugins will be able to use  `self._options[<optionname>]` to access the settings, except callbacks that use `self._plugin_options[<optionname>]`.
+
+Plugins that support embedded documentation (see `ansible-doc` for the list) are now required to provide well-formed doc strings to be considered for merge into the Ansible repo.
+
+If you inherit from a plugin, you must document the options it takes, either via a documentation fragment or as a copy.
 
 .. _developing_callbacks:
 
@@ -28,75 +55,50 @@ Callback Plugins
 
 Callback plugins enable adding new behaviors to Ansible when responding to events. By default, callback plugins control most of the output you see when running the command line programs.
 
-.. _callback_examples:
-
-Example Callback Plugins
-++++++++++++++++++++++++
-
-Ansible comes with a number of callback plugins that you can look at for examples. These can be found in `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_.
-
-The `log_plays
-<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/log_plays.py>`_
-callback is an example of how to intercept playbook events to a log
-file, and the `mail
-<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/mail.py>`_
-callback sends email when playbooks complete.
-
-The `osx_say
-<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/osx_say.py>`_
-callback provided is particularly entertaining -- it will respond with
-computer synthesized speech on OS X in relation to playbook events,
-and is guaranteed to entertain and/or annoy coworkers.
-
-.. _configuring_callbacks:
-
-Configuring Callback Plugins
-++++++++++++++++++++++++++++
-
-You can activate a custom callback by either dropping it into a callback_plugins directory adjacent to your play or inside a role or by putting it in one of the callback directory sources configured in `ansible.cfg`.
-
-Plugins are loaded in alphanumeric order; for example, a plugin implemented in a file named `1_first.py` would run before a plugin file named `2_second.py`.
-
-Most callbacks shipped with Ansible are disabled by default and need to be whitelisted in your `ansible.cfg` file in order to function. For example::
-
-  #callback_whitelist = timer, mail, mycallbackplugin
-
-
-Managing stdout
-```````````````
-
-You can only have one plugin be the main manager of your console output. If you want to replace the default, you should define CALLBACK_TYPE = stdout in the subclass and then configure the stdout plugin in `ansible.cfg`. For example::
-
-  #stdout_callback = mycallbackplugin
-
-
-
-.. _callback_development:
-
-Developing Callback Plugins
-+++++++++++++++++++++++++++
-
 Callback plugins are created by creating a new class with the Base(Callbacks) class as the parent:
 
 .. code-block:: python
 
   from ansible.plugins.callback import CallbackBase
-  from ansible import constants as C
 
   class CallbackModule(CallbackBase):
       pass
 
-From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`. For a complete list of methods that you can override, please see ``__init__.py`` in the `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_ directory.
+From there, override the specific methods from the CallbackBase that you want to provide a callback for.
+For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`.
+For a complete list of methods that you can override, please see ``__init__.py`` in the
+`lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_ directory.
 
 
-The following example shows how Ansible's timer plugin is implemented:
+The following is a modified example of how Ansible's timer plugin is implemented,
+but with an extra option so you can see how configuration works in Ansible version 2.4 and later:
 
 .. code-block:: python
 
-  # Make coding more python3-ish
+  # Make coding more python3-ish, this is required for contributions to Ansible
   from __future__ import (absolute_import, division, print_function)
   __metaclass__ = type
 
+  # not only visible to ansible-doc, it also 'declares' the options the plugin requires and how to configure them.
+  DOCUMENTATION = '''
+    callback: timer
+    callback_type: aggregate
+    requirements:
+      - whitelist in configuration
+    short_description: Adds time to play stats
+    version_added: "2.0"
+    description:
+        - This callback just adds total play duration to the play stats.
+    options:
+      format_string:
+        description: format of the string shown to user at play end
+        ini:
+          - section: callback_timer
+            key: format_string
+        env:
+          - name: ANSIBLE_CALLBACK_TIMER_FORMAT
+        default: "Playbook run took %s days, %s hours, %s minutes, %s seconds"
+  '''
   from datetime import datetime
 
   from ansible.plugins.callback import CallbackBase
@@ -109,40 +111,86 @@ The following example shows how Ansible's timer plugin is implemented:
       CALLBACK_VERSION = 2.0
       CALLBACK_TYPE = 'aggregate'
       CALLBACK_NAME = 'timer'
+
+      # only needed if you ship it and don't want to enable by default
       CALLBACK_NEEDS_WHITELIST = True
 
       def __init__(self):
 
+          # make sure the expected objects are present, calling the base's __init__
           super(CallbackModule, self).__init__()
 
+          # start the timer when the plugin is loaded, the first play should start a few milliseconds after.
           self.start_time = datetime.now()
 
-      def days_hours_minutes_seconds(self, runtime):
+      def _days_hours_minutes_seconds(self, runtime):
+          ''' internal helper method for this callback '''
           minutes = (runtime.seconds // 60) % 60
           r_seconds = runtime.seconds - (minutes * 60)
           return runtime.days, runtime.seconds // 3600, minutes, r_seconds
 
-      def playbook_on_stats(self, stats):
-          self.v2_playbook_on_stats(stats)
-
+      # this is only event we care about for display, when the play shows its summary stats; the rest are ignored by the base class
       def v2_playbook_on_stats(self, stats):
           end_time = datetime.now()
           runtime = end_time - self.start_time
-          self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds" % (self.days_hours_minutes_seconds(runtime)))
 
-Note that the CALLBACK_VERSION and CALLBACK_NAME definitions are required for properly functioning plugins for Ansible >=2.0.
+          # Shows the usage of a config option declared in the DOCUMENTATION variable. Ansible will have set it when it loads the plugin.
+          # Also note the use of the display object to print to screen. This is available to all callbacks, and you should use this over printing yourself
+          self._display.display(self._plugin_options['format_string'] % (self._days_hours_minutes_seconds(runtime)))
+
+Note that the CALLBACK_VERSION and CALLBACK_NAME definitions are required for properly functioning plugins for Ansible version 2.0 and later. CALLBACK_TYPE is mostly needed to distinguish 'stdout' plugins from the rest, since you can only load one plugin that writes to stdout.
 
 .. _developing_connection_plugins:
 
 Connection Plugins
 ------------------
 
-By default, Ansible ships with a 'paramiko' SSH, native ssh (just called 'ssh'), 'local' connection type, and there are also some minor players like 'chroot' and 'jail'.  All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines.  The basics of these connection types
-are covered in the :doc:`../intro_getting_started` section.  Should you want to extend Ansible to support other transports (SNMP, Message bus, etc) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins
-directory.   The value of 'smart' for a connection allows selection of paramiko or openssh based on system capabilities, and chooses
-'ssh' if OpenSSH supports ControlPersist, in Ansible 1.2.1 and later.  Previous versions did not support 'smart'.
+Connection plugins allow Ansible to connect to the target hosts so it can execute tasks on them. Ansible ships with many connection plugins, but only one can be used per host at a time.
 
-More documentation on writing connection plugins is pending, though you can jump into `lib/ansible/plugins/connection <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/connection>`_ and figure things out pretty easily.
+By default, Ansible ships with several plugins. The most commonly used are the 'paramiko' SSH, native ssh (just called 'ssh'), and 'local' connection types.  All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines.  
+
+The basics of these connection types are covered in the :doc:`../intro_getting_started` section.  
+
+Should you want to extend Ansible to support other transports (SNMP, Message bus, etc) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins directory.  
+
+Ansible version 2.1 introduced the 'smart' connection plugin. The 'smart' connection type allows Ansible to automatically select either the 'paramiko' or 'openssh' connection plugin based on system capabilities, or the 'ssh' connection plugin if OpenSSH supports ControlPersist. 
+
+For examples on how to implement a connection plug in, see the source code here: 
+`lib/ansible/plugins/connection <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/connection>`_.
+
+.. _developing_inventory_plugins:
+
+Inventory Plugins
+-----------------
+
+Inventory plugins were added in Ansible version 2.4. Inventory plugins parse inventory sources and form an in memory representation of the inventory.
+
+Inventory plugins are invoked via the InventoryManager and are given access to any existing inventory data. They are given an 'inventory source' as supplied to Ansible (via config/options/defaults/etc), which they can either ignore
+by returning false from the `verify_file` method, or attempting to parse (with the `parse` method) and return an `AnsibleParserError` on failure.
+
+.. code-block:: python
+
+   def parse(self, inventory, loader, path, cache=True):
+        pass # your code goes here
+
+Inventory plugins take the following parameters:
+
+ * inventory: inventory object with existing data and the methods to add hosts/groups/variables to inventory
+ * loader: Ansible's DataLoader. The DataLoader can read files, auto load JSON/YAML and decrypt vaulted data, and cache read files.
+ * path: string with inventory source (this is usually a path, but is not required)
+ * cache: indicates whether the plugin should use or avoid caches (cache plugin and/or loader)
+
+Inventory sources are strings. They usually correspond to a file path, but they can also be a comma separated list,
+a URI, or anything your plugin can use as input.
+The 'inventory source' provided can be either a string (`host_list` plugin), a data file (like consumed by the `yaml` and `ini` plugins), a configuration file (see `virtualbox` and `constructed`) or even a script or executable (the `script` uses those).
+
+When using the 'persistent' cache, inventory plugins can also use the configured cache plugin to store and retrieve data to avoid costly external calls. 
+
+Inventory plugins normally only execute at the start of a run, before playbooks/plays and roles are found,
+but they can be 're-executed' via the `meta: refresh_inventory` task, which will clear out the existing inventory and rebuild it.
+
+For examples on how to implement a connection plug in, see the source code here:
+`lib/ansible/plugins/inventory <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/inventory>`_.
 
 .. _developing_lookup_plugins:
 
@@ -155,6 +203,25 @@ Here's a simple lookup plugin implementation - this lookup returns the contents
 
 .. code-block:: python
 
+  # python 3 headers, required if submitting to Ansible
+  from __future__ import (absolute_import, division, print_function)
+  __metaclass__ = type
+
+  DOCUMENTATION = """
+        lookup: file
+          author: Daniel Hokka Zakrisson <daniel@hozac.com>
+          version_added: "0.9"
+          short_description: read file contents
+          description:
+              - This lookup returns the contents from a file on the Ansible controller's file system.
+          options:
+            _terms:
+              description: path(s) of files to read
+              required: True
+          notes:
+            - if read in variable context, the file can be interpreted as YAML if the content is valid to the parser.
+            - this lookup does not understand 'globing' - use the fileglob lookup instead.
+  """
   from ansible.errors import AnsibleError, AnsibleParserError
   from ansible.plugins.lookup import LookupBase
 
@@ -169,26 +236,35 @@ Here's a simple lookup plugin implementation - this lookup returns the contents
 
       def run(self, terms, variables=None, **kwargs):
 
-          ret = []
 
+          # lookups in general are expected to both take a list as input and output a list
+          # this is done so they work with the looping construct `with_`.
+          ret = []
           for term in terms:
               display.debug("File lookup term: %s" % term)
 
-              # Find the file in the expected search path
+              # Find the file in the expected search path, using a class method
+              # that implements the 'expected' search path for Ansible plugins.
               lookupfile = self.find_file_in_search_path(variables, 'files', term)
+
+              # Don't use print or your own logging, the display class
+              # takes care of it in a unified way.
               display.vvvv(u"File lookup using %s as file" % lookupfile)
               try:
                   if lookupfile:
                       contents, show_data = self._loader._get_file_contents(lookupfile)
                       ret.append(contents.rstrip())
                   else:
+                      # Always use ansible error classes to throw 'final' exceptions,
+                      # so the Ansible engine will know how to deal with them.
+                      # The Parser error indicates invalid options passed
                       raise AnsibleParserError()
               except AnsibleParserError:
                   raise AnsibleError("could not locate file in lookup: %s" % term)
 
           return ret
 
-An example of how this lookup is called::
+The following is an example of how this lookup is called::
 
   ---
   - hosts: all
@@ -199,30 +275,48 @@ An example of how this lookup is called::
 
        - debug: msg="the value of foo.txt is {{ contents }} as seen today {{ lookup('pipe', 'date +"%Y-%m-%d"') }}"
 
-Errors encountered during execution should be returned by raising AnsibleError() with a message describing the error. Any strings returned by your lookup plugin implementation that could ever contain non-ASCII characters must be converted into Python's unicode type because the strings will be run through jinja2.  To do this, you can use:
-
-.. code-block:: python
-
-    from ansible.module_utils._text import to_text
-    result_string = to_text(result_string)
-
 For more example lookup plugins, check out the source code for the lookup plugins that are included with Ansible here: `lib/ansible/plugins/lookup <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/lookup>`_.
 
-For usage examples of lookup plugins, see `Using Lookups <http://docs.ansible.com/ansible/playbooks_lookups.html>`_.
+For more usage examples of lookup plugins, see `Using Lookups <http://docs.ansible.com/ansible/playbooks_lookups.html>`_.
 
 .. _developing_vars_plugins:
 
 Vars Plugins
 ------------
 
-Playbook constructs like 'host_vars' and 'group_vars' work via 'vars' plugins.  They inject additional variable
-data into ansible runs that did not come from an inventory, playbook, or command line.  Note that variables
-can also be returned from inventory, so in most cases, you won't need to write or understand vars_plugins.
+Vars plugins inject additional variable data into Ansible runs that did not come from an inventory source, playbook, or command line. Playbook constructs like 'host_vars' and 'group_vars' work using vars plugins.
 
-More documentation on writing vars plugins is pending, though you can jump into `lib/ansible/plugins <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/>`_ and figure
-things out pretty easily.
+Vars plugins were partially implemented in Ansible 2.0 and rewritten to be fully implemented starting with Ansible 2.4.
+
+Older plugins used a `run` method as their main body/work:
+
+.. code-block:: python
+
+    def run(self, name, vault_password=None):
+        pass # your code goes here
+
+
+Ansible 2.0 did not pass passwords to older plugins, so vaults were unavailable.
+Most of the work now  happens in the `get_vars` method which is called from the VariableManager when needed.
+
+.. code-block:: python
+
+    def get_vars(self, loader, path, entities):
+        pass # your code goes here
+
+The parameters are:
+
+ * loader: Ansible's DataLoader. The DataLoader can read files, auto load JSON/YAML and decrypt vaulted data, and cache read files.
+ * path: this is 'directory data' for every inventory source and the current play's playbook directory, so they can search for data in reference to them. `get_vars` will be called at least once per available path.
+ * entities: these are host or group names that are pertinent to the variables needed. The plugin will get called once for hosts and again for groups.
+
+This `get vars` method just needs to return a dictionary structure with the variables.
+
+Since Ansible version 2.4, vars plugins only execute as needed when preparing to execute a task. This avoids the costly 'always execute' behavior that occurred during inventory construction in older versions of Ansible.
+
+For implementation examples of vars plugins, check out the source code for the vars plugins that are included with Ansible:
+`lib/ansible/plugins/vars <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/vars>`_  .
 
-If you find yourself wanting to write a vars_plugin, it's more likely you should write an inventory script instead.
 
 .. _developing_filter_plugins:
 
@@ -255,11 +349,14 @@ Plugins are automatically loaded when you have one of the following subfolders a
     * lookup_plugins
     * callback_plugins
     * connection_plugins
+    * inventory_plugins
     * filter_plugins
     * strategy_plugins
     * cache_plugins
     * test_plugins
     * shell_plugins
+    * vars_plugins
+
 
 When shipped as part of a role, the plugin will be available as soon as the role is called in the play.
 

docs/docsite/rst/dev_guide/developing_program_flow_modules.rst

@@ -277,7 +277,7 @@ substitutions:
 
   - :code:`"<<ANSIBLE_VERSION>>"` is substituted with the Ansible version.  In
     :ref:`new-style Python modules <flow_python_modules>` under the
-    :ref:`Ansiballz` frameworkthe proper way is to instead instantiate an
+    :ref:`Ansiballz` framework the proper way is to instead instantiate an
     :class:`AnsibleModule` and then access the version from
     :attr:``AnsibleModule.ansible_version``.
   - :code:`"<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>"` is substituted with

docs/docsite/rst/dev_guide/repomerge.rst

@@ -27,19 +27,3 @@ As part of this move, we will be introducing module metadata, which will contain
 
 
 The documentation pages for modules will be updated to reflect the above information as well, so that users can evaluate the status of a module before committing to using it in playbooks and roles.
-
-
-.. _PRMover:
-
-Move Issues and PRs to new Repo
--------------------------------
-A tool has been developed to move a PR from the old repos to `ansible/ansible` this can be found at `prmover tool <https://prmover.pythonanywhere.com/>`_ 
-
-Before using prmover please ensure you have a fork of the Ansible repo.
-
-To move issues please use `GitHub Issue Mover <https://github-issue-mover.appspot.com/>`_
-
-If you have *any* issues with updating your PR please ask for support in `#ansible-devel`
-
-For support please use `#ansible-devel` on Freenode IRC
-

docs/docsite/rst/dev_guide/style_guide/grammar_punctuation.rst

@@ -55,7 +55,7 @@ Commas are always important, considering the vast difference in meanings of the
 - Let's eat, Grandma
 - Let's eat Grandma.
 
-Correct punctation could save Grandma's life.
+Correct punctuation could save Grandma's life.
 
 If that does not convince you, maybe this will:
 

docs/docsite/rst/dev_guide/style_guide/resources.rst

@@ -2,17 +2,15 @@ Resources
 ````````````````
 Internal resources
 -------------------------
-- http://docs.ansible.com/
+- https://docs.ansible.com/
 - https://sites.google.com/a/ansibleworks.com/ansible-intranet/
 - ??? intranet Engineering doc???
 
 
 External Resources
 -------------------------
-- www.apstylebook.com
-- www.chicagomanualofstyle.org—home.html
-- www.crockford.com—style.html
-- orwell.ru—index.htm
-- www.sun.com—sun_tech_pub.xml
-- webopedia.internet.com
-- www.computeruser.com—index.html
+- https://www.apstylebook.com
+- http://www.chicagomanualofstyle.org/home.html
+- https://www.crockford.com/wrrrld/style.html
+- http://orwell.ru
+- http://www.computeruser.com

docs/docsite/rst/dev_guide/style_guide/spelling_word_choice.rst

@@ -250,7 +250,7 @@ Correct. Do not use "trouble shoot" or "trouble-shoot." To isolate the source of
 
 UK
 ++++++++++++++
-Correst as is, no periods.
+Correct as is, no periods.
 
 UNIX®
 ++++++++++++++
@@ -262,7 +262,7 @@ Don't use. Use Clear.
 
 US
 ++++++++++++++
-Correst as is, no periods.
+Correct as is, no periods.
 
 User
 ++++++++++++++

docs/docsite/rst/dev_guide/style_guide/trademarks.rst

@@ -87,10 +87,10 @@ Other Common Trademarks and Resource Sites:
 ++++++++++++++++++++++++++++++++++++++++++++++++
 - Linux is a registered trademark of Linus Torvalds.
 - UNIX® is a registered trademark of The Open Group.
-- Microsoft, Windows, Vista, XP, and NT are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. http://members.microsoft.com—en-us.mspx
-- Apple, Mac, Mac OS, Macintosh, Pages and TrueType are either registered trademarks or trademarks of Apple Computer, Inc. in the United States and/or other countries. http://www.apple.com—appletmlist.html
-- Adobe, Acrobat, GoLive, InDesign, Illustrator, PostScript , PhotoShop and the OpenType logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. htto:// www.adobe.com—trade.html
-- Macromedia and Macromedia Flash are trademarks of Macromedia, Inc. http://www.adobe.com—trademarkguideline.html
-- IBM is a registered trademark of International Business Machines Corporation. http://www.ibm.com—copytrade.shtml
-- Celeron, Celeron Inside, Centrino, Centrino logo, Core Inside, Intel Core, Intel Inside, Intel Inside logo, Itanium, Itanium Inside, Pentium, Pentium Inside,VTune, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. http://www.intel.com—tradmarx.htm
+- Microsoft, Windows, Vista, XP, and NT are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/en-us.aspx
+- Apple, Mac, Mac OS, Macintosh, Pages and TrueType are either registered trademarks or trademarks of Apple Computer, Inc. in the United States and/or other countries. https://www.apple.com/legal/intellectual-property/trademark/appletmlist.html
+- Adobe, Acrobat, GoLive, InDesign, Illustrator, PostScript , PhotoShop and the OpenType logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. http://www.adobe.com/legal/permissions/trademarks.html
+- Macromedia and Macromedia Flash are trademarks of Macromedia, Inc. http://www.adobe.com/legal/permissions/trademarks.html
+- IBM is a registered trademark of International Business Machines Corporation. https://www.ibm.com/legal/us/en/copytrade.shtml
+- Celeron, Celeron Inside, Centrino, Centrino logo, Core Inside, Intel Core, Intel Inside, Intel Inside logo, Itanium, Itanium Inside, Pentium, Pentium Inside,VTune, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. https://www.intel.com/content/www/us/en/legal/trademarks.html
 

docs/docsite/rst/dev_guide/testing.rst

@@ -33,14 +33,9 @@ At a high level we have the following classifications of tests:
   * Tests directly against individual parts of the code base.
 
 
-If you're a developer, one of the most valuable things you can do is look at the GitHub
-issues list and help fix bugs.  We almost always prioritize bug fixing over feature
-development.
+If you're a developer, one of the most valuable things you can do is look at the GitHub issues list and help fix bugs.  We almost always prioritize bug fixing over feature development, so helping to fix bugs is one of the best things you can do.
 
-Even for non developers, helping to test pull requests for bug fixes and features is still
-immensely valuable.  Ansible users who understand writing playbooks and roles should be
-able to add integration tests and so Github pull requests with integration tests that show
-bugs in action will also be a great way to help.
+Even if you're not a developer, helping to test pull requests for bug fixes and features is still immensely valuable.
 
 
 Testing within GitHub & Shippable
@@ -52,6 +47,7 @@ Organization
 
 When Pull Requests (PRs) are created they are tested using Shippable, a Continuous Integration (CI) tool. Results are shown at the end of every PR.
 
+
 When Shippable detects an error and it can be linked back to a file that has been modified in the PR then the relevant lines will be added as a GitHub comment. For example::
 
    The test `ansible-test sanity --test pep8` failed with the following errors:
@@ -73,6 +69,7 @@ Then run the tests detailed in the GitHub comment::
   ansible-test sanity --test pep8
   ansible-test sanity --test validate-modules
 
+
 If there isn't a GitHub comment stating what's failed you can inspect the results by clicking on the "Details" button under the "checks have failed" message at the end of the PR.
 
 Rerunning a failing CI job
@@ -94,6 +91,10 @@ If the issue persists, please contact us in ``#ansible-devel`` on Freenode IRC.
 How to test a PR
 ================
 
+If you're a developer, one of the most valuable things you can do is look at the GitHub issues list and help fix bugs.  We almost always prioritize bug fixing over feature development, so helping to fix bugs is one of the best things you can do.
+
+Even if you're not a developer, helping to test pull requests for bug fixes and features is still immensely valuable.
+
 Ideally, code should add tests that prove that the code works. That's not always possible and tests are not always comprehensive, especially when a user doesn't have access to a wide variety of platforms, or is using an API or web service. In these cases, live testing against real equipment can be more valuable than automation that runs against simulated interfaces. In any case, things should always be tested manually the first time as well.
 
 Thankfully, helping to test Ansible is pretty straightforward, assuming you are familiar with how Ansible works.
@@ -183,30 +184,12 @@ If the PR does not resolve the issue, or if you see any failures from the unit/i
    |
    |   \```
    |   some output
-   |   StrackTrace
+   |   StackTrace
    |   some other output
    |   \```
 
-Code Coverage Online
-````````````````````
-
-`The online code coverage reports <https://codecov.io/gh/ansible/ansible>` are a good way
-to identify areas for testing improvement in Ansible.  By following red colors you can
-drill down through the reports to find files which have no tests at all.  Adding both
-integration and unit tests which show clearly how code should work, verify important
-Ansible functions and increase testing coverage in areas where there is none is a valuable
-way to help improve Ansible.
-
-The code coverage reports only cover the ``devel`` branch of Ansible where new feature
-development takes place.  Pull requests and new code will be missing from the codecov.io
-coverage reports so local reporting is needed.  Most ``ansible-test`` commands allow you
-to collect code coverage, this is particularly useful to indicate where to extend
-testing. See :doc:`testing_running_locally` for more information.
-
-
 Want to know more about testing?
 ================================
 
-If you'd like to know more about the plans for improving testing Ansible then why not join the
-`Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
+If you'd like to know more about the plans for improving testing Ansible then why not join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
 

docs/docsite/rst/dev_guide/testing/sanity/ansible-var-precedence-check.rst

@@ -1,4 +1,4 @@
 Sanity Tests » ansible-var-precedence-check
 ===========================================
 
-Check the order of precedence for Ansible variables against :ref:`variable_precedence`.
+Check the order of precedence for Ansible variables against :ref:`ansible_variable_precedence`.

docs/docsite/rst/dev_guide/testing/sanity/azure-requirements.rst

@@ -1,10 +0,0 @@
-Sanity Tests » azure-requirements
-=================================
-
-Update the Azure integration test requirements file when changes are made to the Azure packaging requirements file:
-
-.. code-block:: bash
-
-    cp packaging/requirements/requirements-azure.txt test/runner/requirements/integration.cloud.azure.txt
-
-This copy of the requirements file is used when building the ``ansible/ansible:default`` Docker container from ``test/runner/Dockerfile``.

docs/docsite/rst/dev_guide/testing_running_locally.rst

@@ -45,19 +45,7 @@ Use the ``ansible-test shell`` command to get an interactive shell in the same e
 Code Coverage
 =============
 
-Code coverage reports make it easy to identify untested code for which more tests should
-be written.  Online reports are available but only cover the ``devel`` branch (see
-:doc:`testing`).  For new code local reports are needed.
-
-Add the ``--coverage`` option to any test command to collect code coverage data.  If you
-aren't using the ``--tox`` or ``--docker`` options which create an isolated python
-environment then you may have to use the ``--requirements`` option to ensure that the
-correct version of the coverage module is installed
-
-   ansible-test units --coverage apt
-   ansible-test integration --coverage aws_lambda --tox --requirements
-   ansible-test coverage html
-
+Add the ``--coverage`` option to any test command to collect code coverage data.
 
 Reports can be generated in several different formats:
 
@@ -65,7 +53,4 @@ Reports can be generated in several different formats:
 * ``ansible-test coverage html`` - HTML report.
 * ``ansible-test coverage xml`` - XML report.
 
-To clear data between test runs, use the ``ansible-test coverage erase`` command. For a full list of features see the online help::
-
-   ansible-test coverage --help
-
+To clear data between test runs, use the ``ansible-test coverage erase`` command.

docs/docsite/rst/dev_guide/testing_units.rst

@@ -2,24 +2,19 @@
 Unit Tests
 **********
 
-Unit tests are small isolated tests that target a specific library or module.  Unit tests
-in Ansible are currently the only way of driving tests from python within Ansible's
-continuous integration process. This means that in some circumstances the tests may be a
-bit wider than just units.
+Unit tests are small isolated tests that target a specific library or module.
 
 .. contents:: Topics
 
 Available Tests
 ===============
 
-Unit tests can be found in `test/units
-<https://github.com/ansible/ansible/tree/devel/test/units>`_. Notice that the directory
-structure of the tests matches that of ``lib/ansible/``.
+Unit tests can be found in `test/units <https://github.com/ansible/ansible/tree/devel/test/units>`_, notice that the directory structure matches that of ``lib/ansible/``
 
 Running Tests
 =============
 
-The Ansible unit tests can be run across the whole code base by doing:
+Unit tests can be run across the whole code base by doing:
 
 .. code:: shell
 
@@ -40,22 +35,18 @@ Or against a specific Python version by doing:
    ansible-test units --tox --python 2.7 apt
 
 
+
 For advanced usage see the online help::
 
    ansible-test units --help
 
-You can also run tests in Ansible's continuous integration system by opening a pull
-request.  This will automatically determine which tests to run based on the changes made
-in your pull request.
-
 
 Installing dependencies
 =======================
 
-``ansible-test`` has a number of dependencies. For ``units`` tests we suggest using ``tox``.
+``ansible-test`` has a number of dependencies , for ``units`` tests we suggest using ``tox``
 
-The dependencies can be installed using the ``--requirements`` argument, which will
-install all the required dependencies needed for unit tests. For example:
+The dependencies can be installed using the ``--requirements`` argument, which will install all the required dependencies needed for unit tests. For example:
 
 .. code:: shell
 
@@ -67,11 +58,7 @@ install all the required dependencies needed for unit tests. For example:
    When using ``ansible-test`` with ``--tox`` requires tox >= 2.5.0
 
 
-The full list of requirements can be found at `test/runner/requirements
-<https://github.com/ansible/ansible/tree/devel/test/runner/requirements>`_. Requirements
-files are named after their respective commands. See also the `constraints
-<https://github.com/ansible/ansible/blob/devel/test/runner/requirements/constraints.txt>`_
-applicable to all commands.
+The full list of requirements can be found at `test/runner/requirements <https://github.com/ansible/ansible/tree/devel/test/runner/requirements>`_. Requirements files are named after their respective commands. See also the `constraints <https://github.com/ansible/ansible/blob/devel/test/runner/requirements/constraints.txt>`_ applicable to all commands.
 
 
 Extending unit tests
@@ -80,98 +67,22 @@ Extending unit tests
 
 .. warning:: What a unit test isn't
 
-   If you start writing a test that requires external services then
-   you may be writing an integration test, rather than a unit test.
-
-
-Structuring Unit Tests
-``````````````````````
-
-Ansible drives unit tests through `pytest <https://docs.pytest.org/en/latest/>`_. This
-means that tests can either be written a simple functions which are included in any file
-name like ``test_<something>.py`` or as classes. 
-
-Here is an example of a function::
-
-  #this function will be called simply because it is called test_*()
-
-  def test_add()
-      a = 10
-      b = 23
-      c = 33
-      assert a + b = c
-    
-Here is an example of a class::
-
-  import unittest:
-      
-  class AddTester(unittest.TestCase)
-      
-      def SetUp()
-          self.a = 10
-          self.b = 23
- 
-      # this function will 
-      def test_add()
-        c = 33
-        assert self.a + self.b = c
-
-     # this function will 
-      def test_subtract()
-        c = -13
-        assert self.a - self.b = c
-
-Both methods work fine in most circumstances; the function-based interface is simpler and
-quicker and so that's probably where you should start when you are just trying to add a
-few basic tests for a module.  The class-based test allows more tidy set up and tear down
-of pre-requisites, so if you have many test cases for your module you may want to refactor
-to use that.  
-
-Assertions using the simple ``assert`` function inside the tests will give give full
-information on the cause of the failure with a trace-back of functions called during the
-assertion.  This means that plain asserts are recommended over other external assertion
-libraries.
-
-A number of the unit test suites include functions that are shared between several
-modules, especially in the networking arena.  In these cases a file is created in the same
-directory, which is then included directly.
-
-
-Module test case common code
-````````````````````````````
-
-Keep common code as specific as possible within the `test/units/` directory structure. For
-example, if it's specific to testing Amazon modules, it should be in
-`test/units/modules/cloud/amazon/`. Don't import common unit test code from directories
-outside the current or parent directories.
-
-Don't import other unit tests from a unit test. Any common code should be in dedicated
-files that aren't themselves tests.
-
+   If you start writing a test that requires external services then you may be writing an integration test, rather than a unit test.
 
 Fixtures files
 ``````````````
 
-To mock out fetching results from devices, or provide other complex datastructures that
-come from external libraries, you can use ``fixtures`` to read in pre-generated data.
+To mock out fetching results from devices, you can use ``fixtures`` to read in pre-generated data.
 
 Text files live in ``test/units/modules/network/PLATFORM/fixtures/``
 
 Data is loaded using the ``load_fixture`` method
 
-See `eos_banner test
-<https://github.com/ansible/ansible/blob/devel/test/units/modules/network/eos/test_eos_banner.py>`_
-for a practical example.
+See  `eos_banner test <https://github.com/ansible/ansible/blob/devel/test/units/modules/network/eos/test_eos_banner.py>`_ for a practical example.
 
-If you are simulating APIs you may find that python placebo is useful.  See
-doc:`testing_units_modules` for more information.
-
-
-Code Coverage For New or Updated Unit Tests
-```````````````````````````````````````````
-New code will be missing from the codecov.io coverage reports (see :doc:`testing`), so
-local reporting is needed.  Most ``ansible-test`` commands allow you to collect code
-coverage; this is particularly useful when to indicate where to extend testing.
+Code Coverage
+`````````````
+Most ``ansible-test`` commands allow you to collect code coverage, this is particularly useful when to indicate where to extend testing.
 
 To collect coverage data add the ``--coverage`` argument to your ``ansible-test`` command line:
 
@@ -188,21 +99,7 @@ Reports can be generated in several different formats:
 * ``ansible-test coverage html`` - HTML report.
 * ``ansible-test coverage xml`` - XML report.
 
-To clear data between test runs, use the ``ansible-test coverage erase`` command.  See
-:doc:`testing_units_running_locally` for more information about generating coverage
-reports.
+To clear data between test runs, use the ``ansible-test coverage erase`` command. For a full list of features see the online help::
 
-
-.. seealso::
-
-   :doc:`testing_units_modules`
-       Special considerations for unit testing modules
-   :doc:`testing_running_locally`
-       Running tests locally including gathering and reporting coverage data
-   `Python 3 documentation - 26.4. unittest — Unit testing framework <https://docs.python.org/3/library/unittest.html>`_
-       The documentation of the unittest framework in python 3 
-   `Python 2 documentation - 25.3. unittest — Unit testing framework <https://docs.python.org/3/library/unittest.html>`_
-       The documentation of the earliest supported unittest framework - from Python 2.6
-   `pytest: helps you write better programs <https://docs.pytest.org/en/latest/>`_
-       The documentation of pytest - the framework actually used to run Ansible unit tests
+   ansible-test coverage --help
 

docs/docsite/rst/dev_guide/testing_units_modules.rst

@@ -1,555 +0,0 @@
-****************************
-Unit Testing Ansible Modules
-****************************
-
-.. contents:: Topics
-
-Introduction
-============
-
-This document explains why, how and when you should use unit tests for Ansible modules.
-The document doesn't apply to other parts of Ansible for which the recommendations are
-normally closer to the Python standard.  There is basic documentation for Ansible unit
-tests in the developer guide :doc:`testing_units`.  This document should
-be readable for a new Ansible module author. If you find it incomplete or confusing, 
-please open a bug or ask for help on Ansible IRC.
-
-What Are Unit Tests?
-====================
-
-Ansible includes a set of unit tests in the :file:`test/unit` directory. These tests primarily cover the
-internals but can also can cover Ansible modules.  The structure of the unit tests matches
-the structure of the code base, so the tests that reside in the :file:`test/unit/modules/` directory
-are organized by module groups.
-
-Integration tests can be used for most modules, but there are situations where
-cases cannot be verified using integration tests.  This means that Ansible unit test cases
-may extend beyond testing only minimal units and in some cases will include some
-level of functional testing.
-
-
-Why Use Unit Tests?
-===================
-
-Ansible unit tests have advantages and disadvantages. It is important to understand these.
-Advantages include:
-
-* Most unit tests are much faster than most Ansible integration tests.  The complete suite
-  of unit tests can be run regularly by a developer on their local system.
-* Unit tests can be run by developers who don't have access to the system which the module is
-  designed to work on, allowing a level of verification that changes to core functions
-  haven't broken module expectations.
-* Unit tests can easily substitute system functions allowing testing of software that
-  would be impractical.  For example, the ``sleep()`` function can be replaced and we check
-  that a ten minute sleep was called without actually waiting ten minutes.
-* Unit tests are run on different Python versions. This allows us to
-  ensure that the code behaves in the same way on different Python versions.
-
-There are also some potential disadvantages of unit tests. Unit tests don't normally
-directly test actual useful valuable features of software, instead just internal
-implementation
-
-* Unit tests that test the internal, non-visible features of software may make
-  refactoring difficult if those internal features have to change (see also naming in How
-  below)
-* Even if the internal feature is working correctly it is possible that there will be a
-  problem between the internal code tested and the actual result delivered to the user
-
-Normally the Ansible integration tests (which are written in Ansible YAML) provide better
-testing for most module functionality.  If those tests already test a feature and perform
-well there may be little point in providing a unit test covering the same area as well.
-
-When To Use Unit Tests
-======================
-
-There are a number of situations where unit tests are a better choice than integration
-tests. For example, testing things which are impossible, slow or very difficult to test
-with integration tests, such as:
-    
-* Forcing rare / strange / random situations that can't be forced, such as specific network
-  failures and exceptions
-* Extensive testing of slow configuration APIs 
-* Situations where the integration tests cannot be run as part of the main Ansible
-  continuous integraiton running in Shippable.
-
-
-
-Providing quick feedback
-------------------------
-
-Example:
-  A single step of the rds_instance test cases can take up to 20
-  minutes (the time to create an RDS instance in Amazon).  The entire
-  test run can last for well over an hour.  All 16 of the unit tests
-  complete execution in less than 2 seconds.
-
-The time saving provided by being able to run the code in a unit test makes it worth
-creating a unit test when bug fixing a module, even if those tests do not often identify
-problems later.  As a basic goal, every module should have at least one unit test which
-will give quick feedback in easy cases without having to wait for the integration tests to
-complete.
-    
-Ensuring correct use of external interfaces
--------------------------------------------
-
-Unit tests can check the way in which external services are run to ensure that they match 
-specifications or are as efficient as possible *even when the final output will not be changed*.
-
-Example:
-  Package managers are often far more efficient when installing multiple packages at once
-  rather than each package separately. The final result is the
-  same: the packages are all installed, so the efficiency is difficult to verify through
-  integration tests. By providing a mock package manager and verifying that it is called
-  once, we can build a valuable test for module efficiency.
-
-Another related use is in the situation where an API has versions which behave
-differently. A programmer working on a new version may change the module to work with the
-new API version and unintentially break the old version.  A test case
-which checks that the call happens properly for the old version can help avoid the
-problem.  In this situation it is very important to include version numbering in the test case
-name (see `Naming unit tests`_ below).
-
-Providing specific design tests 
---------------------------------
-
-By building a requirement for a particular part of the
-code and then coding to that requirement, unit tests _can_ sometimes improve the code and
-help future developers understand that code. 
-
-Unit tests that test internal implementation details of code, on the other hand, almost
-always do more harm than good.  Testing that your packages to install are stored in a list
-would slow down and confuse a future developer who might need to change that list into a
-dictionary for efficiency. This problem can be reduced somewhat with clear test naming so
-that the future developer immediately knows to delete the test case, but it is often
-better to simply leave out the test case altogether and test for a real valuable feature
-of the code, such as installing all of the packages supplied as arguments to the module. 
-
-
-How to unit test Ansible modules
-================================
-
-There are a number of techniques for unit testing modules.  Beware that most
-modules without unit tests are structured in a way that makes testing quite difficult and
-can lead to very complicated tests which need more work than the code.  Effectively using unit
-tests may lead you to restructure your code. This is often a good thing and leads
-to better code overall. Good restructuring can make your code clearer and easier to understand.
-
-
-Naming unit tests
------------------
-
-Unit tests should have logical names. If a developer working on the module being tested
-breaks the test case, it should be easy to figure what the unit test covers from the name.
-If a unit test is designed to verify compatibility with a specific software or API version
-then include the version in the name of the unit test.
-
-As an example, ``test_v2_state_present_should_call_create_server_with_name()`` would be a
-good name, ``test_create_server()`` would not be.  
-
-
-Use of Mocks
-------------
-
-Mock objects (from https://docs.python.org/3/library/unittest.mock.html) can be very
-useful in building unit tests for special / difficult cases, but they can also
-lead to complex and confusing coding situations.  One good use for mocks would be in
-simulating an API. As for 'six', the 'mock' python package is bundled with Ansible (use
-'import ansible.compat.tests.mock'). See for example
-
-Ensuring failure cases are visible with mock objects
-----------------------------------------------------
-
-Functions like module.fail_json() are normally expected to terminate execution. When you 
-run with a mock module object this doesn't happen since the mock always returns another mock 
-from a function call. You can set up the mock to raise an exception as shown above, or you can
-assert that these functions have not been called in each test. For example::
-
-  module = MagicMock()
-  function_to_test(module, argument)
-  module.fail_json.assert_not_called() 
-
-This applies not only to calling the main module but almost any other
-function in a module which gets the module object.  
-
-
-Mocking of the actual module
-----------------------------
-
-The setup of an actual module is quite complex (see `Passing Arguments`_ below) and often
-isn't needed for most functions which use a module. Instead you can use a mock object as
-the module and create any module attributes needed by the function you are testing. If
-you do this, beware that the module exit functions need special handling as mentioned
-above, either by throwing an exception or ensuring that they haven't been called. For example::
-
-    class AnsibleExitJson(Exception):
-        """Exception class to be raised by module.exit_json and caught by the test case"""
-        pass
-    #you may also do the same to fail json
-    module=MagicMock()
-    module.exit_json.side_effect = AnsibleExitJson(Exception)
-    with self.assertRaises(AnsibleExitJson) as result:
-        return = my_module.test_this_function(module, argument)
-    module.fail_json.assert_not_called() 
-    assert return["changed"] == True
-    
-API definition with unit test cases
------------------------------------
-
-API interaction is usually best tested with the function tests defined in Ansible's
-integration testing section, which run against the actual API.  There are several cases
-where the unit tests are likely to work better.
-
-Defining a module against an API specification
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This case is especially important for modules interacting with web services, which provide
-an API that Ansible uses but which are beyond the control of the user.
-
-By writing a custom emulation of the calls that return data from the API, we can ensure
-that only the features which are clearly defined in the specification of the API are
-present in the message.  This means that we can check that we use the correct
-parameters and nothing else.
-
-
-*Example:  in rds_instance unit tests a simple instance state is defined*::
-
-    def simple_instance_list(status, pending):
-        return {u'DBInstances': [{u'DBInstanceArn': 'arn:aws:rds:us-east-1:1234567890:db:fakedb',
-                                  u'DBInstanceStatus': status,
-                                  u'PendingModifiedValues': pending,
-                                  u'DBInstanceIdentifier': 'fakedb'}]}
-
-This is then used to create a list of states::
-
-    rds_client_double = MagicMock()
-    rds_client_double.describe_db_instances.side_effect = [
-        simple_instance_list('rebooting', {"a": "b", "c": "d"}),
-        simple_instance_list('available', {"c": "d", "e": "f"}),
-        simple_instance_list('rebooting', {"a": "b"}),
-        simple_instance_list('rebooting', {"e": "f", "g": "h"}),
-        simple_instance_list('rebooting', {}),
-        simple_instance_list('available', {"g": "h", "i": "j"}),
-        simple_instance_list('rebooting', {"i": "j", "k": "l"}),
-        simple_instance_list('available', {}),
-        simple_instance_list('available', {}),
-    ]
-    
-These states are then used as returns from a mock object to ensure that the ``await`` function
-waits through all of the states that would mean the RDS instance has not yet completed
-configuration::
-
-   rds_i.await_resource(rds_client_double, "some-instance", "available", mod_mock,
-                        await_pending=1)
-   assert(len(sleeper_double.mock_calls) > 5), "await_pending didn't wait enough"
-
-By doing this we check that the ``await`` function will keep waiting through
-potentially unusual that it would be impossible to reliably trigger through the
-integration tests but which happen unpredictably in reality.
-
-Defining a module to work against multiple API versions
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This case is especially important for modules interacting with many different versions of
-software; for example, package installation modules that might be expected to work with
-many different operating system versions.
-
-By using previously stored data from various versions of an API we can ensure that the
-code is tested against the actual data which will be sent from that version of the system
-even when the version is very obscure and unlikely to be available during testing.
-
-Ansible special cases for unit testing
-======================================
-
-There are a number of special cases for unit testing the environment of an Ansible module.
-The most common are documented below, and suggestions for others can be found by looking
-at the source code of the existing unit tests or asking on the Ansible IRC channel or mailing
-lists.
-
-Module argument processing 
---------------------------
-
-There are two problems with running the main function of a module:  
-
-* Since the module is supposed to accept arguments on ``STDIN`` it is a bit difficult to
-  set up the arguments correctly so that the module will get them as parameters.
-* All modules should finish by calling either the ``module.fail_json`` or
-  ``module.exit_json``, but these won't work correctly in a testing environment.
-
-Passing Arguments
------------------
-
-.. This section should be updated once https://github.com/ansible/ansible/pull/31456 is
-   closed since the function below will be provided in a library file.
-
-To pass arguments to a module correctly, use a function that stores the
-parameters in a special string variable.  Module creation and argument processing is
-handled through the AnsibleModule object in the basic section of the utilities. Normally
-this accepts input on ``STDIN``, which is not convenient for unit testing. When the special
-variable is set it will be treated as if the input came on ``STDIN`` to the module.::
-
-    import json
-    from ansible.module_utils._text import to_bytes
-
-    def set_module_args(args):
-        args = json.dumps({'ANSIBLE_MODULE_ARGS': args})
-        basic._ANSIBLE_ARGS = to_bytes(args)
-
-    simply call that function before setting up your module
-
-        def test_already_registered(self):
-            set_module_args({
-                'activationkey': 'key',
-                'username': 'user',
-                'password': 'pass',
-            })
-
-Handling exit correctly
------------------------
-
-.. This section should be updated once https://github.com/ansible/ansible/pull/31456 is
-   closed since the exit and failure functions below will be provided in a library file.
-
-The ``module.exit_json()`` function won't work properly in a testing environment since it
-writes error information to ``STDOUT`` upon exit, where it
-is difficult to examine. This can be mitigated by replacing it (and module.fail_json) with
-a function that raises an exception::
-
-    def exit_json(*args, **kwargs):
-        if 'changed' not in kwargs:
-            kwargs['changed'] = False
-        raise AnsibleExitJson(kwargs)
-
-Now you can ensure that the first function called is the one you expected simply by
-testing for the correct exception::
-
-    def test_returned_value(self):
-        set_module_args({
-            'activationkey': 'key',
-            'username': 'user',
-            'password': 'pass',
-        })
-       with self.assertRaises(AnsibleExitJson) as result:
-           my_module.main()
-
-The same technique can be used to replace ``module.fail_json()`` (which is used for failure
-returns from modules) and for the ``aws_module.fail_json_aws()`` (used in modules for Amazon
-Web Services).
-
-Running the main function
--------------------------
-
-If you do want to run the actual main function of a module you must import the module, set
-the arguments as above, set up the appropriate exit exception and then run the module::
-
-    # This test is based around pytest's features for individual test functions
-    import pytest
-    import ansible.modules.module.group.my_modulle as my_module
-
-    def test_main_function(monkeypatch):
-        monkeypatch.setattr(my_module.AnsibleModule, "exit_json", fake_exit_json)
-        set_module_args({
-            'activationkey': 'key',
-            'username': 'user',
-            'password': 'pass',
-        })
-        my_module.main()
-
-
-Handling calls to external executables
---------------------------------------
-
-Module must use AnsibleModule.run_command in order to execute an external command. This
-method needs to be mocked:
-
-Here is a simple mock of AnsibleModule.run_command (taken from test/units/modules/packaging/os/test_rhn_register.py and
-test/units/modules/packaging/os/rhn_utils.py)::
-
-        with patch.object(basic.AnsibleModule, 'run_command') as run_command:
-            run_command.return_value = 0, '', ''  # successful execution, no output
-                with self.assertRaises(AnsibleExitJson) as result:
-                    self.module.main()
-                self.assertFalse(result.exception.args[0]['changed'])
-        # Check that run_command has been called
-        run_command.assert_called_once_with('/usr/bin/command args')
-        self.assertEqual(run_command.call_count, 1)
-        self.assertFalse(run_command.called)
-
-
-A Complete Example
-------------------
-
-The following example is a complete skeleton that reuses the mocks explained above and adds a new
-mock for Ansible.get_bin_path::
-    
-    import json
-
-    from ansible.compat.tests import unittest
-    from ansible.compat.tests.mock import patch
-    from ansible.module_utils import basic
-    from ansible.module_utils._text import to_bytes
-    from ansible.modules.namespace import my_module
-
-
-    def set_module_args(args):
-        """prepare arguments so that they will be picked up during module creation"""
-        args = json.dumps({'ANSIBLE_MODULE_ARGS': args})
-        basic._ANSIBLE_ARGS = to_bytes(args)
-
-
-    class AnsibleExitJson(Exception):
-        """Exception class to be raised by module.exit_json and caught by the test case"""
-        pass
-
-
-    class AnsibleFailJson(Exception):
-        """Exception class to be raised by module.fail_json and caught by the test case"""
-        pass
-
-
-    def exit_json(*args, **kwargs):
-        """function to patch over exit_json; package return data into an exception"""
-        if 'changed' not in kwargs:
-            kwargs['changed'] = False
-        raise AnsibleExitJson(kwargs)
-
-
-    def fail_json(*args, **kwargs):
-        """function to patch over fail_json; package return data into an exception"""
-        kwargs['failed'] = True
-        raise AnsibleFailJson(kwargs)
-
-
-    def get_bin_path(self, arg, required=False):
-        """Mock AnsibleModule.get_bin_path"""
-        if arg.endswith('my_command'):
-            return '/usr/bin/my_command'
-        else:
-            if required:
-                fail_json(msg='%r not found !' % arg)
-
-
-    class TestMyModule(unittest.TestCase):
-
-        def setUp(self):
-            self.mock_module_helper = patch.multiple(basic.AnsibleModule,
-                                                     exit_json=exit_json,
-                                                     fail_json=fail_json,
-                                                     get_bin_path=get_bin_path)
-            self.mock_module_helper.start()
-            self.addCleanup(self.mock_module_helper.stop)
-
-        def test_module_fail_when_required_args_missing(self):
-            with self.assertRaises(AnsibleFailJson):
-                set_module_args({})
-                self.module.main()
-
-
-        def test_ensure_command_called(self):
-            set_module_args({
-                'param1': 10,
-                'param2': 'test',
-            })
-
-            with patch.object(basic.AnsibleModule, 'run_command') as mock_run_command:
-                stdout = 'configuration updated'
-                stderr = ''
-                rc = 0
-                mock_run_command.return_value = rc, stdout, stderr  # successful execution
-
-                with self.assertRaises(AnsibleExitJson) as result:
-                    my_module.main()
-                self.assertFalse(result.exception.args[0]['changed']) # ensure result is changed
-
-            mock_run_command.assert_called_once_with('/usr/bin/my_command --value 10 --name test')
-
-
-Restructuring modules to enable testing module set up and other processes
--------------------------------------------------------------------------
-
-Often modules have a main() function which sets up the module and then performs other
-actions. This can make it difficult to check argument processing. This can be made easier by
-moving module configuration and initialization into a separate function. For example::
-
-    argument_spec = dict(
-        # module function variables
-        state=dict(choices=['absent', 'present', 'rebooted', 'restarted'], default='present'),
-        apply_immediately=dict(type='bool', default=False),
-        wait=dict(type='bool', default=False),
-        wait_timeout=dict(type='int', default=600),
-        allocated_storage=dict(type='int', aliases=['size']),
-        db_instance_identifier=dict(aliases=["id"], required=True),
-    )
-
-    def setup_module_object():
-        module = AnsibleAWSModule(
-            argument_spec=argument_spec,
-            required_if=required_if,
-            mutually_exclusive=[['old_instance_id', 'source_db_instance_identifier',
-                                 'db_snapshot_identifier']],
-        )
-        return module
-
-    def main():
-        module = setup_module_object()
-        validate_parameters(module)
-        conn = setup_client(module)
-        return_dict = run_task(module, conn)
-        module.exit_json(**return_dict)
-
-This now makes it possible to run tests against the module initiation function::
-
-    def test_rds_module_setup_fails_if_db_instance_identifier_parameter_missing():
-        # db_instance_identifier parameter is missing
-        set_module_args({
-            'state': 'absent',
-            'apply_immediately': 'True',
-         })
-
-        with self.assertRaises(AnsibleFailJson) as result:
-             self.module.setup_json
-
-See also ``test/units/module_utils/aws/test_rds.py``
-
-Note that the argument_spec dictionary is visible in a module variable. This has
-advantages, both in allowing explicit testing of the arguments and in allowing the easy
-creation of module objects for testing.
-
-The same restructuring technique can be valuable for testing other functionality, such as the part of the module which queries the object that the module configures.
-
-Traps for maintaining Python 2 compatibility
-============================================
-
-If you use the ``mock`` library from the Python 2.6 standard library, a number of the
-assert functions are missing but will return as if successful.  This means that test cases should take great care *not* use
-functions marked as _new_ in the Python 3 documentation, since the tests will likely always
-succeed even if the code is broken when run on older versions of Python.
-
-A helpful development approach to this should be to ensure that all of the tests have been
-run under Python 2.6 and that each assertion in the test cases has been checked to work by breaking
-the code in Ansible to trigger that failure.
-
-.. seealso::
-
-   :doc:`testing_units`
-       Ansible unit tests documentation
-   :doc:`testing_running_locally`
-       Running tests locally including gathering and reporting coverage data
-   :doc:`developing_modules`
-       How to develop modules
-   `Python 3 documentation - 26.4. unittest — Unit testing framework <https://docs.python.org/3/library/unittest.html>`_
-       The documentation of the unittest framework in python 3 
-   `Python 2 documentation - 25.3. unittest — Unit testing framework <https://docs.python.org/3/library/unittest.html>`_
-       The documentation of the earliest supported unittest framework - from Python 2.6
-   `pytest: helps you write better programs <https://docs.pytest.org/en/latest/>`_
-       The documentation of pytest - the framework actually used to run Ansible unit tests
-   `Development Mailing List <http://groups.google.com/group/ansible-devel>`_
-       Mailing list for development topics
-   `Testing Your Code (from The Hitchhiker's Guide to Python!) <http://docs.python-guide.org/en/latest/writing/tests/>`_
-       General advice on testing Python code
-   `Uncle Bob's many videos on YouTube <https://www.youtube.com/watch?v=QedpQjxBPMA&list=PLlu0CT-JnSasQzGrGzddSczJQQU7295D2>`_
-       Unit testing is a part of the of various philosophies of software development, including
-       Extreme Programming (XP), Clean Coding.  Uncle Bob talks through how to benfit from this
-   `"Why Most Unit Testing is Waste" http://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf`
-       An article warning against the costs of unit testing
-   `'A Response to "Why Most Unit Testing is Waste"' https://henrikwarne.com/2014/09/04/a-response-to-why-most-unit-testing-is-waste/` 
-       An response pointing to how to maintain the value of unit tests

docs/docsite/rst/faq.rst

@@ -56,7 +56,7 @@ by default if OpenSSH is new enough to support ControlPersist as an option.
 
 Paramiko is great for starting out, but the OpenSSH type offers many advanced options.  You will want to run Ansible
 from a machine new enough to support ControlPersist, if you are using this connection type.  You can still manage
-older clients.  If you are using RHEL 6, CentOS 6, SLES 10 or SLES 11 the version of OpenSSH is still a bit old, so 
+older clients.  If you are using RHEL 6, CentOS 6, SLES 10 or SLES 11 the version of OpenSSH is still a bit old, so
 consider managing from a Fedora or openSUSE client even though you are managing older nodes, or just use paramiko.
 
 We keep paramiko as the default as if you are first installing Ansible on an EL box, it offers a better experience
@@ -116,7 +116,7 @@ By default, Ansible assumes it can find a /usr/bin/python on your remote system
 
 Setting the inventory variable 'ansible_python_interpreter' on any host will allow Ansible to auto-replace the interpreter
 used when executing python modules.   Thus, you can point to any python you want on the system if /usr/bin/python on your
-system does not point to a Python 2.X interpreter.  
+system does not point to a Python 2.X interpreter.
 
 Some Linux operating systems, such as Arch, may only have Python 3 installed by default.  This is not sufficient and you will
 get syntax errors trying to run modules with Python 3.  Python 3 is essentially not the same language as Python 2.  Python 3
@@ -305,6 +305,20 @@ You shouldn't put plaintext passwords in your playbook or host_vars; instead, us
 
 .. _commercial_support:
 
+Ansible supports dot notation and array notation for variables. Which notation should I use?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+The dot notation comes from Jinja and works fine for variables without special
+characters. If your variable contains dots (.), colons (:), or dashes (-) it is
+safer to use the array notation for variables.
+
+.. code-block:: jinja
+
+    item[0]['checksum:md5']
+    item['section']['2.1']
+    item['region']['Mid-Atlantic']
+    It is {{ temperature['Celsius']['-3'] }} outside.
+
 Can I get training on Ansible?
 ++++++++++++++++++++++++++++++
 
@@ -362,7 +376,7 @@ When should I use {{ }}? Also, how to interpolate variables or dynamic variable
 
 A steadfast rule is 'always use {{ }} except when `when:`'.
 Conditionals are always run through Jinja2 as to resolve the expression,
-so `when:`, `failed_when:`, and `changed_when:` are always templated and you should avoid adding `{{}}`.
+so `when:`, `failed_when:` and `changed_when:` are always templated and you should avoid adding `{{}}`.
 
 In most other cases you should always use the brackets, even if previously you could use variables without specifying (like `with_` clauses),
 as this made it hard to distinguish between an undefined variable and a string.
@@ -407,6 +421,3 @@ Please see the section below for a link to IRC and the Google Group, where you c
        Have a question?  Stop by the google group!
    `irc.freenode.net <http://irc.freenode.net>`_
        #ansible IRC chat channel
-
-
-

docs/docsite/rst/galaxy.rst

@@ -274,7 +274,7 @@ Alternatively, the role_skeleton and ignoring of files can be configured via ans
 Search for Roles
 ----------------
 
-Search the Galaxy database by tags, platforms, author, and multiple keywords. For example:
+Search the Galaxy database by tags, platforms, author and multiple keywords. For example:
 
 ::
 
@@ -361,7 +361,7 @@ Use ``remove`` to delete a role from *roles_path*:
 Authenticate with Galaxy
 ------------------------
 
-Using the ``import``, ``delete``, and ``setup`` commands to manage your roles on the Galaxy website requires authentication, and the ``login`` command
+Using the ``import``, ``delete`` and ``setup`` commands to manage your roles on the Galaxy website requires authentication, and the ``login`` command
 can be used to do just that. Before you can use the ``login`` command, you must create an account on the Galaxy website.
 
 The ``login`` command requires using your GitHub credentials. You can use your username and password, or you can create a `personal access token <https://help.github.com/articles/creating-an-access-token-for-command-line-use/>`_. If you choose to create a token, grant minimal access to the token, as it is used just to verify identify.
@@ -382,8 +382,8 @@ The following shows authenticating with the Galaxy website using a GitHub userna
     Password for dsmith:
     Successfully logged into Galaxy as dsmith
 
-When you choose to use your username and password, your password is not sent to Galaxy. It is used to authenticate with GitHub and create a personal access token.
-It then sends the token to Galaxy, which in turn verifies your identity and returns a Galaxy access token. After authentication completes the GitHub token is
+When you choose to use your username and password, your password is not sent to Galaxy. It is used to authenticates with GitHub and create a personal access token.
+It then sends the token to Galaxy, which in turn verifies that your identity and returns a Galaxy access token. After authentication completes the GitHub token is
 destroyed.
 
 If you do not wish to use your GitHub password, or if you have two-factor authentication enabled with GitHub, use the *--github-token* option to pass a personal access token

docs/docsite/rst/guide_azure.rst

@@ -131,14 +131,14 @@ Creating Virtual Machines
 -------------------------
 
 There are two ways to create a virtual machine, both involving the azure_rm_virtualmachine module. We can either create
-a storage account, network interface, security group, and public IP address and pass the names of these objects to the
+a storage account, network interface, security group and public IP address and pass the names of these objects to the
 module as parameters, or we can let the module do the work for us and accept the defaults it chooses.
 
 Creating Individual Components
 ..............................
 
 An Azure module is available to help you create a storage account, virtual network, subnet, network interface,
-security group, and public IP. Here is a full example of creating each of these and passing the names to the
+security group and public IP. Here is a full example of creating each of these and passing the names to the
 azure_rm_virtualmachine module at the end:
 
 .. code-block:: yaml
@@ -315,7 +315,7 @@ azure_rm.ini file in your current working directory.
 NOTE: An .ini file will take precedence over environment variables.
 
 NOTE: The name of the .ini file is the basename of the inventory script (i.e. 'azure_rm') with a '.ini'
-extension. This allows you to copy, rename, and customize the inventory script and have matching .ini files all in
+extension. This allows you to copy, rename and customize the inventory script and have matching .ini files all in
 the same directory.
 
 Control grouping using the following variables defined in the environment:

docs/docsite/rst/guide_cloudstack.rst

@@ -134,7 +134,7 @@ Environment Variables
 `````````````````````
 .. versionadded:: 2.3
 
-Since Ansible 2.3 it is possible to use environment variables for domain (``CLOUDSTACK_DOMAIN``), account (``CLOUDSTACK_ACCOUNT``), project (``CLOUDSTACK_PROJECT``), VPC (``CLOUDSTACK_VPC``), and zone (``CLOUDSTACK_ZONE``). This simplifies the tasks by not repeating the arguments for every tasks.
+Since Ansible 2.3 it is possible to use environment variables for domain (``CLOUDSTACK_DOMAIN``), account (``CLOUDSTACK_ACCOUNT``), project (``CLOUDSTACK_PROJECT``), VPC (``CLOUDSTACK_VPC``) and zone (``CLOUDSTACK_ZONE``). This simplifies the tasks by not repeating the arguments for every tasks.
 
 Below you see an example how it can be used in combination with Ansible's block feature:
 
@@ -165,7 +165,7 @@ Below you see an example how it can be used in combination with Ansible's block
 
 .. Note:: You are still able overwrite the environment variables using the module arguments, e.g. ``zone: sf-2``
 
-.. Note:: Unlike ``CLOUDSTACK_REGION`` these additional environment variables are ignored in the CLI ``cs``.
+.. Note:: Unlike ``CLOUDSTACK_REGION`` these additional environment variables are ingored in the CLI ``cs``.
 
 Use Cases
 `````````
@@ -197,7 +197,7 @@ This is how our inventory looks like:
 
 As you can see, the public IPs for our web servers and jumphost has been assigned as variable ``public_ip`` directly in the inventory.
 
-To configure the jumphost, web servers, and database servers, we use ``group_vars``. The ``group_vars`` directory contains 4 files for configuration of the groups: cloud-vm, jumphost, webserver, and db-server. The cloud-vm is there for specifying the defaults of our cloud infrastructure.
+The configure the jumphost, web servers and database servers, we use ``group_vars``. The ``group_vars`` directory contains 4 files for configuration of the groups: cloud-vm, jumphost, webserver and db-server. The cloud-vm is there for specifying the defaults of our cloud infrastructure.
 
 .. code-block:: yaml
 
@@ -262,9 +262,9 @@ Now to the fun part. We create a playbook to create our infrastructure we call i
           cs_staticnat: vm="{{ inventory_hostname_short }}" ip_address="{{ public_ip }}"
           when: public_ip is defined
 
-In the above play we defined 3 tasks and used the group ``cloud-vm`` as target to handle all VMs in the cloud but instead SSH to these VMs, we use ``connection=local`` to execute the API calls locally from our workstation.
+In the above play we defined 3 tasks and use the group ``cloud-vm`` as target to handle all VMs in the cloud but instead SSH to these VMs, we use ``connetion=local`` to execute the API calls locally from our workstation.
 
-In the first task, we ensure we have a running VM created with the Debian template. If the VM is already created but stopped, it would just start it. If you like to change the offering on an existing VM, you must add ``force: yes`` to the task, which would stop the VM, change the offering, and start the VM again.
+In the first task, we ensure we have a running VM created with the Debian template. If the VM is already created but stopped, it would just start it. If you like to change the offering on an existing VM, you must add ``force: yes`` to the task, which would stop the VM, change the offering and start the VM again.
 
 In the second task we ensure the ports are opened if we give a public IP to the VM.
 
@@ -364,12 +364,12 @@ The playbook looks like the following:
       - name: show VM IP
         debug: msg="VM {{ inventory_hostname }} {{ vm.default_ip }}"
 
-      - name: assign IP to the inventory
+      - name: assing IP to the inventory
         set_fact: ansible_ssh_host={{ vm.default_ip }}
 
       - name: waiting for SSH to come up
         wait_for: port=22 host={{ vm.default_ip }} delay=5
 
-In the first play we setup the security groups, in the second play the VMs will created be assigned to these groups. Further you see, that we assign the public IP returned from the modules to the host inventory. This is needed as we do not know the IPs we will get in advance. In a next step you would configure the DNS servers with these IPs for accessing the VMs with their DNS name.
+In the first play we setup the security groups, in the second play the VMs will created be assigned to these groups. Further you see, that we assign the public IP returned from the modules to the host inventory. This is needed as we do not know the IPs we will get in advance. In a next step you would configure the DNS servers with these IPs for accassing the VMs with their DNS name.
 
 In the last task we wait for SSH to be accessible, so any later play would be able to access the VM by SSH without failure.

docs/docsite/rst/guide_docker.rst

@@ -8,11 +8,11 @@ Ansible offers the following modules for orchestrating Docker containers:
         Swarm. Supports compose versions 1 and 2.
 
     docker_container
-        Manages the container lifecycle by providing the ability to create, update, stop, start, and destroy a
+        Manages the container lifecycle by providing the ability to create, update, stop, start and destroy a
         container.
 
     docker_image
-        Provides full control over images, including: build, pull, push, tag, and remove.
+        Provides full control over images, including: build, pull, push, tag and remove.
 
     docker_image_facts
         Inspects one or more images in the Docker host's image cache, providing the information as facts for making
@@ -126,7 +126,7 @@ running Ansible:
         The maximum amount of time in seconds to wait on a response from the API.
 
     DOCKER_CERT_PATH
-        Path to the directory containing the client certificate, client key, and CA certificate.
+        Path to the directory containing the client certificate, client key and CA certificate.
 
     DOCKER_SSL_VERSION
         Provide a valid SSL version number.
@@ -205,7 +205,7 @@ options. These are the same environment variables used by the Docker modules.
         by docker-py.
 
     DOCKER_TIMEOUT:
-        The maximum amount of time in seconds to wait on a response from the API. Defaults to 60 seconds.
+        The maximum amount of time in seconds to wait on a response fromm the API. Defaults to 60 seconds.
 
     DOCKER_TLS:
         Secure the connection to the API by using TLS without verifying the authenticity of the Docker host server.
@@ -213,18 +213,18 @@ options. These are the same environment variables used by the Docker modules.
 
     DOCKER_TLS_VERIFY:
         Secure the connection to the API by using TLS and verifying the authenticity of the Docker host server.
-        Defaults to False.
+        Default is False
 
     DOCKER_TLS_HOSTNAME:
         When verifying the authenticity of the Docker Host server, provide the expected name of the server. Defaults
         to localhost.
 
     DOCKER_CERT_PATH:
-        Path to the directory containing the client certificate, client key, and CA certificate.
+        Path to the directory containing the client certificate, client key and CA certificate.
 
     DOCKER_SSL_VERSION:
         Provide a valid SSL version number. Default value determined by docker-py, which at the time of this writing
-        was 1.0.
+        was 1.0
 
 In addition to the connection variables there are a couple variables used to control the execution and output of the
 script:

docs/docsite/rst/guide_packet.rst

@@ -8,7 +8,7 @@ Introduction
 `Packet.net <https://packet.net>`_ is a bare metal infrastructure host that's supported by Ansible (>=2.3) via a dynamic inventory script and two cloud modules. The two modules are:
 
 - packet_sshkey: adds a public SSH key from file or value to the Packet infrastructure. Every subsequently-created device will have this public key installed in .ssh/authorized_keys.
-- packet_device: manages servers on Packet. You can use this module to create, restart, and delete devices.
+- packet_device: manages servers on Packet. You can use this module to create, restart and delete devices.
 
 Note, this guide assumes you are familiar with Ansible and how it works. If you're not, have a look at their `docs <http://docs.ansible.com/>`_ before getting started.
 
@@ -183,7 +183,7 @@ The following playbook will create an SSH key, 3 Packet servers, and then wait u
 
 As with most Ansible modules, the default states of the Packet modules are idempotent, meaning the resources in your project will remain the same after re-runs of a playbook. Thus, we can keep the ``packet_sshkey`` module call in our playbook. If the public key is already in your Packet account, the call will have no effect.
 
-The second module call provisions 3 Packet Type 0 (specified using the 'plan' parameter) servers in the project identified via the 'project_id' parameter. The servers are all provisioned with CoreOS beta (the 'operating_system' parameter) and are customized with cloud-config user data passed to the 'user_data' parameter.
+The second module call provisions 3 Packet Type 0 (specified using the 'plan' parameter) servers in the project identified via the 'project_id' parameter. The servers are all provisioned with CoresOS beta (the 'operating_system' parameter) and are customized with cloud-config user data passed to the 'user_data' parameter.
 
 The ``packet_device`` module has a boolean 'wait' parameter that defaults to 'false'. If set to 'true', Ansible will wait until the GET API call for a device will contain an Internet-routeable IP address. The 'wait' parameter allows us to use the IP address of the device as soon as it's available.
 

docs/docsite/rst/guide_rax.rst

@@ -457,7 +457,7 @@ Create an isolated cloud network and build a server
 Complete Environment
 ++++++++++++++++++++
 
-Build a complete webserver environment with servers, custom networks and load balancers, install nginx, and create a custom index.html.
+Build a complete webserver environment with servers, custom networks and load balancers, install nginx and create a custom index.html
 
 .. code-block:: yaml
    

docs/docsite/rst/guide_rolling_upgrade.rst

@@ -176,7 +176,7 @@ Here's another example, from the same template:
    {% endfor %}
 
 This loops over all of the hosts in the group called ``monitoring``, and adds an ACCEPT line for 
-each monitoring hosts' default IPv4 address to the current machine's iptables configuration, so that Nagios can monitor those hosts.
+each monitoring hosts' default IPV4 address to the current machine's iptables configuration, so that Nagios can monitor those hosts.
 
 You can learn a lot more about Jinja2 and its capabilities `here <http://jinja.pocoo.org/docs/>`_, and you 
 can read more about Ansible variables in general in the :doc:`playbooks_variables` section.
@@ -220,11 +220,6 @@ Here is the next part of the update play::
     delegate_to: "{{ item }}"
     with_items: "{{ groups.lbservers }}"
 
-
-.. note::
-   - The ``serial`` keyword forces the play to be executed in 'batches'. Each batch counts as a full play with a subselection of hosts.
-     This has some consequences on play behavior. For example, if all hosts in a batch fails, the play fails, which in turn fails the entire run. You should consider this when combining with ``max_fail_percentage``.
-
 The ``pre_tasks`` keyword just lets you list tasks to run before the roles are called. This will make more sense in a minute. If you look at the names of these tasks, you can see that we are disabling Nagios alerts and then removing the webserver that we are currently updating from the HAProxy load balancing pool.
 
 The ``delegate_to`` and ``with_items`` arguments, used together, cause Ansible to loop over each monitoring server and load balancer, and perform that operation (delegate that operation) on the monitoring or load balancing server, "on behalf" of the webserver. In programming terms, the outer loop is the list of web servers, and the inner loop is the list of monitoring servers.

docs/docsite/rst/guide_vagrant.rst

@@ -113,7 +113,7 @@ single machine environment may look something like this:
 
 If you want to run Ansible manually, you will want to make sure to pass
 ``ansible`` or ``ansible-playbook`` commands the correct arguments, at least
-for the *username*, the *SSH private key*, and the *inventory*.
+for the *username*, the *SSH private key* and the *inventory*.
 
 Here is an example using the Vagrant global insecure key (``config.ssh.insert_key``
 must be set to ``false`` in your ``Vagrantfile``):

docs/docsite/rst/index.rst

@@ -9,14 +9,14 @@ Welcome to the Ansible documentation!
 Ansible is an IT automation tool.  It can configure systems, deploy software, and orchestrate more advanced IT tasks
 such as continuous deployments or zero downtime rolling updates.
 
-Ansible's main goals are simplicity and ease-of-use. It also has a strong focus on security and reliability, featuring a minimum of moving parts, usage of OpenSSH for transport (with an accelerated socket mode and pull modes as alternatives), and a language that is designed around auditability by humans--even those not familiar with the program.
+Ansible's main goals are simplicity and ease-of-use. It also has a strong focus on security and reliability, featuring a minimum of moving parts, usage of OpenSSH for transport (with other transports and pull modes as alternatives), and a language that is designed around auditability by humans--even those not familiar with the program.
 
 We believe simplicity is relevant to all sizes of environments, so we design for busy users of all types: developers, sysadmins, release engineers, IT managers, and everyone in between. Ansible is appropriate for managing all environments, from small setups with a handful of instances to enterprise environments with many thousands of instances.
 
 Ansible manages machines in an agent-less manner. There is never a question of how to
 upgrade remote daemons or the problem of not being able to manage systems because daemons are uninstalled.  Because OpenSSH is one of the most peer-reviewed open source components, security exposure is greatly reduced. Ansible is decentralized--it relies on your existing OS credentials to control access to remote machines. If needed, Ansible can easily connect with Kerberos, LDAP, and other centralized authentication management systems.
 
-This documentation covers the current released version of Ansible (2.4) and also some development version features ('devel').  For recent features, we note in each section the version of Ansible where the feature was added.
+This documentation covers the current released version of Ansible (2.3) and also some development version features (2.4).  For recent features, we note in each section the version of Ansible where the feature was added.
 
 Ansible, Inc. releases a new major release of Ansible approximately every two months.  The core application evolves somewhat conservatively, valuing simplicity in language design and setup. However, the community around new modules and plugins being developed and contributed moves very quickly, typically adding 20 or so new modules in each release.
 
@@ -33,6 +33,7 @@ Ansible, Inc. releases a new major release of Ansible approximately every two mo
    modules_by_category
    vault
    command_line_tools
+   plugins_by_category
    guides
    dev_guide/index
    tower

docs/docsite/rst/intro_adhoc.rst

@@ -3,8 +3,6 @@ Introduction To Ad-Hoc Commands
 
 .. contents:: Topics
 
-.. highlight:: bash
-
 The following examples show how to use `/usr/bin/ansible` for running
 ad hoc tasks. 
 

docs/docsite/rst/intro_bsd.rst

@@ -66,7 +66,7 @@ Using BSD as the control machine is as simple as installing the Ansible package
 BSD Facts
 `````````
 
-Ansible gathers facts from the BSDs in a similar manner to Linux machines, but since the data, names, and structures can vary for network, disks, and other devices, one should expect the output to be slightly different yet still familiar to a BSD administrator.
+Ansible gathers facts from the BSDs in a similar manner to Linux machines, but since the data, names and structures can vary for network, disks and other devices, one should expect the output to be slightly different yet still familiar to a BSD administrator.
 
 .. _bsd_contributions:
 

docs/docsite/rst/intro_dynamic_inventory.rst

@@ -185,7 +185,7 @@ Security Group
   ``security_group_Pete_s_Fancy_Group``
 
 Tags
-  Each instance can have a variety of key/value pairs associated with it called Tags. The most common tag key is 'Name', though anything is possible. Each key/value pair is its own group of instances, again with special characters converted to underscores, in the format ``tag_KEY_VALUE``.
+  Each instance can have a variety of key/value pairs associated with it called Tags. The most common tag key is 'Name', though anything is possible. Each key/value pair is its own group of instances, again with special characters converted to underscores, in the format ``tag_KEY_VALUE``
   e.g.
   ``tag_Name_Web`` can be used as is
   ``tag_Name_redis-master-001`` becomes ``tag_Name_redis_master_001``
@@ -265,7 +265,7 @@ Source an OpenStack RC file::
 
 .. note::
 
-    An OpenStack RC file contains the environment variables required by the client tools to establish a connection with the cloud provider, such as the authentication URL, user name, password, and region name. For more information on how to download, create or source an OpenStack RC file, please refer to `Set environment variables using the OpenStack RC file <http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html>`_.
+    An OpenStack RC file contains the environment variables required by the client tools to establish a connection with the cloud provider, such as the authentication URL, user name, password and region name. For more information on how to download, create or source an OpenStack RC file, please refer to `Set environment variables using the OpenStack RC file <http://docs.openstack.org/user-guide/common/cli_set_environment_variables_using_openstack_rc.html>`_.
 
 You can confirm the file has been successfully sourced by running a simple command, such as `nova list` and ensuring it return no errors.
 
@@ -286,13 +286,13 @@ Once you confirm the dynamic inventory script is working as expected, you can te
 Implicit use of inventory script
 ++++++++++++++++++++++++++++++++
 
-Download the latest version of the OpenStack dynamic inventory script, make it executable, and copy it to `/etc/ansible/hosts`::
+Download the latest version of the OpenStack dynamic inventory script, make it executable and copy it to `/etc/ansible/hosts`::
 
     wget https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/openstack.py
     chmod +x openstack.py
     sudo cp openstack.py /etc/ansible/hosts
 
-Download the sample configuration file, modify it to suit your needs, and copy it to `/etc/ansible/openstack.yml`::
+Download the sample configuration file, modify it to suit your needs and copy it to `/etc/ansible/openstack.yml`::
 
     wget https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/openstack.yml
     vi openstack.yml

docs/docsite/rst/intro_getting_started.rst

@@ -26,7 +26,7 @@ machines over SSH.
 
 By default, Ansible 1.3 and later will try to use native 
 OpenSSH for remote communication when possible.  This enables ControlPersist (a performance feature), Kerberos, and options in ``~/.ssh/config`` such as Jump Host setup.  However, when using Enterprise Linux 6 operating systems as the control machine (Red Hat Enterprise Linux and derivatives such as CentOS), the version of OpenSSH may be too old to support ControlPersist. On these operating systems, Ansible will fallback into using a high-quality Python implementation of
-OpenSSH called 'paramiko'.  If you wish to use features like Kerberized SSH and more, consider using Fedora, OS X, or Ubuntu as your control machine until a newer version of OpenSSH is available for your platform -- or engage 'accelerated mode' in Ansible.  See :doc:`playbooks_acceleration`.
+OpenSSH called 'paramiko'.  If you wish to use features like Kerberized SSH and more, consider using Fedora, OS X, or Ubuntu as your control machine until a newer version of OpenSSH is available for your platform.
 
 In releases up to and including Ansible 1.2, the default was strictly paramiko.  Native SSH had to be explicitly selected with the ``-c`` ssh option or set in the configuration file.
 

docs/docsite/rst/intro_installation.rst

@@ -84,7 +84,7 @@ default this uses sftp. If that's not available, you can switch to scp in
    By default, Ansible uses Python 2 in order to maintain compatibility with older distributions
    such as RHEL 6. However, some Linux distributions (Gentoo, Arch) may not have a
    Python 2.X interpreter installed by default.  On those systems, you should install one, and set
-   the ``ansible_python_interpreter`` variable in inventory (see :ref:`inventory`) to point at your 2.X Python.  Distributions
+   the 'ansible_python_interpreter' variable in inventory (see :doc:`intro_inventory`) to point at your 2.X Python.  Distributions
    like Red Hat Enterprise Linux, CentOS, Fedora, and Ubuntu all have a 2.X interpreter installed
    by default and this does not apply to those distributions.  This is also true of nearly all
    Unix systems.
@@ -101,34 +101,26 @@ Installing the Control Machine
 ``````````````````````````````
 .. _from_yum:
 
-Latest Release via DNF or Yum
-+++++++++++++++++++++++++++++
+Latest Release Via Yum
+++++++++++++++++++++++
 
-On Fedora:
+.. note:: We've changed how the Ansible community packages are distributed.
+  For users of RHEL/CentOS/Scientific Linux version 7, the Ansible community RPM
+  package will transition from the EPEL repository to the Extras channel.  There will be no
+  change for version 6 of RHEL/CentOS/Scientific Linux since Extras is not a part of version 6.
 
-.. code-block:: bash
+RPMs for RHEL7 are available from `the Extras channel <https://access.redhat.com/solutions/912213>`_.
 
-    $ sudo dnf install ansible
+RPMs for RHEL6 are available from yum for `EPEL
+<http://fedoraproject.org/wiki/EPEL>`_ 6 and currently supported
+Fedora distributions.
 
-On RHEL and CentOS:
+Ansible will also have RPMs/YUM-repo available `here <https://releases.ansible.com/ansible/rpm/>`_.
 
-.. code-block:: bash
+Ansible version 2.4 can manage earlier operating
+systems that contain Python 2.6 or higher.
 
-    $ sudo yum install ansible
-
-RPMs for RHEL 7 are available from the `Ansible Engine repository <https://access.redhat.com/articles/3174981>`_.
-
-To enable the Ansible Engine repository, run the following command:
-
-.. code-block:: bash
-
-    $ sudo subsription-manager repos --enable rhel-7-server-ansible-2.4-rpms
-
-RPMs for currently supported versions of RHEL, CentOS, and Fedora are available from `EPEL <http://fedoraproject.org/wiki/EPEL>`_ as well as `releases.ansible.com <https://releases.ansible.com/ansible/rpm>`_.
-
-Ansible version 2.4 and later can manage earlier operating systems that contain Python 2.6 or higher.
-
-You can also build an RPM yourself. From the root of a checkout or tarball, use the ``make rpm`` command to build an RPM you can distribute and install.
+You can also build an RPM yourself.  From the root of a checkout or tarball, use the ``make rpm`` command to build an RPM you can distribute and install.
 
 .. code-block:: bash
 
@@ -277,7 +269,7 @@ If you are installing on OS X Mavericks, you may encounter some noise from your
 
    $ sudo CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments pip install ansible
 
-Readers that use virtualenv can also install Ansible under virtualenv, though we'd recommend to not worry about it and just install Ansible globally.  Do not use easy_install to install Ansible directly.
+Readers that use virtualenv can also install Ansible under virtualenv, though we'd recommend to not worry about it and just install Ansible globally.  Do not use easy_install to install ansible directly.
 
 .. _tagged_releases:
 

docs/docsite/rst/intro_inventory.rst

@@ -11,7 +11,7 @@ which defaults to being saved in the location ``/etc/ansible/hosts``.
 You can specify a different inventory file using the ``-i <path>`` option on the command line.
 
 Not only is this inventory configurable, but you can also use multiple inventory files at the same time and
-pull inventory from dynamic or cloud sources or different formats (YAML, ini, etc.), as described in :doc:`intro_dynamic_inventory`.
+pull inventory from dynamic or cloud sources or different formats (YAML, ini, etc), as described in :doc:`intro_dynamic_inventory`.
 Introduced in version 2.4, Ansible has inventory plugins to make this flexible and customizable.
 
 .. _inventoryformat:
@@ -282,7 +282,7 @@ the 'raleigh' group might look like::
 It is okay if these files do not exist, as this is an optional feature.
 
 As an advanced use case, you can create *directories* named after your groups or hosts, and
-Ansible will read all the files in these directories in lexicographical order. An example with the 'raleigh' group::
+Ansible will read all the files in these directories. An example with the 'raleigh' group::
 
     /etc/ansible/group_vars/raleigh/db_settings
     /etc/ansible/group_vars/raleigh/cluster_settings
@@ -299,12 +299,40 @@ directory will override variables set in the inventory directory.
 Tip: Keeping your inventory file and variables in a git repo (or other version control)
 is an excellent way to track changes to your inventory and host variables.
 
+.. _how_we_merge:
+
+How Variables Are Merged
+++++++++++++++++++++++++
+
+By default variables are merged/flattened to the specific host before a play is run. This keeps Ansible focused on the Host and Task, so groups don't really surive outside of inventory and host matching. By default, Ansible overwrites variables including the ones defined for a group and/or host (see the `hash_merge` setting to change this) . The order/precedence is (from lowest to highest):
+
+- all group (becauseit is the 'parent' of all other groups)
+- parent group
+- child group
+- host
+
+When groups of the same parent/child level are merged, it is done alphabetically, and the last group loaded overwrites the previous groups. For example, an a_group will be merged with b_group and b_group vars that match will overwrite the ones in a_group.
+
+.. versionadded:: 2.4
+
+Starting in Ansible version 2.4, users can use the group variable ``ansible_group_priority`` to change the merge order for groups of the same level (after the parent/child order is resolved). The larger the number, the later it will be merged, giving it higher priority. This variable defaults to ``1`` if not set. For example:
+
+.. code-block:: yaml
+
+    a_group:
+        testvar: a
+        ansible_group_priority: 10
+    b_group
+        testvar: b
+
+In this example, if both groups have the same priority, the result would normally have been ``testvar == b``, but since we are giving the ``a_group`` a higher priority the result will be ``testvar == a``.
+
 .. _behavioral_parameters:
 
 List of Behavioral Inventory Parameters
 +++++++++++++++++++++++++++++++++++++++
 
-As alluded to above, setting the following variables controls how ansible interacts with remote hosts.
+As described above, setting the following variables control how Ansible interacts with remote hosts.
 
 Host connection:
 
@@ -319,7 +347,7 @@ General for all connections:
 ansible_host
     The name of the host to connect to, if different from the alias you wish to give to it.
 ansible_port
-    The ssh port number, if not 22.
+    The ssh port number, if not 22
 ansible_user
     The default ssh user name to use.
 
@@ -327,7 +355,7 @@ ansible_user
 Specific to the SSH connection:
 
 ansible_ssh_pass
-    The ssh password to use (never store this variable in plain text; always use a vault. See :ref:`best_practices_for_variables_and_vaults`).
+    The ssh password to use (never store this variable in plain text; always use a vault. See :ref:`best_practices_for_variables_and_vaults`)
 ansible_ssh_private_key_file
     Private key file used by ssh.  Useful if using multiple keys and you don't want to use SSH agent.
 ansible_ssh_common_args
@@ -349,17 +377,17 @@ ansible_ssh_executable (added in version 2.2)
 Privilege escalation (see :doc:`Ansible Privilege Escalation<become>` for further details):
 
 ansible_become
-    Equivalent to ``ansible_sudo`` or ``ansible_su``, allows to force privilege escalation.
+    Equivalent to ``ansible_sudo`` or ``ansible_su``, allows to force privilege escalation
 ansible_become_method
-    Allows to set privilege escalation method.
+    Allows to set privilege escalation method
 ansible_become_user
-    Equivalent to ``ansible_sudo_user`` or ``ansible_su_user``, allows to set the user you become through privilege escalation.
+    Equivalent to ``ansible_sudo_user`` or ``ansible_su_user``, allows to set the user you become through privilege escalation
 ansible_become_pass
-    Equivalent to ``ansible_sudo_pass`` or ``ansible_su_pass``, allows you to set the privilege escalation password (never store this variable in plain text; always use a vault. See :ref:`best_practices_for_variables_and_vaults`).
+    Equivalent to ``ansible_sudo_pass`` or ``ansible_su_pass``, allows you to set the privilege escalation password (never store this variable in plain text; always use a vault. See :ref:`best_practices_for_variables_and_vaults`)
 ansible_become_exe
-    Equivalent to ``ansible_sudo_exe`` or ``ansible_su_exe``, allows you to set the executable for the escalation method selected.
+    Equivalent to ``ansible_sudo_exe`` or ``ansible_su_exe``, allows you to set the executable for the escalation method selected
 ansible_become_flags
-    Equivalent to ``ansible_sudo_flags`` or ``ansible_su_flags``, allows you to set the flags passed to the selected escalation method. This can be also set globally in :file:`ansible.cfg` in the ``sudo_flags`` option.
+    Equivalent to ``ansible_sudo_flags`` or ``ansible_su_flags``, allows you to set the flags passed to the selected escalation method. This can be also set globally in :file:`ansible.cfg` in the ``sudo_flags`` option
 
 Remote host environment parameters:
 
@@ -384,7 +412,7 @@ ansible_shell_executable
     overrides ``executable`` in :file:`ansible.cfg` which defaults to
     :command:`/bin/sh`.  You should really only change it if is not possible
     to use :command:`/bin/sh` (i.e. :command:`/bin/sh` is not installed on the target
-    machine or cannot be run from sudo).
+    machine or cannot be run from sudo.).
 
 Examples from an Ansible-INI host file::
 
@@ -449,7 +477,7 @@ Here is an example of how to instantly deploy to created containers::
    :doc:`intro_adhoc`
        Examples of basic commands
    :doc:`playbooks`
-       Learning Ansible's configuration, deployment, and orchestration language
+       Learning Ansible's configuration, deployment, and orchestration language.
    `Mailing List <http://groups.google.com/group/ansible-project>`_
        Questions? Help? Ideas?  Stop by the list on Google Groups
    `irc.freenode.net <http://irc.freenode.net>`_

docs/docsite/rst/intro_networking.rst

@@ -216,5 +216,5 @@ conditional.
 
 The waitfor argument must always start with result and then the
 command index in [], where 0 is the first command in the commands list,
-1 is the second command, 2 is the third, and so on.
+1 is the second command, 2 is the third and so on.
 

docs/docsite/rst/intro_patterns.rst

@@ -58,7 +58,7 @@ is uncommonly used::
 
     webservers:!{{excluded}}:&{{required}}
 
-You also don't have to manage by strictly defined groups.  Individual host names, IPs, and groups, can also be referenced using
+You also don't have to manage by strictly defined groups.  Individual host names, IPs and groups, can also be referenced using
 wildcards
 
 .. code-block:: none

docs/docsite/rst/intro_windows.rst

@@ -59,7 +59,7 @@ Once WSL is enabled, you can open the Bash terminal. At the prompt, you can quic
     # https://github.com/Microsoft/BashOnWindows/issues/816#issuecomment-301216901 for details
     source ~/.profile
 
-After you've successfully run these commands, you can start to create your inventory, write example playbooks, and start targeting systems using the plethora of available Windows modules.
+After you've successfully run these commands, you can start to create your inventory, write example playbooks and start targeting systems using the plethora of available Windows modules.
 
 If you want to run Ansible from source for development purposes, simply uninstall the pip-installed version (which will leave all the necessary dependencies behind), then clone the Ansible source, and run the hacking script to configure it to run from source::
 
@@ -94,8 +94,12 @@ You can specify which authentication option you wish to use by setting it to the
 Certificate
 +++++++++++
 
-Certificate authentication is similar to SSH where a certificate is assigned to a local user and instead of using a password to authenticate a certificate is used.
+Certificate authentication is similar to SSH because it uses both a public and private key to access a local Windows user account. This removes the need for using a password.
 
+This requires two variables that specify the path to both the PEM file containing the certificate (cert_pem) and the related private key file (cert_key_pem).
+
+    ansible_winrm_cert_pem:
+    ansible_winrm_cert_key_pem:
 
 Kerberos
 ++++++++
@@ -142,7 +146,7 @@ Kerberos is installed and configured by default on OS X and many Linux distribut
 Configuring Kerberos
 --------------------
 
-Edit your `/etc/krb5.conf` (which should be installed as a result of installing packages above) and add the following information for each domain you need to connect to:
+Edit your /etc/krb5.conf (which should be installed as a result of installing packages above) and add the following information for each domain you need to connect to:
 
 In the section that starts with
 
@@ -204,13 +208,13 @@ If you unable to connect using kerberos, check the following:
 
 Ensure that forward and reverse DNS lookups are working properly on your domain.
 
-To test this, ping the Windows host you want to control by name then use the ip address returned with nslookup.  You should get the same name back from DNS when you use nslookup on the ip address.  
+To test this, ping the windows host you want to control by name then use the ip address returned with nslookup.  You should get the same name back from DNS when you use nslookup on the ip address.  
 
 If you get different hostnames back than the name you originally pinged, speak to your active directory administrator and get them to check that DNS Scavenging is enabled and that DNS and DHCP are updating each other.
 
 Ensure that the Ansible controller has a properly configured computer account in the domain.
 
-Check your Ansible controller's clock is synchronised with your domain controller.  Kerberos is time sensitive and a little clock drift can cause tickets not to be granted.
+Check your Ansible controller's clock is synchronised with your domain controller.  Kerberos is time sensitive and a little clock drift can cause tickets not be granted.
 
 Check you are using the real fully qualified domain name for the domain.  Sometimes domains are commonly known to users by aliases.  To check this run:
 
@@ -242,7 +246,7 @@ To install credssp you can use pip to install the requests-credssp library:
 CredSSP and TLS 1.2
 -------------------
 
-CredSSP requires the remote host to have TLS 1.2 configured or else the connection will fail. TLS 1.2 is installed by default from Server 2012 and Windows 8 onward. For Server 2008, 2008 R2, and Windows 7 you can add TLS 1.2 support by:
+CredSSP requires the remote host to have TLS 1.2 configured or else the connection will fail. TLS 1.2 is installed by default from Server 2012 and Windows 8 onwards. For Server 2008, 2008 R2 and Windows 7 you can add TLS 1.2 support by:
 
 * Installing the `TLS 1.2 update from Microsoft <https://support.microsoft.com/en-us/help/3080079/update-to-add-rds-support-for-tls-1.1-and-tls-1.2-in-windows-7-or-windows-server-2008-r2>`_
 * Adding the TLS 1.2 registry keys as shown on this `page <https://technet.microsoft.com/en-us/library/dn786418.aspx#BKMK_SchannelTR_TLS12>`_
@@ -255,7 +259,7 @@ If you need to interact with a remote resource or run a process that requires th
 Inventory
 `````````
 
-Ansible's Windows support relies on a few standard variables to indicate the username, password, and connection type (Windows) of the remote hosts.  These variables are most easily set up in inventory.  This is used instead of SSH-keys or passwords as normally fed into Ansible::
+Ansible's windows support relies on a few standard variables to indicate the username, password, and connection type (windows) of the remote hosts.  These variables are most easily set up in inventory.  This is used instead of SSH-keys or passwords as normally fed into Ansible::
 
     [windows]
     winserver1.example.com
@@ -311,7 +315,7 @@ Since 2.0, the following custom inventory variables are also supported for addit
 Windows System Prep
 ```````````````````
 
-In order for Ansible to manage your Windows machines, you will have to enable and configure PowerShell remoting.
+In order for Ansible to manage your windows machines, you will have to enable and configure PowerShell remoting.
 
 To automate the setup of WinRM, you can run the `examples/scripts/ConfigureRemotingForAnsible.ps1 <https://github.com/ansible/ansible/blob/devel/examples/scripts/ConfigureRemotingForAnsible.ps1>`_ script on the remote machine in a PowerShell console as an administrator.
 
@@ -340,7 +344,7 @@ To troubleshoot the ``ConfigureRemotingForAnsible.ps1`` writes every change it m
    Management Framework 3.0, it may be necessary to install this
    hotfix http://support.microsoft.com/kb/2842230 to avoid receiving
    out of memory and stack overflow exceptions.  Newly-installed Server 2008
-   R2 systems which are not fully up to date with Windows updates are known
+   R2 systems which are not fully up to date with windows updates are known
    to have this issue.
 
    Windows 8.1 and Server 2012 R2 are not affected by this issue as they
@@ -386,9 +390,9 @@ In addition, the following core modules/action-plugins work with Windows:
 * template (also: win_template)
 * wait_for_connection
 
-Some modules can be utilised in playbooks that target Windows by delegating to localhost, depending on what you are
+Some modules can be utilised in playbooks that target windows by delegating to localhost, depending on what you are
 attempting to achieve.  For example, ``assemble`` can be used to create a file on your ansible controller that is then 
-sent to your Windows targets using ``win_copy``.
+sent to your windows targets using ``win_copy``.
 
 In many cases, there is no need to use or write an Ansible module. In particular, the ``script`` module can be used to run arbitrary PowerShell scripts, allowing Windows administrators familiar with PowerShell a very native way to do things, as in the following playbook::
 
@@ -425,7 +429,7 @@ Modules (ps1 files) should start as follows::
 
     # code goes here, reading in stdin as JSON and outputting JSON
 
-The above magic is necessary to tell Ansible to mix in some common code and also know how to push modules out.  The common code contains some nice wrappers around working with hash data structures and emitting JSON results, and possibly a few more useful things.  Regular Ansible has this same concept for reusing Python code - this is just the Windows equivalent.
+The above magic is necessary to tell Ansible to mix in some common code and also know how to push modules out.  The common code contains some nice wrappers around working with hash data structures and emitting JSON results, and possibly a few more useful things.  Regular Ansible has this same concept for reusing Python code - this is just the windows equivalent.
 
 What modules you see in ``windows/`` are just a start.  Additional modules may be submitted as pull requests to github.
 
@@ -434,7 +438,7 @@ What modules you see in ``windows/`` are just a start.  Additional modules may b
 Windows Facts
 `````````````
 
-Just as with Linux/Unix, facts can be gathered for Windows hosts, which will return things such as the operating system version.  To see what variables are available about a Windows host, run the following::
+Just as with Linux/Unix, facts can be gathered for windows hosts, which will return things such as the operating system version.  To see what variables are available about a windows host, run the following::
 
     ansible winhost.example.com -m setup
 

docs/docsite/rst/network_debug_troubleshooting.rst

@@ -32,7 +32,7 @@ Errors generally fall into one of the following categories:
   * Can occur when trying to pull a large amount of data
   * May actually be masking a authentication issue
 :Playbook issues:
-  * Use of ``delegate_to``, instead of ``ProxyCommand``. See :ref:`network proxy guide <network_delegate_to_vs_ProxyCommand>` for more information.
+  * Use of ``delegate_to``, instead of ``ProxyCommand``
   * Not using ``connection: local``
 
 
@@ -51,9 +51,7 @@ Enabling Networking logging and how to read the logfile
 
 Ansible 2.3 features improved logging to help diagnose and troubleshoot issues regarding Ansible Networking modules.
 
-Because logging is very verbose it is disabled by default. It can be enabled via the :envvar:`ANSIBLE_LOG_PATH` and :envvar:`ANSIBLE_DEBUG` options on the ansible-controller, that is the machine running ansible-playbook.
-
-Before running ``ansible-playbook`` run the following commands to enable logging::
+Because logging is very verbose it is disabled by default. It can be enabled via the :envvar:`ANSIBLE_LOG_PATH` and :envvar:`ANSIBLE_DEBUG` options::
 
    # Specify the location for the log file
    export ANSIBLE_LOG_PATH=~/ansible.log
@@ -63,12 +61,10 @@ Before running ``ansible-playbook`` run the following commands to enable logging
    # Run with 4*v for connection level verbosity
    ansible-playbook -vvvv ...
 
-After Ansible has finished running you can inspect the log file which has been created on the ansible-controller:
+After Ansible has finished running you can inspect the log file:
 
 .. code::
 
-  less $ANSIBLE_LOG_PATH
-
   2017-03-30 13:19:52,740 p=28990 u=fred |  creating new control socket for host veos01:22 as user admin
   2017-03-30 13:19:52,741 p=28990 u=fred |  control socket path is /home/fred/.ansible/pc/ca5960d27a
   2017-03-30 13:19:52,741 p=28990 u=fred |  current working directory is /home/fred/ansible/test/integration
@@ -414,10 +410,10 @@ For example:
 
 Suggestions to resolve:
 
-Increase the value of the persistent connection idle timeout.
-Note: This value should be greater than the SSH timeout value (the timeout value under the defaults
-section in the configuration file) and less than the value of the persistent
-connection idle timeout (connect_timeout).
+Increase value of presistent connection idle timeout.
+Note: This value should be greater than SSH timeout ie. timeout value under defaults
+section in configuration file and less than the value of the persistent
+connection idle timeout (connect_timeout)
 
 .. code-block:: yaml
 
@@ -497,7 +493,7 @@ Add ``authorize: yes`` to the task. For example:
 If the user requires a password to go into privileged mode, this can be specified with ``auth_pass``; if ``auth_pass`` isn't set, the environment variable :envvar:`ANSIBLE_NET_AUTHORIZE` will be used instead.
 
 
-Add ``authorize: yes`` to the task. For example:
+Add `authorize: yes` to the task. For example:
 
 .. code-block:: yaml
 
@@ -510,41 +506,40 @@ Add ``authorize: yes`` to the task. For example:
   register: result
 
 
-Proxy Issues
-============
+.. delete_to not honoured
+   ----------------------
 
- .. _network_delegate_to_vs_ProxyCommand:
+   FIXME Do we get an error message
 
-delegate_to vs ProxyCommand
----------------------------
-
-The new connection framework for Network Modules in Ansible 2.3 that uses ``cli`` transport
-no longer supports the use of the ``delegate_to`` directive.
-In order to use a bastion or intermediate jump host to connect to network devices over ``cli``
-transport, network modules now support the use of ``ProxyCommand``.
-
-To use ``ProxyCommand``, configure the proxy settings in the Ansible inventory
-file to specify the proxy host.
-
-.. code-block:: ini
-
-    [nxos]
-    nxos01
-    nxos02
-
-    [nxos:vars]
-    ansible_ssh_common_args='-o ProxyCommand="ssh -W %h:%p -q bastion01"'
+   FIXME Link to howto
 
 
-With the configuration above, simply build and run the playbook as normal with
-no additional changes necessary.  The network module will now connect to the
-network device by first connecting to the host specified in
-``ansible_ssh_common_args``, which is ``bastion01`` in the above example.
 
 
-.. note:: Using ``ProxyCommand`` with passwords via variables
+   fixmes
+   ======
 
-   By design, SSH doesn't support providing passwords via environment variables.
-   This is done to prevent secrets from leaking out, for example in ``ps`` output.
+   Error: "number of connection attempts exceeded, unable to connect to control socket"
+   ------------------------------------------------------------------------------------
 
-   We recommend using SSH Keys, and if needed an ssh-agent, rather than passwords, where ever possible.
+   **Platforms:** Any
+
+   This occurs when Ansible wasn't able to connect to the remote device and obtain a shell with the timeout.
+
+
+   This information is available when :ref:`DEFAULT_LOG_PATH` is set see (FIXMELINKTOSECTION):
+
+   .. code-block:: yaml
+
+     less $ANSIBLE_LOG_PATH
+     2017-03-10 15:32:06,173 p=19677 u=fred |  connect retry timeout expired, unable to connect to control socket
+     2017-03-10 15:32:06,174 p=19677 u=fred |  persistent_connect_retry_timeout is 15 secs
+     2017-03-10 15:32:06,222 p=19669 u=fred |  fatal: [veos01]: FAILED! => {
+
+   Suggestions to resolve:
+
+   Do stuff For example:
+
+   .. code-block:: yaml
+
+   	Example stuff

docs/docsite/rst/playbooks_acceleration.rst

@@ -1,82 +0,0 @@
-Accelerated Mode
-================
-
-.. versionadded:: 1.3
-
-.. note::
-
-     Accelerated mode is deprecated. Consider using SSH with ControlPersist and pipelining enabled instead. This feature will be removed in a future release. Deprecation warnings can be disabled by setting :code:`deprecation_warnings=False` in :code:`ansible.cfg`.
-
-You Might Not Need This!
-````````````````````````
-
-Are you running Ansible 1.5 or later? If so, you may not need accelerated mode due to a new feature called "SSH pipelining" and should read the :ref:`pipelining` section of the documentation.
-
-For users on 1.5 and later, accelerated mode only makes sense if you (A) are managing from an Enterprise Linux 6 or earlier host and still are on paramiko, or (B) can't enable TTYs with sudo as described in the pipelining docs.
-
-If you can use pipelining, Ansible will reduce the amount of files transferred over the wire, 
-making everything much more efficient, and performance will be on par with accelerated mode in nearly all cases, possibly excluding very large file transfer. Because less moving parts are involved, pipelining is better than accelerated mode for nearly all use cases.
-
-Accelerated mode remains around in support of EL6
-control machines and other constrained environments.
-
-Accelerated Mode Details
-````````````````````````
-
-While OpenSSH using the ControlPersist feature is quite fast and scalable, there is a certain small amount of overhead involved in
-using SSH connections. While many people will not encounter a need, if you are running on a platform that doesn't have ControlPersist support (such as an EL6 control machine), you'll probably be even more interested in tuning options.
-
-Accelerated mode is there to help connections work faster, but still uses SSH for initial secure key exchange. There is no
-additional public key infrastructure to manage, and this does not require things like NTP or even DNS. 
-
-Accelerated mode can be anywhere from 2-6x faster than SSH with ControlPersist enabled, and 10x faster than paramiko.
-
-Accelerated mode works by launching a temporary daemon over SSH. Once the daemon is running, Ansible will connect directly
-to it via a socket connection. Ansible secures this communication by using a temporary AES key that is exchanged during
-the SSH connection (this key is different for every host, and is also regenerated periodically). 
-
-By default, Ansible will use port 5099 for the accelerated connection, though this is configurable. Once running, the daemon will accept connections for 30 minutes, after which time it will terminate itself and need to be restarted over SSH.
-
-In order to use accelerated mode, simply add `accelerate: true` to your play::
-
-    ---
-
-    - hosts: all
-      accelerate: true
-
-      tasks:
-
-      - name: some task
-        command: echo {{ item }}
-        with_items:
-        - foo
-        - bar
-        - baz
-
-If you wish to change the port Ansible will use for the accelerated connection, just add the `accelerate_port` option::
-
-    ---
-
-    - hosts: all
-      accelerate: true
-      # default port is 5099
-      accelerate_port: 10000
-
-The `accelerate_port` option can also be specified in the environment variable :envvar:`ACCELERATE_PORT`, or in your `ansible.cfg` configuration::
-
-    [accelerate]
-    accelerate_port = 5099
-
-As noted above, accelerated mode also supports running tasks via sudo, however there are two important caveats:
-
-* You must remove requiretty from your sudoers options.
-* Prompting for the sudo password is not yet supported, so the NOPASSWD option is required for sudo'ed commands.
-
-As of Ansible version `1.6`, you can also allow the use of multiple keys for connections from multiple Ansible management nodes. To do so, add the following option
-to your `ansible.cfg` configuration::
-
-    accelerate_multi_key = yes
-
-When enabled, the daemon will open a UNIX socket file (by default `$ANSIBLE_REMOTE_TEMP/.ansible-accelerate/.local.socket`). New connections over SSH can
-use this socket file to upload new keys to the daemon.
-

docs/docsite/rst/playbooks_blocks.rst

@@ -61,7 +61,8 @@ Blocks also introduce the ability to handle errors in a way similar to exception
 
 The tasks in the ``block`` would execute normally, if there is any error the ``rescue`` section would get executed
 with whatever you need to do to recover from the previous error. The ``always`` section runs no matter what previous
-error did or did not occur in the ``block`` and ``rescue`` sections.
+error did or did not occur in the ``block`` and ``rescue`` sections. It should be noted that the play continues if a
+``rescue`` section completes successfully as it 'erases' the error status (but not the reporting), this means it won't trigger ``max_fail_percentage`` nor ``any_errors_fatal`` configurations but will appear in the playbook statistics.
 
 
 Another example is how to run handlers after an error occurred :

docs/docsite/rst/playbooks_filters.rst

@@ -155,26 +155,26 @@ To get a random item from a list::
     "{{ ['a','b','c']|random }}"
     # => 'c'
 
-To get a random number between 0 and a specified number::
+To get a random number from 0 to supplied end::
 
-    "{{ 60 |random}} * * * * root /script/from/cron"
+    "{{ 59 |random}} * * * * root /script/from/cron"
     # => '21 * * * * root /script/from/cron'
 
 Get a random number from 0 to 100 but in steps of 10::
 
-    {{ 101 |random(step=10) }}
+    {{ 100 |random(step=10) }}
     # => 70
 
 Get a random number from 1 to 100 but in steps of 10::
 
-    {{ 101 |random(1, 10) }}
+    {{ 100 |random(1, 10) }}
     # => 31
-    {{ 101 |random(start=1, step=10) }}
+    {{ 100 |random(start=1, step=10) }}
     # => 51
 
 As of Ansible version 2.3, it's also possible to initialize the random number generator from a seed. This way, you can create random-but-idempotent numbers::
 
-    "{{ 60 |random(seed=inventory_hostname) }} * * * * root /script/from/cron"
+    "{{ 59 |random(seed=inventory_hostname) }} * * * * root /script/from/cron"
 
 
 Shuffle Filter
@@ -286,6 +286,15 @@ This example shows ports from cluster1::
 
 .. note:: You can use a variable to make the query more readable.
 
+Or, alternatively::
+
+    - name: "Display all server names from cluster1"
+      debug:
+        var: item
+      with_items: "{{domain_definition|json_query('domain.server[?cluster=`cluster1`].port')}}"
+
+.. note:: Here, quoting literals using backticks avoids escaping quotes and maintains readability.
+
 In this example, we get a hash map with all ports and names of a cluster::
 
     - name: "Display all server ports and names from cluster1"
@@ -442,7 +451,10 @@ To get a sha512 password hash (random salt)::
 To get a sha256 password hash with a specific salt::
 
     {{ 'secretpassword'|password_hash('sha256', 'mysecretsalt') }}
+    
+An idempotent method to generate unique hashes per system is to use a salt that is consistent between runs::
 
+    {{ 'secretpassword'|password_hash('sha512', 65534|random(seed=inventory_hostname)|string) }}
 
 Hash types available depend on the master system running ansible,
 'hash' depends on hashlib password_hash depends on passlib (http://passlib.readthedocs.io/en/stable/lib/passlib.hash.html).
@@ -601,34 +613,34 @@ URL Split Filter
 
 The ``urlsplit`` filter extracts the fragment, hostname, netloc, password, path, port, query, scheme, and username from an URL. With no arguments, returns a dictionary of all the fields::
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('hostname') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('hostname') }}
     # => 'www.acme.com'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('netloc') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('netloc') }}
     # => 'user:password@www.acme.com:9000'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('username') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('username') }}
     # => 'user'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('password') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('password') }}
     # => 'password'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('path') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('path') }}
     # => '/dir/index.html'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('port') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('port') }}
     # => '9000'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('scheme') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('scheme') }}
     # => 'http'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('query') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('query') }}
     # => 'query=term'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('fragment') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('fragment') }}
     # => 'fragment'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit }}
     # =>
     #   {
     #       "fragment": "fragment",
@@ -695,11 +707,11 @@ To add quotes for shell usage::
 
 To use one value on true and another on false (new in version 1.9)::
 
-    {{ (name == "John") | ternary('Mr','Ms') }}
+   {{ (name == "John") | ternary('Mr','Ms') }}
 
 To concatenate a list into a string::
 
-    {{ list | join(" ") }}
+   {{ list | join(" ") }}
 
 To get the last name of a file path, like 'foo.txt' out of '/etc/asdf/foo.txt'::
 
@@ -735,7 +747,7 @@ To expand a path containing a tilde (`~`) character (new in version 1.5)::
 
 To get the real path of a link (new in version 1.8)::
 
-    {{ path | realpath }}
+   {{ path | realpath }}
 
 To get the relative path of a link, from a start point (new in version 1.7)::
 

docs/docsite/rst/playbooks_intro.rst

@@ -114,18 +114,26 @@ the web servers, and then the database servers. For example::
 
       tasks:
       - name: ensure apache is at the latest version
-        yum: name=httpd state=latest
+        yum:
+          name: httpd
+          state: latest
       - name: write the apache config file
-        template: src=/srv/httpd.j2 dest=/etc/httpd.conf
+        template:
+          src: /srv/httpd.j2
+          dest: /etc/httpd.conf
 
     - hosts: databases
       remote_user: root
 
       tasks:
       - name: ensure postgresql is at the latest version
-        yum: name=postgresql state=latest
+        yum:
+          name: postgresql
+          state: latest
       - name: ensure that postgresql is started
-        service: name=postgresql state=started
+        service:
+          name: postgresql
+          state: started
 
 You can use this method to switch between the host group you're targeting,
 the username logging into the remote servers, whether to sudo or not, and so
@@ -187,7 +195,9 @@ You can also use become on a particular task instead of the whole play::
     - hosts: webservers
       remote_user: yourname
       tasks:
-        - service: name=nginx state=started
+        - service: 
+            name: nginx
+            state: started
           become: yes
           become_method: sudo
 
@@ -302,7 +312,9 @@ the service module takes ``key=value`` arguments::
 
    tasks:
      - name: make sure apache is running
-       service: name=httpd state=started
+       service:
+         name: httpd
+         state: started
 
 The **command** and **shell** modules are the only modules that just take a list
 of arguments and don't use the ``key=value`` form.  This makes
@@ -340,7 +352,9 @@ a variable called ``vhost`` in the ``vars`` section, you could do this::
 
    tasks:
      - name: create a virtual host file for {{ vhost }}
-       template: src=somefile.j2 dest=/etc/httpd/conf.d/{{ vhost }}
+       template:
+         src: somefile.j2
+         dest: /etc/httpd/conf.d/{{ vhost }}
 
 Those same variables are usable in templates, which we'll get to later.
 
@@ -384,7 +398,9 @@ Here's an example of restarting two services when the contents of a file
 change, but only if the file changes::
 
    - name: template configuration file
-     template: src=template.j2 dest=/etc/foo.conf
+     template:
+       src: template.j2
+       dest: /etc/foo.conf
      notify:
         - restart memcached
         - restart apache
@@ -402,18 +418,26 @@ Here's an example handlers section::
 
     handlers:
         - name: restart memcached
-          service: name=memcached state=restarted
+          service:
+            name: memcached
+            state: restarted
         - name: restart apache
-          service: name=apache state=restarted
+          service:
+            name: apache
+            state: restarted
 
 As of Ansible 2.2, handlers can also "listen" to generic topics, and tasks can notify those topics as follows::
 
     handlers:
         - name: restart memcached
-          service: name=memcached state=restarted
+          service:
+            name: memcached
+            state: restarted
           listen: "restart web services"
         - name: restart apache
-          service: name=apache state=restarted
+          service:
+            name: apache
+            state:restarted
           listen: "restart web services"
 
     tasks:

docs/docsite/rst/playbooks_lookups.rst

@@ -1,17 +1,13 @@
 Lookups
 -------
 
-Lookup plugins allow access of data in Ansible from outside sources. Like all templating, these plugins are evaluated on the Ansible control
-machine, and can include reading the filesystem but also contacting external datastores and services.
-These values are then made available using the standard templating system in Ansible, and are typically used to load variables or templates with information from those systems.
+Lookup plugins allow access to outside data sources. Like all templating, these plugins are evaluated on the Ansible control machine, and can include reading the filesystem as well as contacting external datastores and services. This data is then made available using the standard templating system in Ansible. 
 
-.. note:: This is considered an advanced feature, and many users will probably not rely on these features.
-
-.. note:: Lookups occur on the local computer, not on the remote computer.
-
-.. note:: Lookups are executed with a cwd relative to the role or play, as opposed to local tasks which are executed with the cwd of the executed script.
-
-.. note:: Since 1.9 you can pass wantlist=True to lookups to use in jinja2 template "for" loops.
+.. note::
+    - Lookups occur on the local computer, not on the remote computer.
+    - They are executed with in the directory containing the role or play, as opposed to local tasks which are executed with the directory of the executed script.
+    - Since Ansible version 1.9, you can pass wantlist=True to lookups to use in jinja2 template "for" loops.
+    - Lookups are an advanced feature. You should have a good working knowledge of Ansible plays before incorporating them.
 
 .. warning:: Some lookups pass arguments to a shell. When using variables from a remote/untrusted source, use the `|quote` filter to ensure safe usage.
 
@@ -24,7 +20,7 @@ Intro to Lookups: Getting File Contents
 
 The file lookup is the most basic lookup type.
 
-Contents can be read off the filesystem as follows::
+Contents can be read from the filesystem as follows::
 
     ---
     - hosts: all
@@ -42,18 +38,14 @@ The Password Lookup
 
 .. note::
 
-    A great alternative to the password lookup plugin, if you don't need to generate random passwords on a per-host basis, would be to use :doc:`playbooks_vault`.  Read the documentation there and consider using it first, it will be more desirable for most applications.
+    We recommend using :doc:`playbooks_vault` instead of the password lookup plugin if you don't need to generate random passwords on a per-host basis.
 
-``password`` generates a random plaintext password and stores it in
+The ``password lookup`` plugin generates a random plaintext password and stores it in
 a file at a given filepath.
 
-(Docs about crypted save modes are pending)
+If the file already exists, it will retrieve its contents, behaving just like with_file. Usage of variables like "{{ inventory_hostname }}" in the filepath can be used to set up random passwords per host (which simplifies password management in 'host_vars' variables).
 
-If the file exists previously, it will retrieve its contents, behaving just like with_file. Usage of variables like "{{ inventory_hostname }}" in the filepath can be used to set
-up random passwords per host (which simplifies password management in 'host_vars' variables).
-
-A special case is using ``/dev/null`` as a path. The password lookup will generate a new random password each time, but will not write it to ``/dev/null``. This can be used when you need a password
-without storing it on the controller.
+A special case is using ``/dev/null`` as a path. The password lookup will generate a new random password each time, but will not write it to ``/dev/null``. This can be used when you need a password without storing it on the controller.
 
 Generated passwords contain a random mix of upper and lowercase ASCII letters, the
 numbers 0-9 and punctuation (". , : - _"). The default length of a generated password is 20 characters.
@@ -74,9 +66,9 @@ This length can be changed by passing an extra parameter::
 
 .. note:: If the file already exists, no data will be written to it. If the file has contents, those contents will be read in as the password. Empty files cause the password to return as an empty string.
 
-Caution: Since this runs on the ansible host as the user running the playbook, and "become" does not apply, the target file must be readable by the playbook user, or, if it does not exist, the playbook user must have sufficient privileges to create it. (So, for example, attempts to write into areas such as /etc will fail unless the entire playbook is being run as root).
+.. warning:: Since the password lookup runs on the Ansible host as the user running the playbook, and "become" does not apply, the target file must be readable by the playbook user, or, if it does not exist, the playbook user must have sufficient privileges to create it. Attempts to write into areas such as /etc will fail unless the entire playbook is being run as root.
 
-Starting in version 1.4, password accepts a "chars" parameter to allow defining a custom character set in the generated passwords. It accepts comma separated list of names that are either string module attributes (ascii_letters,digits, etc) or are used literally::
+Starting in Ansible version 1.4, password accepts a "chars" parameter to allow defining a custom character set in the generated passwords. It accepts comma separated list of names that are either string module attributes (such as ascii_letters or digits) or are used literally::
 
     ---
     - hosts: all
@@ -100,7 +92,7 @@ Starting in version 1.4, password accepts a "chars" parameter to allow defining
 
         # (...)
 
-To enter comma use two commas ',,' somewhere - preferably at the end. Quotes and double quotes are not supported.
+To enter a comma use two commas ',,' somewhere - preferably at the end. Quotes and double quotes are not supported.
 
 .. _passwordstore_lookup:
 

docs/docsite/rst/playbooks_loops.rst

@@ -408,7 +408,7 @@ Negative numbers are not supported.  This works as follows::
             state: present
           with_sequence: count=4
 
-.. _random_choice:
+.. _playbooks_loops_random_choice:
 
 Random Choices
 ``````````````

docs/docsite/rst/playbooks_reuse.rst

@@ -52,8 +52,8 @@ The primary advantage of using ``include*`` statements is looping. When a loop i
 
 Using ``include*`` does have some limitations when compared to ``import*`` statements:
 
-* Tags which only exist inside a dynamic include will not show up in ``--list-tags`` output.
-* Tasks which only exist inside a dynamic include will not show up in ``--list-tasks`` output.
+* Tags which only exist inside a dynamic include will not show up in --list-tags output.
+* Tasks which only exist inside a dynamic include will not show up in --list-tasks output.
 * You cannot use ``notify`` to trigger a handler name which comes from inside a dynamic include (see note below).
 * You cannot use ``--start-at-task`` to begin execution at a task inside a dynamic include.
 

docs/docsite/rst/playbooks_reuse_includes.rst

@@ -69,7 +69,7 @@ Variables can also be passed to include files using an alternative syntax, which
         - "{{ lookup('file', 'keys/one.pub') }}"
         - "{{ lookup('file', 'keys/two.pub') }}"
 
-Using either syntax, variables passed in can then be used in the included files. These variables will only be available to tasks within the included file. See :ref:`variable_precedence` for more details on variable inheritance and precedence.
+Using either syntax, variables passed in can then be used in the included files. These variables will only be available to tasks within the included file. See :ref:`ansible_variable_precedence` for more details on variable inheritance and precedence.
 
 Task include statements can be used at arbitrary depth.
 

docs/docsite/rst/playbooks_reuse_roles.rst

@@ -198,7 +198,7 @@ To make roles run more than once, there are two options:
 1. Pass different parameters in each role definition.
 2. Add ``allow_duplicates: true`` to the ``meta/main.yml`` file for the role.
 
-Example 1 - passing different parameters::
+Example 1 - passing different paramters::
 
     ---
     - hosts: webservers

docs/docsite/rst/playbooks_special_topics.rst

@@ -16,9 +16,10 @@ and adopt these only if they seem relevant or useful to your environment.
    playbooks_environment
    playbooks_error_handling
    playbooks_advanced_syntax
-   playbooks_lookups
+   plugins
    playbooks_prompts
    playbooks_tags
    playbooks_vault
    playbooks_startnstep
    playbooks_keywords
+   playbooks_lookups

docs/docsite/rst/playbooks_variables.rst

@@ -798,7 +798,7 @@ As of Ansible 1.3, extra vars can be loaded from a JSON file with the ``@`` synt
 Also as of Ansible 1.3, extra vars can be formatted as YAML, either on the command line
 or in a file as above.
 
-.. _variable_precedence:
+.. _ansible_variable_precedence:
 
 Variable Precedence: Where Should I Put A Variable?
 ````````````````````````````````````````````````````
@@ -836,16 +836,13 @@ In 2.x, we have made the order of precedence more specific (with the last listed
 
   * role defaults [1]_
   * inventory file or script group vars [2]_
-  * inventory group_vars/all [3]_
-  * playbook group_vars/all [3]_
-  * inventory group_vars/* [3]_
-  * playbook group_vars/* [3]_
+  * inventory group_vars/all
+  * playbook group_vars/all
+  * inventory group_vars/*
+  * playbook group_vars/*
   * inventory file or script host vars [2]_
   * inventory host_vars/*
   * playbook host_vars/*
-  * host facts / cached set_facts [4]_
-  * inventory host_vars/* [3]_
-  * playbook host_vars/* [3]_
   * host facts
   * play vars
   * play vars_prompt
@@ -865,14 +862,13 @@ Basically, anything that goes into "role defaults" (the defaults folder inside t
 
 .. [1] Tasks in each role will see their own role's defaults. Tasks defined outside of a role will see the last role's defaults.
 .. [2] Variables defined in inventory file or provided by dynamic inventory.
-.. [3] Includes vars added by 'vars plugins' as well as host_vars and group_vars which are added by the default vars plugin shipped with Ansible.
-.. [4] When created with set_facts's cacheable option, variables will have the high precedence in the play,
-       but will be the same as a host facts precedence when they come from the cache.
 
 .. note:: Within any section, redefining a var will overwrite the previous instance.
           If multiple groups have the same variable, the last one loaded wins.
           If you define a variable twice in a play's vars: section, the 2nd one wins.
 .. note:: the previous describes the default config `hash_behavior=replace`, switch to 'merge' to only partially overwrite.
+.. note:: Group loading follows parent/child relationships. Groups of the same 'patent/child' level are then merged following alphabetical order.
+          This last one can be superceeded by the user via `ansible_group_priority`, which defaults to 0 for all groups.
 
 
 Another important thing to consider (for all versions) is that connection variables override config, command line and play/role/task specific options and directives.  For example::
@@ -1006,7 +1002,7 @@ can set variables in there and make use of them in other roles and elsewhere in
         - { role: something, foo: 12 }
         - { role: something_else }
 
-.. note:: There are some protections in place to avoid the need to namespace variables.  
+.. note:: There are some protections in place to avoid the need to namespace variables.
           In the above, variables defined in common_settings are most definitely available to 'something' and 'something_else' tasks, but if
           "something's" guaranteed to have foo set at 12, even if somewhere deep in common settings it set foo to 20.
 

docs/docsite/rst/playbooks_vault.rst

@@ -3,9 +3,9 @@ Using Vault in playbooks
 
 .. contents:: Topics
 
-Added in Ansible 1.5, "Vault" is a feature of ansible that allows keeping sensitive data such as passwords or keys in encrypted files, rather than as plaintext in your playbooks or roles. These vault files can then be distributed or placed in source control.
+New in Ansible 1.5, "Vault" is a feature of ansible that allows keeping sensitive data such as passwords or keys in encrypted files, rather than as plaintext in your playbooks or roles. These vault files can then be distributed or placed in source control.
 
-To enable this feature, a command line tool, :ref:`ansible-vault` is used to edit files, and a command line flag :option:`--ask-vault-pass <ansible-vault --ask-vault-pass>` or :option:`--vault-password-file <ansible-vault --vault-password-file>` is used. You can also modify your ``ansible.cfg`` file to specify the location of a password file or configure Ansible to always prompt for the password. These options require no command line flag usage.
+To enable this feature, a command line tool, `ansible-vault` is used to edit files, and a command line flag `--ask-vault-pass` or `--vault-password-file` is used. Alternately, you may specify the location of a password file or command Ansible to always prompt for the password in your ansible.cfg file. These options require no command line flag usage.
 
 For best practices advice, refer to :ref:`best_practices_for_variables_and_vaults`.
 
@@ -35,7 +35,7 @@ If you are using a script instead of a flat file, ensure that it is marked as ex
 
 This is something you may wish to do if using Ansible from a continuous integration system like Jenkins.
 
-The :option:`--vault-password-file <ansible-pull --vault-password-file>` option can also be used with the :ref:`ansible-pull` command if you wish, though this would require distributing the keys to your nodes, so understand the implications -- vault is more intended for push mode.
+(The `--vault-password-file` option can also be used with the :ref:`ansible-pull` command if you wish, though this would require distributing the keys to your nodes, so understand the implications -- vault is more intended for push mode).
 
 
 .. _single_encrypted_variable:
@@ -66,6 +66,6 @@ Using encrypt_string
 ````````````````````
 
 This command will output a string in the above format ready to be included in a YAML file.
-The string to encrypt can be provided via stdin, command line arguments, or via an interactive prompt.
+The string to encrypt can be provided via stdin, command line args, or via an interactive prompt.
 
 See :ref:`encrypt_string_for_use_in_yaml`.

docs/docsite/rst/plugins.rst

@@ -0,0 +1,29 @@
+Plugins
+=======
+
+Plugins are pieces of code that augment Ansible's core functionality. Ansible uses a plugin architecture to enable a rich, flexible and expandible feature set.
+
+Ansible ships with a number of handy plugins, and you can easily write your own.
+
+This section covers the various types of plugins that are included with Ansible:
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    plugins/*
+    playbooks_filters
+    playbooks_tests
+
+
+.. seealso::
+
+   :doc:`playbooks`
+       An introduction to playbooks
+   :doc:`config`
+       Ansible configuration documentation and settings
+   :doc:`command_line_tools`
+       Ansible tools, description and options
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/action.rst

@@ -0,0 +1,33 @@
+Action Plugins
+---------------
+
+Action plugins act in conjunction with :doc:`modules <../modules>` to execute the actions required by playbook tasks.
+They usually execute automatically in the background doing prerequisite work before modules execute.
+
+The 'normal' action plugin is used for modules that do not already have an action plugin.
+
+Enabling Action Plugins
++++++++++++++++++++++++
+
+You can enable a custom action plugin by either dropping it into the ``action_plugins`` directory adjacent to your play, inside a role, or by putting it in one of the action plugin directory sources configured in :doc:`ansible.cfg <../config>`.
+
+.. seealso::
+
+   :doc:`cache`
+       Ansible Cache plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`shell`
+       Ansible Shell plugins
+   :doc:`strategy`
+       Ansible Strategy plugins
+   :doc:`vars`
+       Ansible Vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/cache.rst

@@ -0,0 +1,64 @@
+.. contents:: Topics
+
+
+Cache Plugins
+-------------
+
+Cache plugin implement a backend caching mechanism that allows Ansible to store gathered facts or inventory source data
+without the performance hit of retrieving them from source.
+
+The default cache plugin is the :doc:`memory <cache/memory>` plugin, which only caches the data for the current execution of Ansible. Other plugins with persistent storage are available to allow caching the data across runs.
+
+
+Enabling Cache Plugins
+++++++++++++++++++++++
+
+Only one cache plugin can be active at a time.
+
+You can enable a cache plugin in the Ansible configuration, either via environment variable:
+
+.. code-block:: shell
+
+    export ANSIBLE_CACHE_PLUGIN=jsonfile
+
+or in the ``ansible.cfg`` file:
+
+.. code-block:: ini
+
+    [defaults]
+    fact_caching=redis
+
+You will also need to configure other settings specific to each plugin. Consult the individual plugin documentation
+or the Ansible :doc:`configuration <../config>` for more details.
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t cache -l`` to see the list of available plugins. 
+Use ``ansible-doc -t cache <plugin name>`` to see specific documentation and examples.
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    cache/*
+
+.. seealso::
+
+   :doc:`action`
+       Ansible Action plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`shell`
+       Ansible Shell plugins
+   :doc:`strategy`
+       Ansible Strategy plugins
+   :doc:`vars`
+       Ansible Vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/callback.rst

@@ -0,0 +1,103 @@
+.. contents:: Topics
+
+
+Callback Plugins
+----------------
+
+Callback plugins enable adding new behaviors to Ansible when responding to events.
+By default, callback plugins control most of the output you see when running the command line programs,
+but can also be used to add additional output, integrate with other tools and marshall the events to a storage backend.
+
+.. _callback_examples:
+
+Example Callback Plugins
+++++++++++++++++++++++++
+
+The :doc:`log_plays <callback/log_plays>` callback is an example of how to record playbook events to a log file,
+and the :doc:`mail callback/mail` callback sends email on playbook failures.
+
+The :doc:`osx_say <callback/oxs_say>` callback responds with computer synthesized speech on OS X in relation to playbook events.
+
+
+.. _enabling_callbacks:
+
+Enabling Callback Plugins
+++++++++++++++++++++++++++
+
+You can activate a custom callback by either dropping it into a ``callback_plugins`` directory adjacent to your play,  inside a role, or by putting it in one of the callback directory sources configured in :doc:`ansible.cfg <../config>`.
+
+Plugins are loaded in alphanumeric order. For example, a plugin implemented in a file named `1_first.py` would run before a plugin file named `2_second.py`.
+
+Most callbacks shipped with Ansible are disabled by default and need to be whitelisted in your :doc:`ansible.cfg <../config>` file in order to function. For example:
+
+.. code-block:: ini
+
+  #callback_whitelist = timer, mail, profile_roles
+
+
+Managing stdout
+```````````````
+
+You can only have one plugin be the main manager of your console output. If you want to replace the default, you should define CALLBACK_TYPE = stdout in the subclass and then configure the stdout plugin in :doc:`ansible.cfg <../config>`. For example:
+
+.. code-block:: ini
+
+  stdout_callback = dense
+
+or for my custom callback:
+
+.. code-block:: ini
+
+  stdout_callback = mycallback
+
+This only affects :doc:`../ansible-playbook` by default.
+
+Managing AdHoc
+``````````````
+
+The :doc:`ansible <../ansible>` AdHoc command specifically uses a different callback plugin for stdout,
+so there is an extra setting in :doc:`ansible.cfg <../config>` you need to add to use the stdout callback defined above:
+
+.. code-block:: ini
+
+    [defaults]
+    bin_ansible_callbacks=True
+
+You can also set this as an environment variable:
+
+.. code-block:: shell
+
+    export ANSIBLE_LOAD_CALLBACK_PLUGINS=1
+
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t callback -l`` to see the list of available plugins. 
+Use ``ansible-doc -t callback <plugin name>`` to see specific documents and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    callback/*
+
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   :doc:`vars`
+       Ansible vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/connection.rst

@@ -0,0 +1,82 @@
+.. contents:: Topics
+
+
+Connection Plugins
+------------------
+
+Connection plugins allow Ansible to connect to the target hosts so it can execute tasks on them. Ansible ships with many connection plugins, but only one can be used per host at a time.
+
+By default, Ansible ships with several plugins. The most commonly used are the 'paramiko' SSH, native ssh (just called 'ssh'), and 'local' connection types.  All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines.  
+
+The basics of these connection types are covered in the :doc:`../intro_getting_started` section.  
+
+
+.. contents:: Topics
+
+.. _ssh_plugins:
+
+ssh Plugins
++++++++++++
+
+Because ssh is the default protocol used in system administration and the protocol most used in Ansible, ssh options are included in the command line tools. See :doc:`../ansible-playbook` for more details.
+
+
+.. _using_connection_plugins:
+
+Using Connection Plugins
+++++++++++++++++++++++++
+
+The transport can be changed via :doc:`configuration <../config>`, in the command line (``-c``, ``--connection``), as a keyword (:ref:`connection`)
+in your play or by setting the a connection variable (:ref:`ansible_connection`), most often, in your inventory.
+For example, for windows machines you might want o use the :doc:`winrm <connection/winrm>` plugin instead.
+
+Most connection plugins can operate with a minimum configuration. By default they use the :ref:`inventory_hostname` and defaults to find the target host.
+
+Plugins are self-documenting. Each plugin should document its configuration options. The following are connection variables common to most connection plugins:
+
+:ref:ansible_host
+    The name of the host to connect to, if different from the :ref:`inventory_hostname`.
+:ref:ansible_port
+    The ssh port number. For :doc:`ssh <connection/ssh>` and :doc:`paramiko <connection/paramiko>` the default value is 22.
+:ref:ansible_user
+    The default user name to use for log in. Most plugins default to the current user running Ansible.
+
+Each plugin might also have a specific version of a variable that overrides the general version. For example, :ref:`ansible_ssh_host` for the :doc:`ssh <connection/ssh>` plugin.
+
+Enabling Connection Plugins
++++++++++++++++++++++++++++
+
+You can extend Ansible to support other transports (such as SNMP or message bus) by dropping a custom plugin
+into the ``connection_plugins`` directory.
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t connection -l`` to see the list of available plugins.
+Use ``ansible-doc -t connection <plugin name>`` to see detailed documentation and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    connection/*
+
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   :doc:`vars`
+       Ansible vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/inventory.rst

@@ -0,0 +1,61 @@
+.. contents:: Topics
+
+
+Inventory Plugins
+-----------------
+
+Inventory plugins allow users to point at data sources to compile the inventory of hosts that Ansible uses to target tasks, either via the ``-i /path/to/file`` and/or ``-i 'host1, host2`` command line parameters or from other configuration sources.
+
+.. _enabling_inventory_plugins:
+
+Enabling Inventory Plugins
+++++++++++++++++++++++++++
+
+Most inventory plugins shipped with Ansible are disabled by default and need to be whitelisted in your
+:doc:`ansible.cfg <../config>` file in order to function.  This is how the default whitelist looks in the
+config file that ships with Ansible:
+
+.. code-block:: ini
+
+   [inventory]
+   enable_plugins = host_list, script, yaml, ini
+
+This list also establishes the order in which each plugin tries to parse an inventory source. Any plugins left out of the list will not be considered, so you can 'optimize' your inventory loading by minimizing it to what you actually use. For example:
+
+.. code-block:: ini
+
+   [inventory]
+   enable_plugins = advanced_host_list, constructed, yaml
+
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t inventory -l`` to see the list of available plugins. 
+Use ``ansible-doc -t inventory <plugin name>`` to see plugin-specific documentation and examples.
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    inventory/*
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   :doc:`vars`
+       Ansible vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/shell.rst

@@ -0,0 +1,31 @@
+Shell Plugins
+-------------
+
+Shell plugins work transparently to ensure that the basic commands Ansible runs are properly formated to work with the target machine.
+
+Enabling Shell Plugins
+++++++++++++++++++++++
+
+.. warning:: These plugins should not be reconfigured unless you have a restricted or exotic setup
+             in which the default ``/bin/sh`` is not a POSIX compatible shell or not availble for execution. 
+
+In addition to modifying the default configuration settings in :doc:`../config`, you can use a 'connection variable' :ref:`ansible_shell_type` to select a shell plugin, and update the :ref:`ansible_executable` to match.
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/strategy.rst

@@ -0,0 +1,71 @@
+.. contents:: Topics
+
+
+Strategy Plugins
+----------------
+
+Strategy plugins control the flow of play execution by handling task and host scheduling.
+
+
+Enabling Strategy Plugins
++++++++++++++++++++++++++
+
+Only one strategy plugin can be used in a play, but you can use different ones for each play in a playbook or Ansible run.
+The default is the :doc:`linear <strategy/linear>` plugin. You can change this default in Ansible :doc:`configuration <../config>` using an environment variable:
+
+.. code-block:: shell
+
+    export ANSIBLE_STRATEGY=free
+
+or in the `ansible.cfg` file:
+
+.. code-block:: ini
+
+    [defaults]
+    strategy=linear
+
+You can also specify the strategy plugin in the play via the :ref:`strategy` keyword::
+
+  - hosts: all
+    strategy: debug
+    tasks:
+      - copy: src=myhosts dest=/etc/hosts
+        notify: restart_tomcat
+
+      - package: name=tomcat state=present
+
+    handlers:
+      - name: restart_tomcat
+        service: name=tomcat state=restarted
+
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t strategy -l`` to see the list of available plugins. 
+Use ``ansible-doc -t strategy <plugin name>`` to see plugin-specific specific documentation and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    strategy/*
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/vars.rst

@@ -0,0 +1,50 @@
+.. contents:: Topics
+
+
+Vars Plugins
+------------
+
+Vars plugins inject additional variable data into Ansible runs that did not come from an inventory source, playbook, or command line. Playbook constructs like 'host_vars' and 'group_vars' work using vars plugins.
+
+Vars plugins were partially implented in Ansible 2.0 and rewritten to be fully implemented starting with Ansible 2.4.
+
+The :doc:`host_group_vars <vars/host_group_vars>` plugin shipped with Ansible enables reading variables from :ref:`host_vars` and :ref:`group_vars`.
+
+
+Enabling Vars Plugins
++++++++++++++++++++++
+
+You can activate a custom vars plugins by either dropping it into a ``vars_plugins`` directory adjacent to your play,  inside a role, or by putting it in one of the directory sources configured in :doc:`ansible.cfg <../config>`.
+
+Plugin Lists
+++++++++++++
+
+You can use ``ansible-doc -t vars -l`` to see the list of available plugins. 
+Use ``ansible-doc -t vars <plugin name>`` to see specific plugin-specific documentation and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    vars/*
+
+.. seealso::
+
+   :doc:`action`
+       Ansible Action plugins
+   :doc:`cache`
+       Ansible Cache plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`shell`
+       Ansible Shell plugins
+   :doc:`strategy`
+       Ansible Strategy plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/porting_guide_2.0.rst

@@ -4,8 +4,393 @@
 Ansible 2.0 Porting Guide
 *************************
 
-Ansible Porting Guides are maintained in the ``devel`` branch only. Please go to `the devel Ansible 2.0 Porting guide <https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_2.0.html>`_ for up to date information.
+This section discusses the behavioral changes between Ansible 1.x and Ansible 2.0.
 
-.. note::
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
+
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#2.0>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
+
+Playbook
+========
+
+This section discusses any changes you may need to make to your playbooks.
+
+.. code-block:: yaml
+
+    # Syntax in 1.9.x
+    - debug:
+        msg: "{{ 'test1_junk 1\\\\3' | regex_replace('(.*)_junk (.*)', '\\\\1 \\\\2') }}"
+    # Syntax in 2.0.x
+    - debug:
+        msg: "{{ 'test1_junk 1\\3' | regex_replace('(.*)_junk (.*)', '\\1 \\2') }}"
+
+    # Output:
+    "msg": "test1 1\\3"
+
+To make an escaped string that will work on all versions you have two options::
+
+- debug: msg="{{ 'test1_junk 1\\3' | regex_replace('(.*)_junk (.*)', '\\1 \\2') }}"
+
+uses key=value escaping which has not changed.  The other option is to check for the ansible version::
+
+"{{ (ansible_version|version_compare('2.0', 'ge'))|ternary( 'test1_junk 1\\3' | regex_replace('(.*)_junk (.*)', '\\1 \\2') , 'test1_junk 1\\\\3' | regex_replace('(.*)_junk (.*)', '\\\\1 \\\\2') ) }}"
+
+* trailing newline When a string with a trailing newline was specified in the
+  playbook via yaml dict format, the trailing newline was stripped. When
+  specified in key=value format, the trailing newlines were kept. In v2, both
+  methods of specifying the string will keep the trailing newlines. If you
+  relied on the trailing newline being stripped, you can change your playbook
+  using the following as an example::
+
+    # Syntax in 1.9.x
+    vars:
+      message: >
+        Testing
+        some things
+    tasks:
+    - debug:
+        msg: "{{ message }}"
+
+    # Syntax in 2.0.x
+    vars:
+      old_message: >
+        Testing
+        some things
+      message: "{{ old_messsage[:-1] }}"
+    - debug:
+        msg: "{{ message }}"
+    # Output
+    "msg": "Testing some things"
+
+* Behavior of templating DOS-type text files changes with Ansible v2.
+
+  A bug in Ansible v1 causes DOS-type text files (using a carriage return and newline) to be templated to Unix-type text files (using only a newline). In Ansible v2 this long-standing bug was finally fixed and DOS-type text files are preserved correctly. This may be confusing when you expect your playbook to not show any differences when migrating to Ansible v2, while in fact you will see every DOS-type file being completely replaced (with what appears to be the exact same content).
+
+* When specifying complex args as a variable, the variable must use the full jinja2
+  variable syntax (```{{var_name}}```) - bare variable names there are no longer accepted.
+  In fact, even specifying args with variables has been deprecated, and will not be
+  allowed in future versions::
+
+    ---
+    - hosts: localhost
+      connection: local
+      gather_facts: false
+      vars:
+        my_dirs:
+          - { path: /tmp/3a, state: directory, mode: 0755 }
+          - { path: /tmp/3b, state: directory, mode: 0700 }
+      tasks:
+        - file:
+          args: "{{item}}" # <- args here uses the full variable syntax
+          with_items: "{{my_dirs}}"
+
+* porting task includes
+* More dynamic. Corner-case formats that were not supposed to work now do not, as expected.
+* variables defined in the yaml dict format https://github.com/ansible/ansible/issues/13324
+* templating (variables in playbooks and template lookups) has improved with regard to keeping the original instead of turning everything into a string.
+  If you need the old behavior, quote the value to pass it around as a string.
+* Empty variables and variables set to null in yaml are no longer converted to empty strings. They will retain the value of `None`.
+  You can override the `null_representation` setting to an empty string in your config file by setting the :envvar:`ANSIBLE_NULL_REPRESENTATION` environment variable.
+* Extras callbacks must be whitelisted in ansible.cfg. Copying is no longer necessary but whitelisting in ansible.cfg must be completed.
+* dnf module has been rewritten. Some minor changes in behavior may be observed.
+* win_updates has been rewritten and works as expected now.
+* from 2.0.1 onwards, the implicit setup task from gather_facts now correctly inherits everything from play, but this might cause issues for those setting
+  `environment` at the play level and depending on `ansible_env` existing. Previously this was ignored but now might issue an 'Undefined' error.
+
+Deprecated
+----------
+
+While all items listed here will show a deprecation warning message, they still work as they did in 1.9.x. Please note that they will be removed in 2.2 (Ansible always waits two major releases to remove a deprecated feature).
+
+* Bare variables in ``with_`` loops should instead use the ``"{ {var }}"`` syntax, which helps eliminate ambiguity.
+* The ansible-galaxy text format requirements file. Users should use the YAML format for requirements instead.
+* Undefined variables within a ``with_`` loop's list currently do not interrupt the loop, but they do issue a warning; in the future, they will issue an error.
+* Using dictionary variables to set all task parameters is unsafe and will be removed in a future version. For example::
+
+    - hosts: localhost
+      gather_facts: no
+      vars:
+        debug_params:
+          msg: "hello there"
+      tasks:
+        # These are both deprecated:
+        - debug: "{{debug_params}}"
+        - debug:
+          args: "{{debug_params}}"
+
+        # Use this instead:
+        - debug:
+            msg: "{{debug_params['msg']}}"
+
+* Host patterns should use a comma (,) or colon (:) instead of a semicolon (;) to separate hosts/groups in the pattern.
+* Ranges specified in host patterns should use the [x:y] syntax, instead of [x-y].
+* Playbooks using privilege escalation should always use "become*" options rather than the old su*/sudo* options.
+* The "short form" for vars_prompt is no longer supported.
+  For example::
+
+    vars_prompt:
+        variable_name: "Prompt string"
+
+* Specifying variables at the top level of a task include statement is no longer supported. For example::
+
+    - include_tasks: foo.yml
+        a: 1
+
+Should now be::
+
+    - include_tasks: foo.yml
+      vars:
+        a: 1
+
+* Setting any_errors_fatal on a task is no longer supported. This should be set at the play level only.
+* Bare variables in the `environment` dictionary (for plays/tasks/etc.) are no longer supported. Variables specified there should use the full variable syntax: '{{foo}}'.
+* Tags (or any directive) should no longer be specified with other parameters in a task include. Instead, they should be specified as an option on the task.
+  For example::
+
+    - include_tasks: foo.yml tags=a,b,c
+
+  Should be::
+
+    - include_tasks: foo.yml
+      tags: [a, b, c]
+
+* The first_available_file option on tasks has been deprecated. Users should use the with_first_found option or lookup ('first_found', …) plugin.
+
+
+Other caveats
+-------------
+
+Here are some corner cases encountered when updating. These are mostly caused by the more stringent parser validation and the capture of errors that were previously ignored.
+
+* Bad variable composition::
+
+    with_items: myvar_{{rest_of_name}}
+
+  This worked 'by accident' as the errors were retemplated and ended up resolving the variable, it was never intended as valid syntax and now properly returns an error, use the following instead.::
+
+    hostvars[inventory_hostname]['myvar_' + rest_of_name]
+
+* Misspelled directives::
+
+    - task: dostuf
+      becom: yes
+
+  The task always ran without using privilege escalation (for that you need `become`) but was also silently ignored so the play 'ran' even though it should not, now this is a parsing error.
+
+
+* Duplicate directives::
+
+    - task: dostuf
+      when: True
+      when: False
+
+  The first `when` was ignored and only the 2nd one was used as the play ran w/o warning it was ignoring one of the directives, now this produces a parsing error.
+
+* Conflating variables and directives::
+
+    - role: {name=rosy, port=435 }
+
+    # in tasks/main.yml
+    - wait_for: port={{port}}
+
+  The `port` variable is reserved as a play/task directive for overriding the connection port, in previous versions this got conflated with a variable named `port` and was usable
+  later in the play, this created issues if a host tried to reconnect or was using a non caching connection. Now it will be correctly identified as a directive and the `port` variable
+  will appear as undefined, this now forces the use of non conflicting names and removes ambiguity when adding settings and variables to a role invocation.
+
+* Bare operations on `with_`::
+
+    with_items: var1 + var2
+
+  An issue with the 'bare variable' features, which was supposed only template a single variable without the need of braces ({{ )}}, would in some versions of Ansible template full expressions.
+  Now you need to use proper templating and braces for all expressions everywhere except conditionals (`when`)::
+
+    with_items: "{{var1 + var2}}"
+
+  The bare feature itself is deprecated as an undefined variable is indistinguishable from a string which makes it difficult to display a proper error.
+
+Porting plugins
+===============
+
+In ansible-1.9.x, you would generally copy an existing plugin to create a new one. Simply implementing the methods and attributes that the caller of the plugin expected made it a plugin of that type. In ansible-2.0, most plugins are implemented by subclassing a base class for each plugin type. This way the custom plugin does not need to contain methods which are not customized.
+
+
+Lookup plugins
+--------------
+
+* lookup plugins ; import version
+
+
+Connection plugins
+------------------
+
+* connection plugins
+
+Action plugins
+--------------
+
+
+* action plugins
+
+Callback plugins
+----------------
+
+Although Ansible 2.0 provides a new callback API the old one continues to work
+for most callback plugins.  However, if your callback plugin makes use of
+:attr:`self.playbook`, :attr:`self.play`, or :attr:`self.task` then you will
+have to store the values for these yourself as ansible no longer automatically
+populates the callback with them.  Here's a short snippet that shows you how:
+
+.. code-block:: python
+
+    import os
+    from ansible.plugins.callback import CallbackBase
+
+    class CallbackModule(CallbackBase):
+        def __init__(self):
+            self.playbook = None
+            self.playbook_name = None
+            self.play = None
+            self.task = None
+
+        def v2_playbook_on_start(self, playbook):
+            self.playbook = playbook
+            self.playbook_name = os.path.basename(self.playbook._file_name)
+
+        def v2_playbook_on_play_start(self, play):
+            self.play = play
+
+        def v2_playbook_on_task_start(self, task, is_conditional):
+            self.task = task
+
+        def v2_on_any(self, *args, **kwargs):
+            self._display.display('%s: %s: %s' % (self.playbook_name,
+            self.play.name, self.task))
+
+
+Connection plugins
+------------------
+
+* connection plugins
+
+
+Hybrid plugins
+==============
+
+In specific cases you may want a plugin that supports both ansible-1.9.x *and* ansible-2.0. Much like porting plugins from v1 to v2, you need to understand how plugins work in each version and support both requirements.
+
+Since the ansible-2.0 plugin system is more advanced, it is easier to adapt your plugin to provide similar pieces (subclasses, methods) for ansible-1.9.x as ansible-2.0 expects. This way your code will look a lot cleaner.
+
+You may find the following tips useful:
+
+* Check whether the ansible-2.0 class(es) are available and if they are missing (ansible-1.9.x) mimic them with the needed methods (e.g. ``__init__``)
+
+* When ansible-2.0 python modules are imported, and they fail (ansible-1.9.x), catch the ``ImportError`` exception and perform the equivalent imports for ansible-1.9.x. With possible translations (e.g. importing specific methods).
+
+* Use the existence of these methods as a qualifier to what version of Ansible you are running. So rather than using version checks, you can do capability checks instead. (See examples below)
+
+* Document for each if-then-else case for which specific version each block is needed. This will help others to understand how they have to adapt their plugins, but it will also help you to remove the older ansible-1.9.x support when it is deprecated.
+
+* When doing plugin development, it is very useful to have the ``warning()`` method during development, but it is also important to emit warnings for deadends (cases that you expect should never be triggered) or corner cases (e.g. cases where you expect misconfigurations).
+
+* It helps to look at other plugins in ansible-1.9.x and ansible-2.0 to understand how the API works and what modules, classes and methods are available.
+
+
+Lookup plugins
+--------------
+
+As a simple example we are going to make a hybrid ``fileglob`` lookup plugin.
+
+.. code-block:: python
+
+    from __future__ import (absolute_import, division, print_function)
+    __metaclass__ = type
+
+    import os
+    import glob
+
+    try:
+        # ansible-2.0
+        from ansible.plugins.lookup import LookupBase
+    except ImportError:
+        # ansible-1.9.x
+
+        class LookupBase(object):
+            def __init__(self, basedir=None, runner=None, **kwargs):
+                self.runner = runner
+                self.basedir = self.runner.basedir
+
+            def get_basedir(self, variables):
+                return self.basedir
+
+    try:
+        # ansible-1.9.x
+        from ansible.utils import (listify_lookup_plugin_terms, path_dwim, warning)
+    except ImportError:
+        # ansible-2.0
+        from __main__ import display
+        warning = display.warning
+
+    class LookupModule(LookupBase):
+
+        # For ansible-1.9.x, we added inject=None as valid argument
+        def run(self, terms, inject=None, variables=None, **kwargs):
+
+            # ansible-2.0, but we made this work for ansible-1.9.x too !
+            basedir = self.get_basedir(variables)
+
+            # ansible-1.9.x
+            if 'listify_lookup_plugin_terms' in globals():
+                terms = listify_lookup_plugin_terms(terms, basedir, inject)
+
+            ret = []
+            for term in terms:
+                term_file = os.path.basename(term)
+
+                # For ansible-1.9.x, we imported path_dwim() from ansible.utils
+                if 'path_dwim' in globals():
+                    # ansible-1.9.x
+                    dwimmed_path = path_dwim(basedir, os.path.dirname(term))
+                else:
+                    # ansible-2.0
+                    dwimmed_path = self._loader.path_dwim_relative(basedir, 'files', os.path.dirname(term))
+
+                globbed = glob.glob(os.path.join(dwimmed_path, term_file))
+                ret.extend(g for g in globbed if os.path.isfile(g))
+
+            return ret
+
+.. Note:: In the above example we did not use the ``warning()`` method as we had no direct use for it in the final version. However we left this code in so people can use this part during development/porting/use.
+
+
+
+Connection plugins
+------------------
+
+* connection plugins
+
+Action plugins
+--------------
+
+* action plugins
+
+Callback plugins
+----------------
+
+* callback plugins
+
+Connection plugins
+------------------
+
+* connection plugins
+
+
+Porting custom scripts
+======================
+
+Custom scripts that used the ``ansible.runner.Runner`` API in 1.x have to be ported in 2.x.  Please refer to: :doc:`dev_guide/developing_api`
 
-	This link takes you to a different version of the Ansible documentation. Use the version selection on the left or your browser back button to return to this version of the documentation.

docs/docsite/rst/porting_guide_2.3.rst

@@ -4,8 +4,243 @@
 Ansible 2.3 Porting Guide
 *************************
 
-Ansible Porting Guides are maintained in the ``devel`` branch only. Please go to `the devel Ansible 2.3 Porting guide <https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_2.3.html>`_ for up to date information.
+This section discusses the behavioral changes between Ansible 2.2 and Ansible 2.3.
 
-.. note::
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
+
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#2.3>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
+
+Playbook
+========
+
+Restructued async to work with action plugins
+---------------------------------------------
+
+In Ansible 2.2 (and possibly earlier) the `async:` keyword could not be used in conjunction with the action plugins such as `service`. This limitation has been removed in Ansible 2.3
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+    - name: Install nginx asynchronously
+      service:
+        name: nginx
+        state: restarted
+      async: 45
+
+
+OpenBSD version facts
+---------------------
+
+The `ansible_distribution_release` and `ansible_distribution_version` facts on OpenBSD hosts were reversed in Ansible 2.2 and earlier. This has been changed so that version has the numeric portion and release has the name of the release.
+
+**OLD** In Ansible 2.2 (and earlier)
+
+
+.. code-block:: yaml
+
+    "ansible_distribution": "OpenBSD"
+    "ansible_distribution_release": "6.0",
+    "ansible_distribution_version": "release",
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+    "ansible_distribution": "OpenBSD",
+    "ansible_distribution_release": "release",
+    "ansible_distribution_version": "6.0",
+
+
+Names Blocks
+------------
+
+Blocks can now have names, this allows you to avoid the ugly `# this block is for...` comments.
+
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+    - name: Block test case
+      hosts: localhost
+      tasks:
+       - name: Attempt to setup foo
+         block:
+           - debug: msg='I execute normally'
+           - command: /bin/false
+           - debug: msg='I never execute, cause ERROR!'
+         rescue:
+           - debug: msg='I caught an error'
+           - command: /bin/false
+           - debug: msg='I also never execute :-('
+         always:
+           - debug: msg="this always executes"
+
+
+Use of multiple tags
+--------------------
+
+Specifying ``--tags`` (or ``--skip-tags``) multiple times on the command line currently leads to the last specified tag overriding all the other specified tags. This behaviour is deprecated. In the future, if you specify --tags multiple times the tags will be merged together. From now on, using ``--tags`` multiple times on one command line will emit a deprecation warning. Setting the ``merge_multiple_cli_tags`` option to True in the ``ansible.cfg`` file will enable the new behaviour.
+
+In 2.4, the default will be to merge the tags. You can enable the old overwriting behavior via the config option.
+In 2.5, multiple ``--tags`` options will be merged with no way to go back to the old behaviour.
+
+
+Other caveats
+-------------
+
+Here are some rare cases that might be encountered when updating. These are mostly caused by the more stringent parser validation and the capture of errors that were previously ignored.
+
+
+* Made ``any_errors_fatal`` inheritable from play to task and all other objects in between.
+
+Modules
+=======
+
+No major changes in this version.
+
+Modules removed
+---------------
+
+No major changes in this version.
+
+Deprecation notices
+-------------------
+
+The following modules will be removed in Ansible 2.5. Please update your playbooks accordingly.
+
+* :ref:`ec2_vpc <ec2_vpc>`
+* :ref:`cl_bond <cl_bond>`
+* :ref:`cl_bridge <cl_bridge>`
+* :ref:`cl_img_install <cl_img_install>`
+* :ref:`cl_interface <cl_interface>`
+* :ref:`cl_interface_policy <cl_interface_policy>`
+* :ref:`cl_license <cl_license>`
+* :ref:`cl_ports <cl_ports>`
+* :ref:`nxos_mtu <nxos_mtu>` use :ref:`nxos_system <nxos_system>` instead
+
+Noteworthy module changes
+-------------------------
+
+AWS lambda
+^^^^^^^^^^
+Previously ignored changes that only affected one parameter. Existing deployments may have outstanding changes that this bug fix will apply.
+
+
+Mount
+^^^^^
+
+Mount: Some fixes so bind mounts are not mounted each time the playbook runs.
+
+
+Plugins
+=======
+
+No major changes in this version.
+
+Porting custom scripts
+======================
+
+No major changes in this version.
+
+Networking
+==========
+
+There have been a number of changes to number of changes to how Networking Modules operate.
+
+Playbooks should still use ``connection: local``.
+
+The following changes apply to:
+
+* dellos6
+* dellos9
+* dellos10
+* eos
+* ios
+* iosxr
+* junos
+* sros
+* vyos
+
+Deprecation of top-level connection arguments
+---------------------------------------------
+
+**OLD** In Ansible 2.2:
+
+.. code-block:: yaml
+
+    - name: example of using top-level options for connection properties
+      ios_command:
+        commands: show version
+        host: "{{ inventory_hostname }}"
+        username: cisco
+        password: cisco
+        authorize: yes
+        auth_pass: cisco
+
+Will result in:
+
+.. code-block:: yaml
+
+   [WARNING]: argument username has been deprecated and will be removed in a future version
+   [WARNING]: argument host has been deprecated and will be removed in a future version
+   [WARNING]: argument password has been deprecated and will be removed in a future version
+
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+   - name: Gather facts
+     eos_facts:
+       gather_subset: all
+       provider:
+         username: myuser
+         password: "{{ networkpassword }}"
+         transport: cli
+         host: "{{ ansible_host }}"
+
+delegate_to vs ProxyCommand
+---------------------------
+
+The new connection framework for Network Modules in Ansible 2.3 that uses ``cli`` transport
+no longer supports the use of the ``delegate_to`` directive.
+In order to use a bastion or intermediate jump host to connect to network devices over ``cli``
+transport, network modules now support the use of ``ProxyCommand``.
+
+To use ``ProxyCommand`` configure the proxy settings in the Ansible inventory
+file to specify the proxy host.
+
+.. code-block:: ini
+
+    [nxos]
+    nxos01
+    nxos02
+
+    [nxos:vars]
+    ansible_ssh_common_args='-o ProxyCommand="ssh -W %h:%p -q bastion01"'
+
+
+With the configuration above, simply build and run the playbook as normal with
+no additional changes necessary.  The network module will now connect to the
+network device by first connecting to the host specified in
+``ansible_ssh_common_args`` which is ``bastion01`` in the above example.
+
+
+.. notes: Using ``ProxyCommand`` with passwords via variables
+
+   It is a feature that SSH doesn't support providing passwords via environment variables.
+   This is done to prevent secrets from leaking out, for example in ``ps`` output.
+
+   We recommend using SSH Keys, and if needed and ssh-agent, rather than passwords, where ever possible.
 
-	This link takes you to a different version of the Ansible documentation. Use the version selection on the left or your browser back button to return to this version of the documentation.

docs/docsite/rst/porting_guide_2.4.rst

@@ -1,14 +1,156 @@
-:orphan:
-
-
 .. _porting_2.4_guide:
 
 *************************
 Ansible 2.4 Porting Guide
 *************************
 
-Ansible Porting Guides are maintained in the ``devel`` branch only. Please go to `the devel Ansible 2.4 Porting guide <https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_2.4.html>`_ for up to date information.
+This section discusses the behavioral changes between Ansible 2.3 and Ansible 2.4.
 
-.. note::
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
 
-	This link takes you to a different version of the Ansible documentation. Use the version selection on the left or your browser back button to return to this version of the documentation.
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/stable-2.4/CHANGELOG.md#2.4>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
+
+Python version
+==============
+
+Ansible will not support Python 2.4 nor 2.5 on the target hosts anymore. Going forward, Python 2.6+ will be required on targets, as already is the case on the controller.
+
+
+Inventory
+=========
+
+Inventory has been refactored to be implemented via plugins and now allows for multiple sources. This change is mostly transparent to users. 
+
+One exception is the ``inventory_dir``, which is now a host variable; previously it could only have one value so it was set globally. This means you cannot use it early in plays anymore to determine ``hosts:`` or similar keywords.
+
+The ``inventory_file`` remains unchaged, as it was always host specific.
+
+
+Deprecated
+==========
+
+Specifying Inventory sources
+-----------------------------
+
+Use of ``--inventory-file`` on the command line is now deprecated. Use ``--inventory`` or ``-i``.
+The associated ini configuration key, ``hostfile``, and environment variable, :envvar:`ANSIBLE_HOSTS`,
+are also deprecated.  Replace them with the configuration key ``inventory`` and environment variable :envvar:        `ANSIBLE_INVENTORY`.
+
+Use of multiple tags
+--------------------
+
+Specifying ``--tags`` (or ``--skip-tags``) multiple times on the command line currently leads to the last one overriding all the previous ones. This behavior is deprecated. In the future, if you specify --tags multiple times the tags will be merged together. From now on, using ``--tags`` multiple times on one command line will emit a deprecation warning. Setting the ``merge_multiple_cli_tags`` option to True in the ``ansible.cfg`` file will enable the new behavior.
+
+In 2.4, the default has change to merge the tags. You can enable the old overwriting behavior via the config option.
+
+In 2.5, multiple ``--tags`` options will be merged with no way to go back to the old behavior.
+
+
+Other caveats
+-------------
+
+No major changes in this version.
+
+Modules
+=======
+
+Major changes in popular modules are detailed here
+
+* The :ref:`win_shell <win_shell>` and :ref:`win_command <win_command>` modules now properly preserve quoted arguments in the command-line. Tasks that attempted to work around the issue by adding extra quotes/escaping may need to be reworked to remove the superfluous escaping. See `Issue 23019 <https://github.com/ansible/ansible/issues/23019>`_ for additional detail.
+
+Modules removed
+---------------
+
+The following modules no longer exist:
+
+* None
+
+Deprecation notices
+-------------------
+
+The following modules will be removed in Ansible 2.8. Please update update your playbooks accordingly.
+
+* :ref:`azure <azure>`, use :ref:`azure_rm_virtualmachine <azure_rm_virtualmachine>`, which uses the new Resource Manager SDK.
+* :ref:`win_msi <win_msi>`, use :ref:`win_package <win_package>` instead
+
+Noteworthy module changes
+-------------------------
+
+* The :ref:`win_get_url <win_get_url>`  module has the dictionary ``win_get_url`` in its results deprecated, its content is now also available directly in the resulting output, like other modules. This dictionary will be removed in Ansible 2.8.
+* The :ref:`win_unzip <win_unzip>` module no longer includes the dictionary ``win_unzip`` in its results; the contents are now included directly in the resulting output, like other modules.
+* The :ref:`win_package <win_package>` module return values ``exit_code`` and ``restart_required`` have been deprecated in favour of ``rc`` and ``reboot_required`` respectively. The deprecated return values will be removed in Ansible 2.6.
+
+
+Plugins
+=======
+
+A new way to configure and document plugins has been introduced.  This does not require changes to existing setups but developers should start adapting to the new infrastructure now. More details will be available in the developer documentation for each plugin type.
+
+Vars plugin changes
+-------------------
+
+There have been many changes to the implementation of vars plugins, but both users and developers should not need to change anything to keep current setups working. Developers should consider changing their plugins take advantage of new features.
+
+The most notable difference to users is that vars plugins now get invoked on demand instead of at inventory build time.  This should make them more efficient for large inventories, especially when using a subset of the hosts.
+
+.. note:: This also creates a difference with group/host_vars when using them adjacent to playbooks. Before, the 'first' playbook loaded determined the variables; now the 'current' playbook does. We are looking to fix this soon, since 'all playbooks' in the path should be considered for variable loading.
+
+
+Inventory plugins
+-----------------
+
+Developers should start migrating from hardcoded inventory with dynamic inventory scripts to the new Inventory Plugins. The scripts will still work via the ``script`` inventory plugin but Ansible development efforts will now concentrate on writing plugins rather than enhancing existing scripts.
+
+Both users and developers should look into the new plugins because they are intended to alleviate the need for many of the hacks and workarounds found in the dynamic inventory scripts.
+
+Callback plugins
+----------------
+
+Users:
+
+* Callbacks are now using the new configuration system.  Users should not need to change anything as the old system still works,
+  but you might see a deprecation notice if any callbacks used are not inheriting from the built in classes. Developers need to update them as stated below.
+
+Developers:
+
+* If your callback does not inherit from ``CallbackBase`` (directly or indirectly via another callback), it will still work, but issue a deprecation notice.
+  To avoid this and ensure it works in the future change it to inherit from ``callbackBase`` so it has the new options handling methods and properties.
+  You can also implement the new options handling methods and properties but that won't automatically inherit changes added in the future.  You can look at ``CallbackBase`` itself and/or ``AnsiblePlugin`` for details.
+* Any callbacks inheriting from other callbacks might need to also be updated to contain the same documented options
+  as the parent or the options won't be available.  This is noted in the developer guide.
+
+Networking
+==========
+
+There have been a number of changes to how Networking Modules operate.
+
+Playbooks should still use ``connection: local``.
+
+Persistent Connection
+---------------------
+
+The configuration variables ``connection_retries`` and ``connect_interval`` which were added in Ansible 2.3 are now deprecated. For Ansible 2.4 and later use ``connection_retry_timeout``.
+
+To control timeouts use ``command_timeout`` rather than the previous top level ``timeout`` variable under ``[default]``
+
+See :ref:`Ansible Network debug guide <network_debug_troubleshooting>` for more information.
+
+
+Configuration
+=============
+
+
+The configuration system has had some major changes. Users should be unaffected except for the following:
+
+* All relative paths defined are relative to the `ansible.cfg` file itself. Previously they varied by setting. The new behavior should be more predictable.
+* A new macro ``{{CWD}}`` is available for paths, which will make paths relative to the 'current working directory',
+  this is unsafe but some users really want to rely on this behaviour.
+
+Developers that were working directly with the previous API should revisit their usage as some methods (for example, ``get_config``) were  kept for backwards compatibility but will warn users that the function has been deprecated.
+
+The new configuration has been designed to minimize the need for code changes in core for new plugins.  The plugins just need to document their settings and the configuration system will use the documentation to provide what they need. This is still a work in progress; currently only 'callback' and 'connection' plugins support this.  More  details will be added to the specific plugin developer guides.