Use events instead of sleep for result reading

Instead of creating a one-time use thread for every host/task

.github/BOTMETA.yml

@@ -28,17 +28,16 @@
 #           deprecated - this file is deprecated but probably not yet renamed
 #           keywords - used to identify this file based on the issue description
 #           support - used for files without internal metadata
-#           labels - list of GitHub labels to apply
+#           labels - list of GitHub labels to apply. Path components of 'file' parent key
+#                    which are valid GitHub labels are automatically added.
 #
 
-automerge: False
+automerge: True
 files:
   .github/BOTMETA.yml:
-    labels:
-    - botmeta
+    labels: botmeta
   $modules/cloud/amazon/:
-    notify:
-    - willthames
+    notify: willthames
   $modules/cloud/amazon/aws_api_gateway.py: mikedlr willthames
   $modules/cloud/amazon/aws_direct_connect_connection.py: s-hertel
   $modules/cloud/amazon/aws_direct_connect_link_aggregation_group.py: s-hertel
@@ -55,7 +54,7 @@ files:
   $modules/cloud/amazon/ec2_ami.py: Constantin07 gunzy83 scicoin-project willthames
   $modules/cloud/amazon/ec2_ami_copy.py: amir343
   $modules/cloud/amazon/ec2_ami_find.py: tombamford
-  $modules/cloud/amazon/ec2_asg.py: $team_ansible garethr
+  $modules/cloud/amazon/ec2_asg.py: $team_ansible garethr s-hertel ryansb
   $modules/cloud/amazon/ec2_asg_facts.py: wimnat
   $modules/cloud/amazon/ec2_customer_gateway.py: MichaelBaydoun michaelbaydoun
   $modules/cloud/amazon/ec2_eip.py: rickmendes
@@ -173,14 +172,18 @@ files:
   $modules/cloud/azure/azure_rm_virtualnetwork_facts.py: $team_azure
   $modules/cloud/centurylink/: clc-runner
   $modules/cloud/cloudscale/cloudscale_server.py: gaudenz
-  $modules/cloud/cloudstack/: resmo
-  $modules/cloud/digital_ocean/: bondanthony
+  $modules/cloud/cloudstack/:
+    labels: cloudstack
+    maintainers: resmo
+  $modules/cloud/digital_ocean/: BondAnthony
   $modules/cloud/digital_ocean/digital_ocean.py: alukovenko zbal
   $modules/cloud/digital_ocean/digital_ocean_domain.py: alukovenko mgregson
   $modules/cloud/digital_ocean/digital_ocean_sshkey.py: alukovenko mgregson
   $modules/cloud/digital_ocean/digital_ocean_tag.py: kontrafiktion
   $modules/cloud/dimensiondata/dimensiondata_network.py: aimonb tintoy
-  $modules/cloud/docker/docker_container.py: ThomasSteinbach $team_ansible cove dusdanig jctanner kassiansun softzilla zfil
+  $modules/cloud/docker/docker_container.py:
+    ignored: ThomasSteinbach
+    maintainers: $team_ansible cove dusdanig jctanner kassiansun softzilla zfil
   $modules/cloud/docker/docker_image.py: $team_ansible jctanner softzilla
   $modules/cloud/docker/docker_image_facts.py: $team_ansible jctanner
   $modules/cloud/docker/docker_login.py: jctanner olsaki
@@ -213,7 +216,7 @@ files:
   $modules/cloud/misc/helm.py: flaper87
   $modules/cloud/misc/ovirt.py:
     ignored: vincentvdk
-    maintainers: joshainglis karmab
+    maintainers: joshainglis karmab machacekondra mwperina
   $modules/cloud/misc/proxmox.py: UnderGreen
   $modules/cloud/misc/proxmox_kvm.py: helldorado
   $modules/cloud/misc/proxmox_template.py: UnderGreen
@@ -227,7 +230,7 @@ files:
   $modules/cloud/openstack/os_keystone_service.py: $team_openstack SamYaple
   $modules/cloud/openstack/os_project.py: $team_openstack agireud
   $modules/cloud/ovh/ovh_ip_loadbalancing_backend.py: pascalheraud
-  $modules/cloud/ovirt/: machacekondra
+  $modules/cloud/ovirt/: machacekondra mwperina
   $modules/cloud/packet/:
   $modules/cloud/profitbricks/: baldwinSPC
   $modules/cloud/pubnub/pubnub_blocks.py: parfeon pubnub
@@ -268,21 +271,21 @@ files:
   $modules/cloud/vmware/vmware_guest.py: $team_vmware
   $modules/cloud/vmware/vmware_guest_facts.py: nerzhul
   $modules/cloud/vmware/vmware_guest_find.py: $team_vmware
-  $modules/cloud/vmware/vmware_host.py: $team_ansible jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_host.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_local_user_manager.py: $team_ansible
   $modules/cloud/vmware/vmware_maintenancemode.py: $team_ansible
-  $modules/cloud/vmware/vmware_migrate_vmk.py: $team_ansible jcpowermac mtnbikenc
-  $modules/cloud/vmware/vmware_portgroup.py: $team_ansible jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_migrate_vmk.py: $team_ansible jcpowermac
+  $modules/cloud/vmware/vmware_portgroup.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_resource_pool.py: dav1x
   $modules/cloud/vmware/vmware_target_canonical_facts.py: $team_ansible
   $modules/cloud/vmware/vmware_vm_facts.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_vm_shell.py: chrrrles ritzk
   $modules/cloud/vmware/vmware_vm_vss_dvs_migrate.py: $team_ansible jcpowermac
-  $modules/cloud/vmware/vmware_vmkernel.py: $team_ansible jcpowermac mtnbikenc
-  $modules/cloud/vmware/vmware_vmkernel_ip_config.py: $team_ansible jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_vmkernel.py: $team_ansible jcpowermac
+  $modules/cloud/vmware/vmware_vmkernel_ip_config.py: $team_ansible jcpowermac
   $modules/cloud/vmware/vmware_vmotion.py: $team_ansible bedecarroll
-  $modules/cloud/vmware/vmware_vsan_cluster.py: $team_ansible mtnbikenc
-  $modules/cloud/vmware/vmware_vswitch.py: akasurde jcpowermac mtnbikenc
+  $modules/cloud/vmware/vmware_vsan_cluster.py: $team_ansible
+  $modules/cloud/vmware/vmware_vswitch.py: akasurde jcpowermac
   $modules/cloud/vmware/vsphere_copy.py: dagwieers
   $modules/cloud/vmware/vsphere_guest.py: $team_ansible rhoop
   $modules/cloud/webfaction/: quentinsf
@@ -306,8 +309,7 @@ files:
   $modules/database/misc/riak.py: drewkerrigan jsmartin
   $modules/database/mongodb/mongodb_parameter.py: nerzhul
   $modules/database/mongodb/mongodb_user.py: lujeni
-  $modules/database/mssql/mssql_db.py: Jmainguy vedit
-  $modules/database/mysql/mysql_db.py: $team_ansible
+  $modules/database/mssql/mssql_db.py: Jmainguy vedit kenichi-ogawa-1988
   $modules/database/mysql/mysql_replication.py: banyek
   $modules/database/mysql/mysql_user.py: Jmainguy
   $modules/database/mysql/mysql_variables.py: banyek
@@ -334,21 +336,16 @@ files:
   $modules/files/synchronize.py: $team_ansible tima
   $modules/files/tempfile.py:
   $modules/files/template.py: $team_ansible
-  $modules/files/unarchive.py: dagwieers pileofrogs
+  $modules/files/unarchive.py:
+    labels: unarchive
+    maintainers: pileofrogs
   $modules/files/xattr.py: $team_ansible bcoca
   $modules/files/xml.py:
     ignored: magnus919
+    labels: xml
     maintainers: tbielawa dagwieers sm4rk0 cmprescott
-  $modules/identity/ipa/ipa_dnsrecord.py: Nosmoht akasurde
-  $modules/identity/ipa/ipa_group.py: Nosmoht
-  $modules/identity/ipa/ipa_hbacrule.py: Nosmoht
-  $modules/identity/ipa/ipa_host.py: Nosmoht
-  $modules/identity/ipa/ipa_hostgroup.py: Nosmoht
-  $modules/identity/ipa/ipa_role.py: Nosmoht
-  $modules/identity/ipa/ipa_sudocmd.py: Nosmoht
-  $modules/identity/ipa/ipa_sudocmdgroup.py: Nosmoht
-  $modules/identity/ipa/ipa_sudorule.py: Nosmoht
-  $modules/identity/ipa/ipa_user.py: Nosmoht
+  $modules/identity/ipa/:
+    maintainers: $team_ipa
   $modules/identity/opendj/opendj_backendprop.py: dj-wasabi
   $modules/inventory/add_host.py: $team_ansible
   $modules/inventory/group_by.py: $team_ansible jhoekx
@@ -410,12 +407,16 @@ files:
   $modules/net_tools/omapi_host.py: nerzhul
   $modules/net_tools/snmp_facts.py: ogenstad
   $modules/network/a10/: ericchou1 mischapeters
-  $modules/network/aci/: dagwieers schunduri jmcgill298
+  $modules/network/aci/:
+    labels: aci
+    maintainers: dagwieers schunduri jmcgill298
   $modules/network/aireos/: jmighion
   $modules/network/aos/: dgarros jeremyschulman
   $modules/network/aruba/: jmighion
   $modules/network/asa/: ogenstad
-  $modules/network/avi/: $team_avi
+  $modules/network/avi/:
+    labels: avi
+    maintainers: $team_avi
   $modules/network/bigswitch/: jayakody tedelhourani vuile
   $modules/network/citrix/netscaler.py: $team_ansible
   $modules/network/cloudengine/: QijunPan
@@ -424,7 +425,10 @@ files:
   $modules/network/dellos6/: abirami-n skg-net
   $modules/network/dellos9/: dhivyap skg-net
   $modules/network/eos/: privateip trishnaguha
-  $modules/network/f5/: caphrim007
+  $modules/network/f5/:
+    ignored: Etienne-Carriere mhite mryanlam perzizzle srvg wojtek0806
+    labels: f5
+    maintainers: caphrim007
   $modules/network/fortios/: bjolivot
   $modules/network/illumos/: xen0l
   $modules/network/interface/: $team_networking
@@ -433,7 +437,7 @@ files:
   $modules/network/junos/: Qalthos ganeshrn
   $modules/network/layer2/: $team_networking
   $modules/network/layer3/: $team_networking
-  $modules/network/lenovo/: dkasberg
+  $modules/network/lenovo/: dkasberg amuraleedhar
   $modules/network/netconf/netconf_config.py: ganeshrn lpenz userlerueda
   $modules/network/netscaler/: $team_netscaler
   $modules/network/netvisor/: $team_netvisor
@@ -510,13 +514,13 @@ files:
   $modules/packaging/os/pkgng.py:
     ignored: bleader
     maintainers: JoergFiedler
-  $modules/packaging/os/pkgutil.py: dermute
+  $modules/packaging/os/pkgutil.py: dermute scathatheworm
   $modules/packaging/os/portage.py:
     ignored: sayap
     maintainers: wltjr
   $modules/packaging/os/portinstall.py: berenddeboer
   $modules/packaging/os/pulp_repo.py: sysadmind
-  $modules/packaging/os/redhat_subscription.py: alikins barnabycourt
+  $modules/packaging/os/redhat_subscription.py: alikins kahowell
   $modules/packaging/os/rhn_channel.py: alikins vincentvdk
   $modules/packaging/os/rhn_register.py: $team_rhn
   $modules/packaging/os/rpm_key.py: hacosta
@@ -526,7 +530,7 @@ files:
   $modules/packaging/os/swdepot.py: melodous
   $modules/packaging/os/swupd.py: albertomurillo
   $modules/packaging/os/urpmi.py: pmakowski
-  $modules/packaging/os/yum.py: $team_ansible berenddeschouwer verm666
+  $modules/packaging/os/yum.py: $team_ansible berenddeschouwer verm666 mkrizek
   $modules/packaging/os/yum_repository.py: jtyr
   $modules/packaging/os/zypper.py:
     ignored: dirtyharrycallahan
@@ -610,7 +614,9 @@ files:
   $modules/system/selinux_permissive.py: mscherer
   $modules/system/seport.py: dankeder
   $modules/system/service.py: $team_ansible
-  $modules/system/setup.py: $team_ansible
+  $modules/system/setup.py:
+    ignored: david_obrien
+    maintainers: $team_ansible
   $modules/system/solaris_zone.py: pmarkham
   $modules/system/svc.py: bcoca
   $modules/system/sysctl.py: akasurde davixx
@@ -639,7 +645,9 @@ files:
   $modules/web_infrastructure/ejabberd_user.py: privateip
   $modules/web_infrastructure/gunicorn.py: agmezr
   $modules/web_infrastructure/htpasswd.py: $team_ansible
-  $modules/web_infrastructure/jboss.py: jhoekx
+  $modules/web_infrastructure/jboss.py:
+    labels: jboss
+    maintainers: jhoekx
   $modules/web_infrastructure/jenkins_job.py: sermilrod
   $modules/web_infrastructure/jenkins_plugin.py: jtyr
   $modules/web_infrastructure/jenkins_script.py: hogarthj
@@ -809,15 +817,13 @@ files:
     - dynamic inventory script
     - dynamic inventory
     - inventory script
-    labels:
-    - "c:inventory/contrib_script"
+    labels: "c:inventory/contrib_script"
     support: community
   contrib/inventory/linode.py:
     keywords:
     - linode dynamic inventory script
     maintainers: intheclouddan rwaweber zbal
-    labels:
-    - cloud
+    labels: cloud
   contrib/inventory/openstack.py:
     keywords:
     - openstack dynamic inventory script
@@ -865,7 +871,9 @@ files:
     labels: networking
   $module_utils/aci.py:
     maintainers: dagwieers schunduri jmcgill298
-    labels: networking
+    labels:
+    - aci
+    - networking
   $module_utils/aireos.py:
     maintainers: jmighion
     labels: networking
@@ -880,7 +888,9 @@ files:
     labels: networking
   $module_utils/avi.py:
     maintainers: $team_avi
-    labels: networking
+    labels:
+    - avi
+    - networking
   $module_utils/bigswitch_utils.py:
     maintainers: jayakody tedelhourani vuile
     labels: networking
@@ -891,13 +901,13 @@ files:
     maintainers: QijunPan
     labels: networking
   $module_utils/cnos_devicerules.py:
-    maintainers: dkasberg
+    maintainers: dkasberg amuraleedhar
     labels: networking
   $module_utils/cnos_errorcodes.py:
-    maintainers: dkasberg
+    maintainers: dkasberg amuraleedhar
     labels: networking
   $module_utils/cnos.py:
-    maintainers: dkasberg
+    maintainers: dkasberg amuraleedhar
     labels: networking
   $module_utils/crypto.py:
     maintainers: Spredzy
@@ -906,7 +916,9 @@ files:
     labels: networking
   $module_utils/f5_utils.py:
     maintainers: caphrim007
-    labels: networking
+    labels:
+    - f5
+    - networking
   $module_utils/fortios.py:
     maintainers: bjolivot
     labels: networking
@@ -950,6 +962,8 @@ files:
   $module_utils/ordnance.py:
     maintainers: alexanderturner djh00t
     labels: networking
+  $module_utils/ovirt.py:
+    maintainers: joshainglis karmab machacekondra
   $module_utils/sros.py:
     maintainers: $team_networking
     labels: networking
@@ -1075,14 +1089,11 @@ files:
   test/sanity/validate-modules:
     keywords:
     - validate-modules
-  test/:
-    labels: test_pull_requests
   docs/:
     labels: docs_pull_request
-    notify:
-      - dharmabumstead
+    notify: dharmabumstead
   packaging/:
-    labels: packaging
+    # 'packaging' automatically added to labels
 macros:
   module_utils: lib/ansible/module_utils
   modules: lib/ansible/modules
@@ -1090,7 +1101,7 @@ macros:
   team_avi: ericsysmin grastogi23 khaltore
   team_azure: haroldwongms nitzmahone tstringer
   team_cumulus: isharacomix jrrivers privateip
-  team_manageiq: gtanzillo abellotti zgalor yaacov
+  team_manageiq: gtanzillo abellotti zgalor yaacov cben
   team_netapp: hulquest lmprice broncofan
   team_netscaler: chiradeep giorgos-nikolopoulos
   team_netvisor: Qalthos amitsi gundalow privateip
@@ -1098,8 +1109,9 @@ macros:
   team_nxos: GGabriele jedelman8 mikewiebe privateip rahushen rcarrillocruz trishnaguha
   team_openstack: emonty j2sol juliakreger rcarrillocruz shrews thingee dagnello
   team_openswitch: Qalthos gundalow privateip
-  team_rabbitmq: chrishoffman manuel-sousa romanek-adam hyperized
+  team_rabbitmq: chrishoffman manuel-sousa hyperized
   team_rhn: alikins barnabycourt flossware vritant
   team_tower: ghjm jlaska matburt wwitzel3
   team_vmware: dagwieers dav1x jctanner nerzhul akasurde
   team_windows: dagwieers jborean93 jhawkesworth
+  team_ipa: Nosmoht akasurde

.github/ISSUE_TEMPLATE.md

@@ -13,21 +13,24 @@ Also test if the latest release, and master branch are affected too.
 <!--- Name of the module/plugin/task/feature -->
 
 ##### ANSIBLE VERSION
-<!--- Paste verbatim output from “ansible --version” between quotes below -->
+<!--- Paste verbatim output from "ansible --version" between quotes below -->
 ```
 
 ```
 
 ##### CONFIGURATION
 <!---
-Mention any settings you have changed/added/removed in ansible.cfg
+If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"
+
+Otherwise, mention any settings you have changed/added/removed in ansible.cfg
 (or using the ANSIBLE_* environment variables).
+
 -->
 
 ##### OS / ENVIRONMENT
 <!---
 Mention the OS you are running Ansible from, and the OS you are
-managing, or say “N/A” for anything that is not platform-specific.
+managing, or say "N/A" for anything that is not platform-specific.
 Also mention the specific version of what you are trying to control,
 e.g. if this is a network bug the version of firmware on the network device.
 -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -18,7 +18,7 @@ the change does.
 <!--- Name of the module/plugin/module/task -->
 
 ##### ANSIBLE VERSION
-<!--- Paste verbatim output from “ansible --version” between quotes below -->
+<!--- Paste verbatim output from "ansible --version" between quotes below -->
 ```
 
 ```

.gitignore

@@ -27,25 +27,30 @@ docs/man/man3/*
 *.sublime-project
 *.sublime-workspace
 # docsite stuff...
+docs/api/_build/
+docs/api/rst/
+docs/docsite/*.html
+docs/docsite/_build
+docs/docsite/_static/*.gif
+docs/docsite/_static/*.png
+docs/docsite/_static/websupport.js
+docs/docsite/htmlout
+docs/docsite/searchindex.js
+docs/docsite/rst_warnings
+docs/docsite/rst/*_module.rst
+docs/docsite/rst/ansible.rst
+docs/docsite/rst/ansible-*.rst
 docs/docsite/rst/community_maintained.rst
 docs/docsite/rst/config.rst
 docs/docsite/rst/core_maintained.rst
 docs/docsite/rst/list_of_*.rst
-docs/docsite/rst/*_module.rst
 docs/docsite/rst/modules_by_category.rst
 docs/docsite/rst/network_maintained.rst
+docs/docsite/rst/plugins_by_category.rst
 docs/docsite/rst/partner_maintained.rst
 docs/docsite/rst/playbook_keywords.rst
 docs/docsite/rst/playbooks_directives.rst
-docs/docsite/*.html
-docs/docsite/_static/*.gif
-docs/docsite/_static/*.png
-docs/docsite/_static/websupport.js
-docs/docsite/searchindex.js
-docs/docsite/htmlout
-docs/docsite/_build
-docs/api/rst/
-docs/api/_build/
+docs/docsite/rst/plugins/*/*.rst
 # deb building stuff...
 /debian/
 deb-build
@@ -69,7 +74,8 @@ Vagrantfile
 .vagrant
 ansible.egg-info/
 /shippable/
-/test/integration/cloud-config-azure.yml
+/test/integration/cloud-config-*.*
+!/test/integration/cloud-config-*.*.template
 # Release directory
 packaging/release/ansible_release
 /.cache/

CHANGELOG.md

@@ -1,7 +1,35 @@
 Ansible Changes By Release
 ==========================
 
-## 2.4 "Dancing Days" - ACTIVE DEVELOPMENT
+<a id="2.5"></a>
+
+## 2.5 "TBD" - ACTIVE DEVELOPMENT
+
+### Major Changes
+* Removed the previously deprecated 'accelerate' mode and all associated keywords and code.
+
+### Deprecations
+
+#### Deprecated Modules (to be removed in 2.9):
+
+#### Removed Modules (previously deprecated):
+* accelerate
+
+### Minor Changes
+* added a few new magic vars corresponding to configuration/command line options:
+  `ansible_diff_mode`, `ansible_inventory_sources`, `ansible_limit`, `ansible_run_tags` , `ansible_forks` and `ansible_skip_tags`
+* Updated the bundled copy of the six library to 1.11.0
+
+### New Modules
+
+#### Windows
+
+  * win_scheduled_task_stat
+
+
+<a id="2.4"></a>
+
+## 2.4 "Dancing Days" - 2017/09/18
 
 ### Major Changes
 
@@ -9,8 +37,6 @@ Ansible Changes By Release
 * New import/include keywords to replace the old bare `include` directives. The use of `static: {yes|no}` on such includes is now deprecated.
     - Using `import_*` (`import_playbook`, `import_tasks`, `import_role`) directives are static.
     - Using `include_*` (`include_tasks`, `include_role`) directives are dynamic.
-* Added fact namespacing, from now on facts will be available under `ansible_facts` namespace (i.e. `ansible_facts.ansible_os_distribution`).
-  They will continue to be added into the main namespace directly, but now a configuration toggle to disable this, currently off by default, in the future it will be on by default.
   This is done to avoid collisions and possible security issues as facts come from the remote targets and they might be compromised.
 * New `order` play level keyword that allows the user to change the order in which Ansible processes hosts when dispatching tasks.
 * Users can now set group merge priority for groups of the same depth (parent child relationship), using the new `ansible_group_priority` variable, when values are the same or don't exist it will fallback to the previous sorting by name'.
@@ -22,14 +48,14 @@ Ansible Changes By Release
   - New inventory plugins for creating inventory
   - Old inventory formats are still supported via plugins
   - Inline host_list is also an inventory plugin, an example alternative `advanced_host_list` is also provided (it supports ranges)
-  - New configuration option to list enabled plugins and precedence order: `whitelist_inventory` in ansible.cfg
+  - New configuration option to list enabled plugins and precedence order `[inventory]enable_plugins` in ansible.cfg
   - vars_plugins have been reworked, they are now run from Vars manager and API has changed (need docs)
   - Loading group_vars/host_vars is now a vars plugin and can be overridden
   - It is now possible to specify mulitple inventory sources in the command line (-i /etc/hosts1 -i /opt/hosts2)
   - Inventory plugins can use the cache plugin (i.e. virtualbox) and is affected by `meta: refresh_inventory`
   - Group variable precedence is now configurable via new 'precedence' option in ansible.cfg (needs docs)
   - Improved warnings and error messages across the board
-* Configuration has been changed from a hardcoded into the constants module to dynamically loaded from yaml definitions
+* Configuration has been changed from a hardcoded listing in the constants module to dynamically loaded from yaml definitions
   - Also added an ansible-config CLI to allow for listing config options and dumping current config (including origin)
   - TODO: build upon this to add many features detailed in ansible-config proposal https://github.com/ansible/proposals/issues/35
 * Windows modules now support the use of multiple shared module_utils files in the form of Powershell modules (.psm1), via `#Requires -Module Ansible.ModuleUtils.Whatever.psm1`
@@ -49,16 +75,18 @@ Ansible Changes By Release
   moved to `ansible.utils.unsafe_proxy` to avoid a circular import.
 * The win_get_url module has the dictionary 'win_get_url' in its results deprecated,
   its content is now also available directly in the resulting output, like other modules.
+* previouslly deprecated 'hostfile' config settings have been 're-deprecated' as before the code did not warn about deprecated configuration settings, but it does now.
 
 #### Deprecated Modules (to be removed in 2.8):
-* ec2_facts: replaced by ec2_metadata_facts
+* azure: use M(azure_rm_virtualmachine) instead
 * cs_nic: replaced by cs_instance_nic_secondaryip, also see new module cs_instance_nic for managing nics
+* ec2_facts: replaced by ec2_metadata_facts
+* ec2_remote_facts: replaced by ec2_instance_facts
 * panos_address: use M(panos_object) instead
-* panos_service: use M(panos_object) instead
-* panos_security_policy: use M(panos_security_rule) instead
 * panos_nat_policy: use M(panos_nat_rule) instead
+* panos_security_policy: use M(panos_security_rule) instead
+* panos_service: use M(panos_object) instead
 * s3: replaced by aws_s3
-* ec2_remote_facts: replaced by 
 * win_msi: use M(win_package) instead
 
 #### Removed Modules (previouslly deprecated):
@@ -67,12 +95,12 @@ Ansible Changes By Release
 * iosxr_template: use iosxr_config instead
 * junos_template: use junos_config instead
 * nxos_template: use nxos_config instead
-* ops_template: use ops_config instead
 * openswitch
+* ops_template: use ops_config instead
 
 
 ### Minor Changes
-* Removed previously deprecated config option `hostfile` and env var `ANSIBLE_HOSTS`
+* Now deprecated configuration options issue warnings when set.
 * Removed unused and deprecated config option `pattern`
 * Updated the copy of six bundled for modules to use from 1.4.1 to 1.10.0
 * The `include_dir` var is not a global anymore, as we now allow multiple inventory sources, it is now host dependant.
@@ -130,6 +158,11 @@ Ansible Changes By Release
     - extensions of files to ignore when using inventory directory
 	- patterns of flies to ignore when using inventory directory
 	- option to toggle failed inventory source parsing between an error or a warning
+* More fixes for Python 3 across the code base.
+* win_shell and win_command modules now properly preserve quoted arguments passed on the command-line. Tasks that attempted to work around the issue by adding extra quotes/escaping may need to be reworked. See https://github.com/ansible/ansible/issues/23019 for additional detail.
+* All configuration paths are now relative to the `ansible.cfg` file used.
+* By user request, a 'configuration macro' (``CWD``) is available to force configured paths to be relative to the current working directory. Please note that this is unsafe and not recommended.
+
 
 #### New Callbacks:
 - full_skip
@@ -578,6 +611,141 @@ Ansible Changes By Release
   * win_wait_for
   * win_wakeonlan
 
+<a id="2.3.3"></a>
+
+## 2.3.3 "Ramble On" - TBD
+
+### Bugfixes
+* Fix alternatives module handlling of non existing options
+* Fix synchronize traceback with the docker connection plugin
+* Do not escape backslashes in the template lookup plugin to mirror what the template module does
+* Fix the expires option of the postgresq_user module
+* Fix for win_acl when settings permissions on registry objects that use `ALL APPLICATION PACKAGES` and `ALL RESTRICTED APPLICATION PACKAGES`
+* Python3 fixes
+  * asorted azure modules
+  * pause module
+  * hacking/env-setup script
+  * Fix traceback when checking for passwords in logged strings when logging executed commands.
+  * docker_login module
+  * Workaround python-libselinux API change in the seboolean module
+  * digital_ocean_tag module
+  * Fix the zip filter
+  * Fix user module combining bytes and text
+  * Fix for security groups in the amazon efs module
+  * Fix for the jail connection plugin not finding the named jail
+  * Fix for blockinfile's parameters insertbefore and insertafter
+* ios_config: Fix traceback when the efaults parameter is not set
+* iosxr_config: Fixed unicode error when UTF-8 characters are in configs
+* Fix check mode in archive module
+* Fix UnboundLocalError in check mode in cs_role module
+* Fix to always use lowercase hostnames for host keys in known_hosts module
+* Added missing return results for win_stat
+* Fix rabbitmq modules to give a helpful error if requests is not installed
+* Fix yum module not deleting rpms that it downloaded
+* Fix yum module failing with a URL to an rpm
+* Fix file module inappropriately expanding literal dollar signs in a path read
+  from the filesystem as an environment variable.
+* Fix the ssh "smart" transport setting which automatically selects the best means of
+  transferring files over ssh (sftp, ssh, piped).
+* Fix authentication by api_key parameter in exoscale modules.
+* vmware module_utils shared code ssl/validate_certs fixes in connection logic
+* allow 'bridge' facts to work for certain containers that create conflicting ones with connection plugins
+* Fix for win_get_url to use TLS 1.2/1.1 if it is available on the host
+* Fix for the filetree lookup with non-ascii group names
+* Better message for invalid keywords/options in task due to undefined expressions
+* Fixed check mode for enable on Solaris for service module
+* Fix cloudtrail module to allow AWS profiles other than the default
+* Fix an encoding issue with secret (password) vars_prompts
+
+<a id="2.3.2"></a>
+
+## 2.3.2 "Ramble On" - 2017-08-04
+
+### Bugfixes
+* Fix partend i18n issues
+* fixed handling of extra vars for tower_job_template (#25272)
+* Python3 bugfixes
+  * Fix sorting of ec2 policies
+  * Fix digital_ocean dynamic inventory script
+  * Fix for the docker connection plugin
+  * Fix pip module when using python3's pyvenv and python3 -m venv to create virtualenvs
+* Fix for the AnsiBallZ wrapper so that it gives a better error message when
+  there's not enough disk space to create its tempdir.
+* Fix so ansilbe-galaxy install --force with unversioned roles will once again
+  overwrite old versions.
+* Fix for RabbitMQ 3.6.7 endpoint return code changing.
+* Fix for Foreman organization creation
+* fixed incorrect fail_json ref in rpm_key
+* Corrected requried on hash_name for dynamodb_table
+* Fix for fetch action plugin not validating correctly
+* Avoid vault view writing display to logs
+* htpasswd: fix passlib module version comparison
+* Fix for flowdock error message when external_user_name is missing
+* fixed corner case for delegate_to, loops and delegate_facts
+* fixed wait_for python2.4/2.5 compatibility (this is last version this is needed)
+* fix for adhoc not obeying callback options
+* fix for win_find where it fails to recursively scan empty nested directories
+* fix non-pipelined code paths for Windows (eg, ANSIBLE_KEEP_REMOTE_FILES, non-pipelined connection plugins)
+* fix for win_updates where args and check mode were ignored due to common code change
+* fix for unprivileged users to Windows runas become method
+* fix starttls code path for mail module
+* fix missing LC_TYPE in parted module
+* fix CN parsing with OpenSSL 1.1 in letsencrypt module
+* fix params assignment in jabber module
+* fix TXT record type handling in exo_dns_record module
+* fix message queue message ttl can't be 0 in rabbitmq_queue module
+* CloudStack bugfixes:
+  * fix template upload for users in cs_template module, change default to is_routing=None
+  * several fixes in cs_host module fixes hypervisor handling
+  * fix network param ignored due typo in cs_nic module
+  * fix missing type bool in module cs_zone
+  * fix KeyError: 'sshkeypair' in cs_instance module for CloudStack v4.5 and before
+* fix for win_chocolatey where trying to upgrade all the packages as per the example docs fails
+* fix for win_chocolatey where it did not fail if the version set did not exist
+* fix for win_regedit always changing a reg key if the dword values set is a hex
+* fix for wait_for on non-Linux systems with newer versions of psutil
+* fix eos_banner code and test issues
+* run tearup and teardown of EAPI service only on EAPI tests
+* fix eos_config tests so only Eth1 and Eth2 are used
+* Fix for potential bug when using legacy inventory vars for configuring the su password.
+* Fix crash in file module when directories contain non-utf8 filenames
+* Fix for dnf groupinstall with dnf-2.x
+* Fix seboolean module for incompatibility in newer Python3 selinux bindings
+* Optimization for inventory, no need to dedup at every stage, its redundant and slow
+* Fix fact gathering for package and service action plugins
+* make random_choice more error resilient (#27380)
+* ensure prefix in plugin loading to avoid conflicts
+* fix for a small number of modules (tempfile, possibly copy) which could fail
+  if the tempdir on the remote box was a symlink
++ fix non-pipelined code paths for Windows (eg, ANSIBLE_KEEP_REMOTE_FILES, non-pipelined connection plugins)
+* fix for win_updates where args and check mode were ignored due to common code change
+
+<a id="2.3.1"></a>
+
+## 2.3.1 "Ramble On" - 2017-06-01
+
+
+### Bugfixes
+* Security fix for CVE-2017-7481 - data for lookup plugins used as variables was not being correctly marked as "unsafe".
+* Fix default value of fetch module's validate_checksum to be True
+* Added fix for "meta: refresh_connection" not working with default 'smart' connection.
+* Fix template so that the --diff command line option works when the destination is a directory
+* Fix python3 bugs in pam_limits
+* Fix unbound error when using module deprecation as a single string
+* Several places in which error handling was broken due to bad conversions or just typos
+* Fix to user module for appending/setting groups on OpenBSD (flags were reversed)
+* assemble fix to use safer os.join.path, avoids charset issues
+* fixed issue with solaris facts and i18n
+* added python2.4 compatiblity fix to sysctl module
+* Fix comparison of exisiting container security opts in the docker_container module
+* fixed service module invocation of insserv on certain platforms
+* Fix traceback in os_user in an error case.
+* Fix docker container to restart a container when changing to fewer exposed ports
+* Fix tracebacks in docker_network
+* Fixes to detection of updated docker images
+* Handle detection of docker image changes when published ports is changed
+* Fix for docker_container restarting images when links list is empty.
+
 <a id="2.3"></a>
 
 ## 2.3 "Ramble On" - 2017-04-12
@@ -613,6 +781,7 @@ Moving to Ansible 2.3 guide http://docs.ansible.com/ansible/porting_guide_2.3.ht
 * 'service' tasks can now use async again, we had lost this capability when changed into an action plugin.
 * made any_errors_fatal inheritable from play to task and all other objects in between.
 * many small performance improvements in inventory and variable handling and in task execution.
+* Added a retry class to the ec2_asg module since customers were running into throttling errors (AWSRetry is a solution for modules using boto3 which isn't applicable here).
 
 ### Deprecations
 * Specifying --tags (or --skip-tags) multiple times on the command line
@@ -844,6 +1013,14 @@ Moving to Ansible 2.3 guide http://docs.ansible.com/ansible/porting_guide_2.3.ht
   * panos_service
 - postgresql_schema
 - proxmox_kvm
+- proxysql:
+  * proxysql_backend_servers
+  * proxysql_global_variables
+  * proxysql_manage_config
+  * proxysql_mysql_users
+  * proxysql_query_rules
+  * proxysql_replication_hostgroups
+  * proxysql_scheduler
 - pubnub_blocks
 - pulp_repo
 - runit
@@ -1640,7 +1817,7 @@ Module fixes:
   This re-executes inventory scripts, but does not force them to ignore any cache they might use.
 * New delegate_facts directive, a boolean that allows you to apply facts to the delegated host (true/yes) instead of the inventory_hostname (no/false) which is the default and previous behaviour.
 * local connections now work with 'su' as a privilege escalation method
-* Ansible 2.0 has deprecated the “ssh” from ansible_ssh_user, ansible_ssh_host, and ansible_ssh_port to become ansible_user, ansible_host, and ansible_port.
+* Ansible 2.0 has deprecated the "ssh" from ansible_ssh_user, ansible_ssh_host, and ansible_ssh_port to become ansible_user, ansible_host, and ansible_port.
 * New ssh configuration variables (`ansible_ssh_common_args`, `ansible_ssh_extra_args`) can be used to configure a
   per-group or per-host ssh ProxyCommand or set any other ssh options.
   `ansible_ssh_extra_args` is used to set options that are accepted only by ssh (not sftp or scp, which have their own analogous settings).

MODULE_GUIDELINES.md

@@ -47,9 +47,9 @@ A detailed explanation of the PR workflow can be seen here: https://github.com/a
 
 ***
 
-# Extras maintainers list
+# Maintainers list
 
-The full list of maintainers for modules is located here: https://github.com/ansible/ansibullbot/blob/master/MAINTAINERS.txt
+The full list of maintainers for modules is located here: https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml
 
 ## Changing Maintainership
 

Makefile

@@ -29,6 +29,7 @@ ASCII2HTMLMAN = a2x -L -D docs/html/man/ -d manpage -f xhtml
 else
 ASCII2MAN = @echo "ERROR: AsciiDoc 'a2x' command is not installed but is required to build $(MANPAGES)" && exit 1
 endif
+GENERATE_CLI = docs/bin/generate_man.py
 
 PYTHON=python
 SITELIB = $(shell $(PYTHON) -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")
@@ -185,7 +186,7 @@ clean:
 	rm -rf logs/
 	rm -rf .cache/
 	rm -f test/units/.coverage*
-	rm -f test/results/*/*
+	rm -rf test/results/*/*
 	find test/ -type f -name '*.retry' -delete
 	@echo "Cleaning up RPM building stuff"
 	rm -rf MANIFEST rpm-build
@@ -343,8 +344,9 @@ webdocs:
 
 .PHONY: generate_asciidoc
 generate_asciidoc: lib/ansible/cli/*.py
-	mkdir -p ./docs/man/man1/
-	PYTHONPATH=./lib ./docs/bin/generate_man.py
+	mkdir -p ./docs/man/man1/ ; \
+	PYTHONPATH=./lib $(GENERATE_CLI) --template-file=docs/templates/man.j2 --output-dir=docs/man/man1/ --output-format man lib/ansible/cli/*.py
+
 
 docs: generate_asciidoc
 	make $(MANPAGES)

README.md

@@ -9,9 +9,7 @@ Ansible is a radically simple IT automation system.  It handles configuration-ma
 
 Read the documentation and more at https://ansible.com/
 
-Many users run straight from the development branch (it's generally fine to do so), but you might also wish to consume a release.
-
-You can find instructions [here](https://docs.ansible.com/ansible/intro_installation.html) for a variety of platforms.
+You can find installation instructions [here](https://docs.ansible.com/ansible/intro_installation.html) for a variety of platforms. Most users should probably install a released version of Ansible from `pip`, a package manager or our [release repository](https://releases.ansible.com/ansible/). [Officially supported](https://www.ansible.com/ansible-engine) builds of Ansible are also available. Some power users run directly from the development branch - while significant efforts are made to ensure that `devel` is reasonably stable, you're more likely to encounter breaking changes when running Ansible this way.
 
 Design Principles
 =================
@@ -41,7 +39,6 @@ Branch Info
 
    * Releases are named after Led Zeppelin songs. (Releases prior to 2.0 were named after Van Halen songs.)
    * The devel branch corresponds to the release actively under development.
-   * For releases 1.8 - 2.2, modules are kept in different repos, you'll want to follow [core](https://github.com/ansible/ansible-modules-core) and [extras](https://github.com/ansible/ansible-modules-extras)
    * Various release-X.Y branches exist for previous releases.
    * We'd love to have your contributions, read [Community Information](https://docs.ansible.com/community.html) for notes on how to get started.
 
@@ -52,9 +49,8 @@ Ansible was created by [Michael DeHaan](https://github.com/mpdehaan) (michael.de
 
 Ansible is sponsored by [Ansible, Inc](https://ansible.com)
 
-Licence
+License
 =======
-GNU
-Click on the [Link](COPYING) to see the full text.
-
+GNU General Public License v3.0
 
+See [COPYING](COPYING) to see the full text.

RELEASES.txt

@@ -5,79 +5,79 @@ VERSION  RELEASE     CODE NAME
 ++++++++++++++++++++++++++++++
 
 2.4.0    TBD         "Dancing Days"
-2.3.2    08-04-2017  "Ramble On"
-2.3.1    06-01-2017  "Ramble On"
-2.3.0    04-12-2017  "Ramble On"
-2.2.3    05-09-2017  "The Battle of Evermore"
-2.2.2    03-27-2017  "The Battle of Evermore"
-2.2.1    01-16-2017  "The Battle of Evermore"
-2.2.0    11-01-2016  "The Battle of Evermore"
-2.1.6    06-01-2017  "The Song Remains the Same"
-2.1.5    03-27-2017  "The Song Remains the Same"
-2.1.4    01-16-2017  "The Song Remains the Same"
-2.1.3    11-04-2016  "The Song Remains the Same"
-2.1.2    09-29-2016  "The Song Remains the Same"
-2.1.1    07-28-2016  "The Song Remains the Same"
-2.1.0    05-25-2016  "The Song Remains the Same"
-2.0.2    04-19-2016  "Over the Hills and Far Away"
-2.0.1    02-24-2016  "Over the Hills and Far Away"
-2.0.0    01-12-2016  "Over the Hills and Far Away"
-1.9.6    04-15-2016  "Dancing In the Streets"
-1.9.5    03-21-2016  "Dancing In the Streets"
-1.9.4    10-09-2015  "Dancing In the Streets"
-1.9.3    09-03-2015  "Dancing In the Streets"
-1.9.2    06-24-2015  "Dancing In the Streets"
-1.9.1    04-27-2015  "Dancing In the Streets"
-1.9.0    03-25-2015  "Dancing In the Streets"
-1.8.4    02-19-2015  "You Really Got Me"
-1.8.3    02-17-2015  "You Really Got Me"
-1.8.2    12-04-2014  "You Really Got Me"
-1.8.1    11-26-2014  "You Really Got Me"
-1.8.0    11-25-2014  "You Really Got Me"
-1.7.2    09-24-2014  "Summer Nights"
-1.7.1    08-14-2014  "Summer Nights"
-1.7.0    08-06-2014  "Summer Nights"
-1.6.10   07-25-2014  "The Cradle Will Rock"
-1.6.9    07-24-2014  "The Cradle Will Rock"
-1.6.8    07-22-2014  "The Cradle Will Rock"
-1.6.7    07-21-2014  "The Cradle Will Rock"
-1.6.6    07-01-2014  "The Cradle Will Rock"
-1.6.5    06-25-2014  "The Cradle Will Rock"
-1.6.4    06-25-2014  "The Cradle Will Rock"
-1.6.3    06-09-2014  "The Cradle Will Rock"
-1.6.2    05-23-2014  "The Cradle Will Rock"
-1.6.1    05-07-2014  "The Cradle Will Rock"
-1.6.0    05-05-2014  "The Cradle Will Rock"
-1.5.5    04-18-2014  "Love Walks In"
-1.5.4    04-01-2014  "Love Walks In"
-1.5.3    03-13-2014  "Love Walks In"
-1.5.2    03-11-2014  "Love Walks In"
-1.5.1    03-10-2014  "Love Walks In"
-1.5.0    02-28-2014  "Love Walks In"
-1.4.5    02-12-2014  "Could This Be Magic?"
-1.4.4    01-06-2014  "Could This Be Magic?"
-1.4.3    12-20-2013  "Could This Be Magic?"
-1.4.2    12-18-2013  "Could This Be Magic?"
-1.4.1    11-27-2013  "Could This Be Magic?"
-1.4.0    11-21-2013  "Could This Be Magic?"
-1.3.4    10-29-2013  "Top of the World"
-1.3.3    10-09-2013  "Top of the World"
-1.3.2    09-19-2013  "Top of the World"
-1.3.1    09-16-2013  "Top of the World"
-1.3.0    09-13-2013  "Top of the World"
-1.2.3    08-21-2013  "Right Now"
-1.2.2    07-05-2013  "Right Now"
-1.2.1    07-04-2013  "Right Now"
-1.2.0    06-10-2013  "Right Now"
-1.1.0    04-02-2013  "Mean Street"
-1.0.0    02-01-2013  "Eruption"
-0.9.0    11-30-2012  "Dreams"
-0.8.0    10-19-2012  "Cathedral"
-0.7.0    09-06-2012  "Panama"
-0.6.0    08-06-2012  "Cabo"
-0.5.0    07-04-2012  "Amsterdam"
-0.4.0    05-23-2012  "Unchained"
-0.3.0    04-23-2012  "Baluchitherium"
+2.3.2    2017-08-04  "Ramble On"
+2.3.1    2017-06-01  "Ramble On"
+2.3.0    2017-04-12  "Ramble On"
+2.2.3    2017-05-09  "The Battle of Evermore"
+2.2.2    2017-03-27  "The Battle of Evermore"
+2.2.1    2017-01-16  "The Battle of Evermore"
+2.2.0    2016-11-01  "The Battle of Evermore"
+2.1.6    2017-06-01  "The Song Remains the Same"
+2.1.5    2017-03-27  "The Song Remains the Same"
+2.1.4    2017-01-16  "The Song Remains the Same"
+2.1.3    2016-11-04  "The Song Remains the Same"
+2.1.2    2016-09-29  "The Song Remains the Same"
+2.1.1    2016-07-28  "The Song Remains the Same"
+2.1.0    2016-05-25  "The Song Remains the Same"
+2.0.2    2016-04-19  "Over the Hills and Far Away"
+2.0.1    2016-02-24  "Over the Hills and Far Away"
+2.0.0    2016-01-12  "Over the Hills and Far Away"
+1.9.6    2016-04-15  "Dancing In the Streets"
+1.9.5    2016-03-21  "Dancing In the Streets"
+1.9.4    2015-10-09  "Dancing In the Streets"
+1.9.3    2015-09-03  "Dancing In the Streets"
+1.9.2    2015-06-24  "Dancing In the Streets"
+1.9.1    2015-04-27  "Dancing In the Streets"
+1.9.0    2015-03-25  "Dancing In the Streets"
+1.8.4    2015-02-19  "You Really Got Me"
+1.8.3    2015-02-17  "You Really Got Me"
+1.8.2    2014-12-04  "You Really Got Me"
+1.8.1    2014-11-26  "You Really Got Me"
+1.8.0    2014-11-25  "You Really Got Me"
+1.7.2    2014-09-24  "Summer Nights"
+1.7.1    2014-08-14  "Summer Nights"
+1.7.0    2014-08-06  "Summer Nights"
+1.6.10   2014-07-25  "The Cradle Will Rock"
+1.6.9    2014-07-24  "The Cradle Will Rock"
+1.6.8    2014-07-22  "The Cradle Will Rock"
+1.6.7    2014-07-21  "The Cradle Will Rock"
+1.6.6    2014-07-01  "The Cradle Will Rock"
+1.6.5    2014-06-25  "The Cradle Will Rock"
+1.6.4    2014-06-25  "The Cradle Will Rock"
+1.6.3    2014-06-09  "The Cradle Will Rock"
+1.6.2    2014-05-23  "The Cradle Will Rock"
+1.6.1    2014-05-07  "The Cradle Will Rock"
+1.6.0    2014-05-05  "The Cradle Will Rock"
+1.5.5    2014-04-18  "Love Walks In"
+1.5.4    2014-04-01  "Love Walks In"
+1.5.3    2014-03-13  "Love Walks In"
+1.5.2    2014-03-11  "Love Walks In"
+1.5.1    2014-03-10  "Love Walks In"
+1.5.0    2014-02-28  "Love Walks In"
+1.4.5    2014-02-12  "Could This Be Magic?"
+1.4.4    2014-01-06  "Could This Be Magic?"
+1.4.3    2013-12-20  "Could This Be Magic?"
+1.4.2    2013-12-18  "Could This Be Magic?"
+1.4.1    2013-11-27  "Could This Be Magic?"
+1.4.0    2013-11-21  "Could This Be Magic?"
+1.3.4    2013-10-29  "Top of the World"
+1.3.3    2013-10-09  "Top of the World"
+1.3.2    2013-09-19  "Top of the World"
+1.3.1    2013-09-16  "Top of the World"
+1.3.0    2013-09-13  "Top of the World"
+1.2.3    2013-08-21  "Right Now"
+1.2.2    2013-07-05  "Right Now"
+1.2.1    2013-07-04  "Right Now"
+1.2.0    2013-06-10  "Right Now"
+1.1.0    2013-04-02  "Mean Street"
+1.0.0    2013-02-01  "Eruption"
+0.9.0    2012-11-30  "Dreams"
+0.8.0    2012-10-19  "Cathedral"
+0.7.0    2012-09-06  "Panama"
+0.6.0    2012-08-06  "Cabo"
+0.5.0    2012-07-04  "Amsterdam"
+0.4.0    2012-05-23  "Unchained"
+0.3.0    2012-04-23  "Baluchitherium"
 0.2.0    ?           "Untitled"
 0.1.0    ?           "Untitled"
 0.0.2    ?           "Untitled"

VERSION

@@ -1 +1 @@
-2.4.0 0.0.devel
+2.5.0 0.0.devel

bin/ansible

@@ -37,27 +37,30 @@ import shutil
 import sys
 import traceback
 
-import ansible.constants as C
 from ansible.errors import AnsibleError, AnsibleOptionsError, AnsibleParserError
-from ansible.utils.display import Display
 from ansible.module_utils._text import to_text
 
 
-########################################
-# OUTPUT OF LAST RESORT
 class LastResort(object):
-    def display(self, msg):
+    # OUTPUT OF LAST RESORT
+    def display(self, msg, log_only=None):
         print(msg, file=sys.stderr)
 
     def error(self, msg, wrap_text=None):
         print(msg, file=sys.stderr)
 
 
-########################################
-
 if __name__ == '__main__':
 
     display = LastResort()
+
+    try:  # bad ANSIBLE_CONFIG or config options can force ugly stacktrace
+        import ansible.constants as C
+        from ansible.utils.display import Display
+    except AnsibleOptionsError as e:
+        display.error(to_text(e), wrap_text=False)
+        sys.exit(5)
+
     cli = None
     me = os.path.basename(sys.argv[0])
 

contrib/inventory/apstra_aos.py

@@ -466,35 +466,30 @@ class AosInventory(object):
         except:
             if 'AOS_SERVER' in os.environ.keys():
                 self.aos_server = os.environ['AOS_SERVER']
-            pass
 
         try:
             self.aos_server_port = config.get('aos', 'port')
         except:
             if 'AOS_PORT' in os.environ.keys():
                 self.aos_server_port = os.environ['AOS_PORT']
-            pass
 
         try:
             self.aos_username = config.get('aos', 'username')
         except:
             if 'AOS_USERNAME' in os.environ.keys():
                 self.aos_username = os.environ['AOS_USERNAME']
-            pass
 
         try:
             self.aos_password = config.get('aos', 'password')
         except:
             if 'AOS_PASSWORD' in os.environ.keys():
                 self.aos_password = os.environ['AOS_PASSWORD']
-            pass
 
         try:
             self.aos_blueprint = config.get('aos', 'blueprint')
         except:
             if 'AOS_BLUEPRINT' in os.environ.keys():
                 self.aos_blueprint = os.environ['AOS_BLUEPRINT']
-            pass
 
         try:
             if config.get('aos', 'blueprint_interface') in ['false', 'no']:

contrib/inventory/collins.py

@@ -169,8 +169,9 @@ class CollinsInventory(object):
         print(data_to_print)
         return successful
 
-    def find_assets(self, attributes={}, operation='AND'):
+    def find_assets(self, attributes=None, operation='AND'):
         """ Obtains Collins assets matching the provided attributes. """
+        attributes = {} if attributes is None else attributes
 
         # Formats asset search query to locate assets matching attributes, using
         # the CQL search feature as described here:
@@ -211,7 +212,7 @@ class CollinsInventory(object):
                 cur_page += 1
                 num_retries = 0
             except:
-                self.log.error("Error while communicating with Collins, retrying:\n%s" % traceback.format_exc())
+                self.log.error("Error while communicating with Collins, retrying:\n%s", traceback.format_exc())
                 num_retries += 1
         return assets
 
@@ -280,7 +281,7 @@ class CollinsInventory(object):
         try:
             server_assets = self.find_assets()
         except:
-            self.log.error("Error while locating assets from Collins:\n%s" % traceback.format_exc())
+            self.log.error("Error while locating assets from Collins:\n%s", traceback.format_exc())
             return False
 
         for asset in server_assets:
@@ -304,7 +305,7 @@ class CollinsInventory(object):
             if self.prefer_hostnames and self._asset_has_attribute(asset, 'HOSTNAME'):
                 asset_identifier = self._asset_get_attribute(asset, 'HOSTNAME')
             elif 'ADDRESSES' not in asset:
-                self.log.warning("No IP addresses found for asset '%s', skipping" % asset)
+                self.log.warning("No IP addresses found for asset '%s', skipping", asset)
                 continue
             elif len(asset['ADDRESSES']) < ip_index + 1:
                 self.log.warning(

contrib/inventory/docker.py

@@ -371,7 +371,6 @@ HAS_DOCKER_PY = True
 HAS_DOCKER_ERROR = False
 
 try:
-    from docker import Client
     from docker.errors import APIError, TLSParameterError
     from docker.tls import TLSConfig
     from docker.constants import DEFAULT_TIMEOUT_SECONDS, DEFAULT_DOCKER_API_VERSION
@@ -379,6 +378,19 @@ except ImportError as exc:
     HAS_DOCKER_ERROR = str(exc)
     HAS_DOCKER_PY = False
 
+# Client has recently been split into DockerClient and APIClient
+try:
+    from docker import Client
+except ImportError as exc:
+    try:
+        from docker import APIClient as Client
+    except ImportError as exc:
+        HAS_DOCKER_ERROR = str(exc)
+        HAS_DOCKER_PY = False
+
+        class Client:
+            pass
+
 DEFAULT_DOCKER_HOST = 'unix://var/run/docker.sock'
 DEFAULT_TLS = False
 DEFAULT_TLS_VERIFY = False
@@ -779,6 +791,10 @@ class DockerInventory(object):
         if config_path:
             try:
                 config_file = os.path.abspath(config_path)
+                # default config path is docker.yml in same directory as this script
+                # old behaviour is docker.yml in current directory. Handle both.
+                if not os.path.exists(config_file):
+                    config_file = os.path.abspath(os.path.basename(config_path))
             except:
                 config_file = None
 
@@ -813,7 +829,7 @@ class DockerInventory(object):
         # Parse command line arguments
 
         basename = os.path.splitext(os.path.basename(__file__))[0]
-        default_config = basename + '.yml'
+        default_config = os.path.join(os.path.dirname(__file__), basename + '.yml')
 
         parser = argparse.ArgumentParser(
             description='Return Ansible inventory for one or more Docker hosts.')

contrib/inventory/ec2.ini

@@ -159,7 +159,9 @@ group_by_elasticache_replication_group = True
 # inventory. For the full list of possible filters, please read the EC2 API
 # docs: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html#query-DescribeInstances-filters
 # Filters are key/value pairs separated by '=', to list multiple filters use
-# a list separated by commas. See examples below.
+# a list separated by commas. To "AND" criteria together, use "&". Note that
+# the "AND" is not useful along with stack_filters and so such usage is not allowed.
+# See examples below.
 
 # If you want to apply multiple filters simultaneously, set stack_filters to
 # True. Default behaviour is to combine the results of all filters. Stacking
@@ -181,6 +183,13 @@ stack_filters = False
 # (ex. webservers15, webservers1a, webservers123 etc)
 # instance_filters = tag:Name=webservers1*
 
+# Retrieve only instances of type t1.micro that also have tag env=stage
+# instance_filters = instance-type=t1.micro&tag:env=stage
+
+# Retrieve instances of type t1.micro AND tag env=stage, as well as any instance
+# that are of type m3.large, regardless of env tag
+# instance_filters = instance-type=t1.micro&tag:env=stage,instance-type=m3.large
+
 # An IAM role can be assumed, so all requests are run as that role.
 # This can be useful for connecting across different accounts, or to limit user
 # access

contrib/inventory/ec2.py

@@ -139,7 +139,7 @@ from ansible.module_utils import ec2 as ec2_utils
 
 HAS_BOTO3 = False
 try:
-    import boto3
+    import boto3  # noqa
     HAS_BOTO3 = True
 except ImportError:
     pass
@@ -152,6 +152,60 @@ try:
 except ImportError:
     import simplejson as json
 
+DEFAULTS = {
+    'all_elasticache_clusters': 'False',
+    'all_elasticache_nodes': 'False',
+    'all_elasticache_replication_groups': 'False',
+    'all_instances': 'False',
+    'all_rds_instances': 'False',
+    'aws_access_key_id': None,
+    'aws_secret_access_key': None,
+    'aws_security_token': None,
+    'boto_profile': None,
+    'cache_max_age': '300',
+    'cache_path': '~/.ansible/tmp',
+    'destination_variable': 'public_dns_name',
+    'elasticache': 'True',
+    'eucalyptus': 'False',
+    'eucalyptus_host': None,
+    'expand_csv_tags': 'False',
+    'group_by_ami_id': 'True',
+    'group_by_availability_zone': 'True',
+    'group_by_aws_account': 'False',
+    'group_by_elasticache_cluster': 'True',
+    'group_by_elasticache_engine': 'True',
+    'group_by_elasticache_parameter_group': 'True',
+    'group_by_elasticache_replication_group': 'True',
+    'group_by_instance_id': 'True',
+    'group_by_instance_state': 'False',
+    'group_by_instance_type': 'True',
+    'group_by_key_pair': 'True',
+    'group_by_platform': 'True',
+    'group_by_rds_engine': 'True',
+    'group_by_rds_parameter_group': 'True',
+    'group_by_region': 'True',
+    'group_by_route53_names': 'True',
+    'group_by_security_group': 'True',
+    'group_by_tag_keys': 'True',
+    'group_by_tag_none': 'True',
+    'group_by_vpc_id': 'True',
+    'hostname_variable': None,
+    'iam_role': None,
+    'include_rds_clusters': 'False',
+    'nested_groups': 'False',
+    'pattern_exclude': None,
+    'pattern_include': None,
+    'rds': 'False',
+    'regions': 'all',
+    'regions_exclude': 'us-gov-west-1, cn-north-1',
+    'replace_dash_in_groups': 'True',
+    'route53': 'False',
+    'route53_excluded_zones': '',
+    'route53_hostnames': None,
+    'stack_filters': 'False',
+    'vpc_destination_variable': 'ip_address'
+}
+
 
 class Ec2Inventory(object):
 
@@ -232,24 +286,32 @@ class Ec2Inventory(object):
         }
 
         if six.PY3:
-            config = configparser.ConfigParser()
+            config = configparser.ConfigParser(DEFAULTS)
         else:
-            config = configparser.SafeConfigParser()
+            config = configparser.SafeConfigParser(DEFAULTS)
         ec2_ini_path = os.environ.get('EC2_INI_PATH', defaults['ec2']['ini_path'])
         ec2_ini_path = os.path.expanduser(os.path.expandvars(ec2_ini_path))
 
         if not os.path.isfile(ec2_ini_path):
             ec2_ini_path = os.path.expanduser(defaults['ec2']['ini_fallback'])
 
-        config.read(ec2_ini_path)
+        if os.path.isfile(ec2_ini_path):
+            config.read(ec2_ini_path)
+
+        # Add empty sections if they don't exist
+        try:
+            config.add_section('ec2')
+        except configparser.DuplicateSectionError:
+            pass
+
+        try:
+            config.add_section('credentials')
+        except configparser.DuplicateSectionError:
+            pass
 
         # is eucalyptus?
-        self.eucalyptus_host = None
-        self.eucalyptus = False
-        if config.has_option('ec2', 'eucalyptus'):
-            self.eucalyptus = config.getboolean('ec2', 'eucalyptus')
-        if self.eucalyptus and config.has_option('ec2', 'eucalyptus_host'):
-            self.eucalyptus_host = config.get('ec2', 'eucalyptus_host')
+        self.eucalyptus = config.getboolean('ec2', 'eucalyptus')
+        self.eucalyptus_host = config.get('ec2', 'eucalyptus_host')
 
         # Regions
         self.regions = []
@@ -259,6 +321,7 @@ class Ec2Inventory(object):
                 self.regions.append(boto.connect_euca(host=self.eucalyptus_host).region.name, **self.credentials)
             else:
                 configRegions_exclude = config.get('ec2', 'regions_exclude')
+
                 for regionInfo in ec2.regions():
                     if regionInfo.name not in configRegions_exclude:
                         self.regions.append(regionInfo.name)
@@ -273,11 +336,7 @@ class Ec2Inventory(object):
         # Destination addresses
         self.destination_variable = config.get('ec2', 'destination_variable')
         self.vpc_destination_variable = config.get('ec2', 'vpc_destination_variable')
-
-        if config.has_option('ec2', 'hostname_variable'):
-            self.hostname_variable = config.get('ec2', 'hostname_variable')
-        else:
-            self.hostname_variable = None
+        self.hostname_variable = config.get('ec2', 'hostname_variable')
 
         if config.has_option('ec2', 'destination_format') and \
            config.has_option('ec2', 'destination_format_tags'):
@@ -289,36 +348,22 @@ class Ec2Inventory(object):
 
         # Route53
         self.route53_enabled = config.getboolean('ec2', 'route53')
-        if config.has_option('ec2', 'route53_hostnames'):
-            self.route53_hostnames = config.get('ec2', 'route53_hostnames')
-        else:
-            self.route53_hostnames = None
+        self.route53_hostnames = config.get('ec2', 'route53_hostnames')
+
         self.route53_excluded_zones = []
-        if config.has_option('ec2', 'route53_excluded_zones'):
-            self.route53_excluded_zones.extend(
-                config.get('ec2', 'route53_excluded_zones', '').split(','))
+        self.route53_excluded_zones = [a for a in config.get('ec2', 'route53_excluded_zones').split(',') if a]
 
         # Include RDS instances?
-        self.rds_enabled = True
-        if config.has_option('ec2', 'rds'):
-            self.rds_enabled = config.getboolean('ec2', 'rds')
+        self.rds_enabled = config.get('ec2', 'rds')
 
         # Include RDS cluster instances?
-        if config.has_option('ec2', 'include_rds_clusters'):
-            self.include_rds_clusters = config.getboolean('ec2', 'include_rds_clusters')
-        else:
-            self.include_rds_clusters = False
+        self.include_rds_clusters = config.getboolean('ec2', 'include_rds_clusters')
 
         # Include ElastiCache instances?
-        self.elasticache_enabled = True
-        if config.has_option('ec2', 'elasticache'):
-            self.elasticache_enabled = config.getboolean('ec2', 'elasticache')
+        self.elasticache_enabled = config.getboolean('ec2', 'elasticache')
 
         # Return all EC2 instances?
-        if config.has_option('ec2', 'all_instances'):
-            self.all_instances = config.getboolean('ec2', 'all_instances')
-        else:
-            self.all_instances = False
+        self.all_instances = config.getboolean('ec2', 'all_instances')
 
         # Instance states to be gathered in inventory. Default is 'running'.
         # Setting 'all_instances' to 'yes' overrides this option.
@@ -343,49 +388,30 @@ class Ec2Inventory(object):
             self.ec2_instance_states = ['running']
 
         # Return all RDS instances? (if RDS is enabled)
-        if config.has_option('ec2', 'all_rds_instances') and self.rds_enabled:
-            self.all_rds_instances = config.getboolean('ec2', 'all_rds_instances')
-        else:
-            self.all_rds_instances = False
+        self.all_rds_instances = config.getboolean('ec2', 'all_rds_instances')
 
         # Return all ElastiCache replication groups? (if ElastiCache is enabled)
-        if config.has_option('ec2', 'all_elasticache_replication_groups') and self.elasticache_enabled:
-            self.all_elasticache_replication_groups = config.getboolean('ec2', 'all_elasticache_replication_groups')
-        else:
-            self.all_elasticache_replication_groups = False
+        self.all_elasticache_replication_groups = config.getboolean('ec2', 'all_elasticache_replication_groups')
 
         # Return all ElastiCache clusters? (if ElastiCache is enabled)
-        if config.has_option('ec2', 'all_elasticache_clusters') and self.elasticache_enabled:
-            self.all_elasticache_clusters = config.getboolean('ec2', 'all_elasticache_clusters')
-        else:
-            self.all_elasticache_clusters = False
+        self.all_elasticache_clusters = config.getboolean('ec2', 'all_elasticache_clusters')
 
         # Return all ElastiCache nodes? (if ElastiCache is enabled)
-        if config.has_option('ec2', 'all_elasticache_nodes') and self.elasticache_enabled:
-            self.all_elasticache_nodes = config.getboolean('ec2', 'all_elasticache_nodes')
-        else:
-            self.all_elasticache_nodes = False
+        self.all_elasticache_nodes = config.getboolean('ec2', 'all_elasticache_nodes')
 
         # boto configuration profile (prefer CLI argument then environment variables then config file)
-        self.boto_profile = self.args.boto_profile or os.environ.get('AWS_PROFILE')
-        if config.has_option('ec2', 'boto_profile') and not self.boto_profile:
-            self.boto_profile = config.get('ec2', 'boto_profile')
+        self.boto_profile = self.args.boto_profile or \
+            os.environ.get('AWS_PROFILE') or \
+            config.get('ec2', 'boto_profile')
 
         # AWS credentials (prefer environment variables)
         if not (self.boto_profile or os.environ.get('AWS_ACCESS_KEY_ID') or
                 os.environ.get('AWS_PROFILE')):
-            if config.has_option('credentials', 'aws_access_key_id'):
-                aws_access_key_id = config.get('credentials', 'aws_access_key_id')
-            else:
-                aws_access_key_id = None
-            if config.has_option('credentials', 'aws_secret_access_key'):
-                aws_secret_access_key = config.get('credentials', 'aws_secret_access_key')
-            else:
-                aws_secret_access_key = None
-            if config.has_option('credentials', 'aws_security_token'):
-                aws_security_token = config.get('credentials', 'aws_security_token')
-            else:
-                aws_security_token = None
+
+            aws_access_key_id = config.get('credentials', 'aws_access_key_id')
+            aws_secret_access_key = config.get('credentials', 'aws_secret_access_key')
+            aws_security_token = config.get('credentials', 'aws_security_token')
+
             if aws_access_key_id:
                 self.credentials = {
                     'aws_access_key_id': aws_access_key_id,
@@ -409,98 +435,59 @@ class Ec2Inventory(object):
         self.cache_path_index = os.path.join(cache_dir, "%s.index" % cache_name)
         self.cache_max_age = config.getint('ec2', 'cache_max_age')
 
-        if config.has_option('ec2', 'expand_csv_tags'):
-            self.expand_csv_tags = config.getboolean('ec2', 'expand_csv_tags')
-        else:
-            self.expand_csv_tags = False
+        self.expand_csv_tags = config.getboolean('ec2', 'expand_csv_tags')
 
         # Configure nested groups instead of flat namespace.
-        if config.has_option('ec2', 'nested_groups'):
-            self.nested_groups = config.getboolean('ec2', 'nested_groups')
-        else:
-            self.nested_groups = False
+        self.nested_groups = config.getboolean('ec2', 'nested_groups')
 
         # Replace dash or not in group names
-        if config.has_option('ec2', 'replace_dash_in_groups'):
-            self.replace_dash_in_groups = config.getboolean('ec2', 'replace_dash_in_groups')
-        else:
-            self.replace_dash_in_groups = True
+        self.replace_dash_in_groups = config.getboolean('ec2', 'replace_dash_in_groups')
 
         # IAM role to assume for connection
-        if config.has_option('ec2', 'iam_role'):
-            self.iam_role = config.get('ec2', 'iam_role')
-        else:
-            self.iam_role = None
+        self.iam_role = config.get('ec2', 'iam_role')
 
         # Configure which groups should be created.
-        group_by_options = [
-            'group_by_instance_id',
-            'group_by_region',
-            'group_by_availability_zone',
-            'group_by_ami_id',
-            'group_by_instance_type',
-            'group_by_instance_state',
-            'group_by_platform',
-            'group_by_key_pair',
-            'group_by_vpc_id',
-            'group_by_security_group',
-            'group_by_tag_keys',
-            'group_by_tag_none',
-            'group_by_route53_names',
-            'group_by_rds_engine',
-            'group_by_rds_parameter_group',
-            'group_by_elasticache_engine',
-            'group_by_elasticache_cluster',
-            'group_by_elasticache_parameter_group',
-            'group_by_elasticache_replication_group',
-            'group_by_aws_account',
-        ]
+
+        group_by_options = [a for a in DEFAULTS if a.startswith('group_by')]
         for option in group_by_options:
-            if config.has_option('ec2', option):
-                setattr(self, option, config.getboolean('ec2', option))
-            else:
-                setattr(self, option, True)
+            setattr(self, option, config.getboolean('ec2', option))
 
         # Do we need to just include hosts that match a pattern?
-        try:
-            pattern_include = config.get('ec2', 'pattern_include')
-            if pattern_include and len(pattern_include) > 0:
-                self.pattern_include = re.compile(pattern_include)
-            else:
-                self.pattern_include = None
-        except configparser.NoOptionError:
-            self.pattern_include = None
+        self.pattern_include = config.get('ec2', 'pattern_include')
+        if self.pattern_include:
+            self.pattern_include = re.compile(self.pattern_include)
 
         # Do we need to exclude hosts that match a pattern?
-        try:
-            pattern_exclude = config.get('ec2', 'pattern_exclude')
-            if pattern_exclude and len(pattern_exclude) > 0:
-                self.pattern_exclude = re.compile(pattern_exclude)
-            else:
-                self.pattern_exclude = None
-        except configparser.NoOptionError:
-            self.pattern_exclude = None
+        self.pattern_exclude = config.get('ec2', 'pattern_exclude')
+        if self.pattern_exclude:
+            self.pattern_exclude = re.compile(self.pattern_exclude)
 
         # Do we want to stack multiple filters?
-        if config.has_option('ec2', 'stack_filters'):
-            self.stack_filters = config.getboolean('ec2', 'stack_filters')
-        else:
-            self.stack_filters = False
+        self.stack_filters = config.getboolean('ec2', 'stack_filters')
 
         # Instance filters (see boto and EC2 API docs). Ignore invalid filters.
-        self.ec2_instance_filters = defaultdict(list)
+        self.ec2_instance_filters = []
+
         if config.has_option('ec2', 'instance_filters'):
+            filters = config.get('ec2', 'instance_filters')
 
-            filters = [f for f in config.get('ec2', 'instance_filters').split(',') if f]
+            if self.stack_filters and '&' in filters:
+                self.fail_with_error("AND filters along with stack_filter enabled is not supported.\n")
 
-            for instance_filter in filters:
-                instance_filter = instance_filter.strip()
-                if not instance_filter or '=' not in instance_filter:
-                    continue
-                filter_key, filter_value = [x.strip() for x in instance_filter.split('=', 1)]
-                if not filter_key:
-                    continue
-                self.ec2_instance_filters[filter_key].append(filter_value)
+            filter_sets = [f for f in filters.split(',') if f]
+
+            for filter_set in filter_sets:
+                filters = {}
+                filter_set = filter_set.strip()
+                for instance_filter in filter_set.split("&"):
+                    instance_filter = instance_filter.strip()
+                    if not instance_filter or '=' not in instance_filter:
+                        continue
+                    filter_key, filter_value = [x.strip() for x in instance_filter.split('=', 1)]
+                    if not filter_key:
+                        continue
+                    filters[filter_key] = filter_value
+                self.ec2_instance_filters.append(filters.copy())
 
     def parse_cli_args(self):
         ''' Command line argument processing '''
@@ -582,12 +569,12 @@ class Ec2Inventory(object):
             if self.ec2_instance_filters:
                 if self.stack_filters:
                     filters_dict = {}
-                    for filter_key, filter_values in self.ec2_instance_filters.items():
-                        filters_dict[filter_key] = filter_values
+                    for filters in self.ec2_instance_filters:
+                        filters_dict.update(filters)
                     reservations.extend(conn.get_all_instances(filters=filters_dict))
                 else:
-                    for filter_key, filter_values in self.ec2_instance_filters.items():
-                        reservations.extend(conn.get_all_instances(filters={filter_key: filter_values}))
+                    for filters in self.ec2_instance_filters:
+                        reservations.extend(conn.get_all_instances(filters=filters))
             else:
                 reservations = conn.get_all_instances()
 
@@ -623,6 +610,33 @@ class Ec2Inventory(object):
                 error = "Error connecting to %s backend.\n%s" % (backend, e.message)
             self.fail_with_error(error, 'getting EC2 instances')
 
+    def tags_match_filters(self, tags):
+        ''' return True if given tags match configured filters '''
+        if not self.ec2_instance_filters:
+            return True
+
+        for filters in self.ec2_instance_filters:
+            for filter_name, filter_value in filters.items():
+                if filter_name[:4] != 'tag:':
+                    continue
+                filter_name = filter_name[4:]
+                if filter_name not in tags:
+                    if self.stack_filters:
+                        return False
+                    continue
+                if isinstance(filter_value, list):
+                    if self.stack_filters and tags[filter_name] not in filter_value:
+                        return False
+                    if not self.stack_filters and tags[filter_name] in filter_value:
+                        return True
+                if isinstance(filter_value, six.string_types):
+                    if self.stack_filters and tags[filter_name] != filter_value:
+                        return False
+                    if not self.stack_filters and tags[filter_name] == filter_value:
+                        return True
+
+        return self.stack_filters
+
     def get_rds_instances_by_region(self, region):
         ''' Makes an AWS API call to the list of RDS instances in a particular
         region '''
@@ -648,8 +662,8 @@ class Ec2Inventory(object):
                         instance.tags = {}
                         for tag in tags:
                             instance.tags[tag['Key']] = tag['Value']
-
-                        self.add_rds_instance(instance, region)
+                        if self.tags_match_filters(instance.tags):
+                            self.add_rds_instance(instance, region)
                     if not marker:
                         break
         except boto.exception.BotoServerError as e:
@@ -688,7 +702,7 @@ class Ec2Inventory(object):
             if 'LatestRestorableTime' in c:
                 del c['LatestRestorableTime']
 
-            if self.ec2_instance_filters == {}:
+            if not self.ec2_instance_filters:
                 matches_filter = True
             else:
                 matches_filter = False
@@ -700,14 +714,18 @@ class Ec2Inventory(object):
                 c['Tags'] = tags['TagList']
 
                 if self.ec2_instance_filters:
-                    for filter_key, filter_values in self.ec2_instance_filters.items():
-                        # get AWS tag key e.g. tag:env will be 'env'
-                        tag_name = filter_key.split(":", 1)[1]
-                        # Filter values is a list (if you put multiple values for the same tag name)
-                        matches_filter = any(d['Key'] == tag_name and d['Value'] in filter_values for d in c['Tags'])
+                    for filters in self.ec2_instance_filters:
+                        for filter_key, filter_values in filters.items():
+                            # get AWS tag key e.g. tag:env will be 'env'
+                            tag_name = filter_key.split(":", 1)[1]
+                            # Filter values is a list (if you put multiple values for the same tag name)
+                            matches_filter = any(d['Key'] == tag_name and d['Value'] in filter_values for d in c['Tags'])
+
+                            if matches_filter:
+                                # it matches a filter, so stop looking for further matches
+                                break
 
                         if matches_filter:
-                            # it matches a filter, so stop looking for further matches
                             break
 
             except Exception as e:

contrib/inventory/proxmox.py

@@ -53,7 +53,9 @@ class ProxmoxVM(dict):
 
 
 class ProxmoxVMList(list):
-    def __init__(self, data=[]):
+    def __init__(self, data=None):
+        data = [] if data is None else data
+
         for item in data:
             self.append(ProxmoxVM(item))
 

contrib/inventory/rax.py

@@ -233,13 +233,17 @@ def _list_into_cache(regions):
                               'RAX_ACCESS_NETWORK', 'public', value_type='list')
     except TypeError:
         # Ansible 2.2.x and below
+        # pylint: disable=unexpected-keyword-arg
         networks = get_config(p, 'rax', 'access_network',
                               'RAX_ACCESS_NETWORK', 'public', islist=True)
     try:
         try:
+            # Ansible 2.3+
             ip_versions = map(int, get_config(p, 'rax', 'access_ip_version',
                                               'RAX_ACCESS_IP_VERSION', 4, value_type='list'))
         except TypeError:
+            # Ansible 2.2.x and below
+            # pylint: disable=unexpected-keyword-arg
             ip_versions = map(int, get_config(p, 'rax', 'access_ip_version',
                                               'RAX_ACCESS_IP_VERSION', 4, islist=True))
     except:
@@ -438,6 +442,7 @@ def setup():
                                      value_type='list')
         except TypeError:
             # Ansible 2.2.x and below
+            # pylint: disable=unexpected-keyword-arg
             region_list = get_config(p, 'rax', 'regions', 'RAX_REGION', 'all',
                                      islist=True)
 

contrib/inventory/vmware_inventory.py

@@ -41,6 +41,7 @@ from __future__ import print_function
 import atexit
 import datetime
 import getpass
+import itertools
 import json
 import os
 import re
@@ -84,6 +85,14 @@ def regex_match(s, pattern):
         return False
 
 
+def select_chain_match(inlist, key, pattern):
+    '''Get a key from a list of dicts, squash values to a single list, then filter'''
+    outlist = [x[key] for x in inlist]
+    outlist = list(itertools.chain(*outlist))
+    outlist = [x for x in outlist if regex_match(x, pattern)]
+    return outlist
+
+
 class VMwareMissingHostException(Exception):
     pass
 
@@ -127,6 +136,7 @@ class VMWareInventory(object):
     # use jinja environments to allow for custom filters
     env = Environment()
     env.filters['regex_match'] = regex_match
+    env.filters['select_chain_match'] = select_chain_match
 
     # translation table for attributes to fetch for known vim types
 

docs/bin/generate_man.py

@@ -1,6 +1,8 @@
 #!/usr/bin/env python
 
+import optparse
 import os
+import pprint
 import sys
 
 from jinja2 import Environment, FileSystemLoader
@@ -8,6 +10,46 @@ from jinja2 import Environment, FileSystemLoader
 from ansible.module_utils._text import to_bytes
 
 
+def generate_parser():
+    p = optparse.OptionParser(
+        version='%prog 1.0',
+        usage='usage: %prog [options]',
+        description='Generate cli documentation from cli docstrings',
+    )
+
+    p.add_option("-t", "--template-file", action="store", dest="template_file", default="../templates/man.j2", help="path to jinja2 template")
+    p.add_option("-o", "--output-dir", action="store", dest="output_dir", default='/tmp/', help="Output directory for rst files")
+    p.add_option("-f", "--output-format", action="store", dest="output_format", default='man', help="Output format for docs (the default 'man' or 'rst')")
+    return p
+
+
+# from https://www.python.org/dev/peps/pep-0257/
+def trim_docstring(docstring):
+    if not docstring:
+        return ''
+    # Convert tabs to spaces (following the normal Python rules)
+    # and split into a list of lines:
+    lines = docstring.expandtabs().splitlines()
+    # Determine minimum indentation (first line doesn't count):
+    indent = sys.maxsize
+    for line in lines[1:]:
+        stripped = line.lstrip()
+        if stripped:
+            indent = min(indent, len(line) - len(stripped))
+    # Remove indentation (first line is special):
+    trimmed = [lines[0].strip()]
+    if indent < sys.maxsize:
+        for line in lines[1:]:
+            trimmed.append(line[indent:].rstrip())
+    # Strip off trailing and leading blank lines:
+    while trimmed and not trimmed[-1]:
+        trimmed.pop()
+    while trimmed and not trimmed[0]:
+        trimmed.pop(0)
+    # Return a single string:
+    return '\n'.join(trimmed)
+
+
 def get_options(optlist):
     ''' get actual options '''
 
@@ -24,107 +66,215 @@ def get_options(optlist):
     return opts
 
 
+def get_option_groups(option_parser):
+    groups = []
+    for option_group in option_parser.option_groups:
+        group_info = {}
+        group_info['desc'] = option_group.get_description()
+        group_info['options'] = option_group.option_list
+        group_info['group_obj'] = option_group
+        groups.append(group_info)
+    return groups
+
+
 def opt_doc_list(cli):
     ''' iterate over options lists '''
 
     results = []
-    for optg in cli.parser.option_groups:
-        results.extend(get_options(optg.option_list))
+    for option_group in cli.parser.option_groups:
+        results.extend(get_options(option_group.option_list))
 
     results.extend(get_options(cli.parser.option_list))
 
     return results
 
 
-def opts_docs(cli, name):
+# def opts_docs(cli, name):
+def opts_docs(cli_class_name, cli_module_name):
     ''' generate doc structure from options '''
 
-    # cli name
-    if '-' in name:
-        name = name.split('-')[1]
-    else:
-        name = 'adhoc'
+    cli_name = 'ansible-%s' % cli_module_name
+    if cli_module_name == 'adhoc':
+        cli_name = 'ansible'
 
-    # cli info
+    # WIth no action/subcommand
+    # shared opts set
+    # instantiate each cli and ask its options
+    cli_klass = getattr(__import__("ansible.cli.%s" % cli_module_name,
+                                   fromlist=[cli_class_name]), cli_class_name)
+    cli = cli_klass([])
+
+    # parse the common options
+    try:
+        cli.parse()
+    except:
+        pass
+
+    # base/common cli info
     docs = {
-        'cli': name,
+        'cli': cli_module_name,
+        'cli_name': cli_name,
         'usage': cli.parser.usage,
         'short_desc': cli.parser.description,
-        'long_desc': cli.__doc__,
+        'long_desc': trim_docstring(cli.__doc__),
+        'actions': {},
     }
+    option_info = {'option_names': [],
+                   'options': [],
+                   'groups': []}
 
+    for extras in ('ARGUMENTS'):
+        if hasattr(cli, extras):
+            docs[extras.lower()] = getattr(cli, extras)
+
+    common_opts = opt_doc_list(cli)
+    groups_info = get_option_groups(cli.parser)
+    shared_opt_names = []
+    for opt in common_opts:
+        shared_opt_names.extend(opt.get('options', []))
+
+    option_info['options'] = common_opts
+    option_info['option_names'] = shared_opt_names
+
+    option_info['groups'].extend(groups_info)
+
+    docs.update(option_info)
+
+    # now for each action/subcommand
     # force populate parser with per action options
-    if cli.VALID_ACTIONS:
-        docs['actions'] = {}
+
+    # use class attrs not the attrs on a instance (not that it matters here...)
+    for action in getattr(cli_klass, 'VALID_ACTIONS', ()):
+        # instantiate each cli and ask its options
+        action_cli_klass = getattr(__import__("ansible.cli.%s" % cli_module_name,
+                                              fromlist=[cli_class_name]), cli_class_name)
+        # init with args with action added?
+        cli = action_cli_klass([])
+        cli.args.append(action)
+
+        try:
+            cli.parse()
+        except:
+            pass
+
+        # FIXME/TODO: needed?
         # avoid dupe errors
         cli.parser.set_conflict_handler('resolve')
-        for action in cli.VALID_ACTIONS:
-            cli.args.append(action)
-            cli.set_action()
-            docs['actions'][action] = getattr(cli, 'execute_%s' % action).__doc__
+
+        cli.set_action()
+
+        action_info = {'option_names': [],
+                       'options': []}
+        # docs['actions'][action] = {}
+        # docs['actions'][action]['name'] = action
+        action_info['name'] = action
+        action_info['desc'] = trim_docstring(getattr(cli, 'execute_%s' % action).__doc__)
+
+        # docs['actions'][action]['desc'] = getattr(cli, 'execute_%s' % action).__doc__.strip()
+        action_doc_list = opt_doc_list(cli)
+
+        uncommon_options = []
+        for action_doc in action_doc_list:
+            # uncommon_options = []
+
+            option_aliases = action_doc.get('options', [])
+            for option_alias in option_aliases:
+
+                if option_alias in shared_opt_names:
+                    continue
+
+                # TODO: use set
+                if option_alias not in action_info['option_names']:
+                    action_info['option_names'].append(option_alias)
+
+                if action_doc in action_info['options']:
+                    continue
+
+                uncommon_options.append(action_doc)
+
+            action_info['options'] = uncommon_options
+
+        docs['actions'][action] = action_info
 
     docs['options'] = opt_doc_list(cli)
-
     return docs
 
+
 if __name__ == '__main__':
 
-    template_file = 'man.j2'
+    parser = generate_parser()
 
+    options, args = parser.parse_args()
+
+    template_file = options.template_file
+    template_path = os.path.expanduser(template_file)
+    template_dir = os.path.abspath(os.path.dirname(template_path))
+    template_basename = os.path.basename(template_file)
+
+    output_dir = os.path.abspath(options.output_dir)
+    output_format = options.output_format
+
+    cli_modules = args
+
+    # various cli parsing things checks sys.argv if the 'args' that are passed in are []
+    # so just remove any args so the cli modules dont try to parse them resulting in warnings
+    sys.argv = [sys.argv[0]]
     # need to be in right dir
     os.chdir(os.path.dirname(__file__))
 
     allvars = {}
     output = {}
     cli_list = []
-    for binary in os.listdir('../../lib/ansible/cli'):
+    cli_bin_name_list = []
+
+    # for binary in os.listdir('../../lib/ansible/cli'):
+    for cli_module_name in cli_modules:
+        binary = os.path.basename(os.path.expanduser(cli_module_name))
 
         if not binary.endswith('.py'):
             continue
         elif binary == '__init__.py':
             continue
 
-        libname = os.path.splitext(binary)[0]
-        print("Found CLI %s" % libname)
+        cli_name = os.path.splitext(binary)[0]
 
-        if libname == 'adhoc':
-            myclass = 'AdHocCLI'
-            output[libname] = 'ansible.1.asciidoc.in'
+        if cli_name == 'adhoc':
+            cli_class_name = 'AdHocCLI'
+            # myclass = 'AdHocCLI'
+            output[cli_name] = 'ansible.1.asciidoc.in'
+            cli_bin_name = 'ansible'
         else:
-            myclass = "%sCLI" % libname.capitalize()
-            output[libname] = 'ansible-%s.1.asciidoc.in' % libname
+            # myclass = "%sCLI" % libname.capitalize()
+            cli_class_name = "%sCLI" % cli_name.capitalize()
+            output[cli_name] = 'ansible-%s.1.asciidoc.in' % cli_name
+            cli_bin_name = 'ansible-%s' % cli_name
 
-        # instantiate each cli and ask its options
-        mycli = getattr(__import__("ansible.cli.%s" % libname, fromlist=[myclass]), myclass)
-        cli_object = mycli([])
-        try:
-            cli_object.parse()
-        except:
-            # no options passed, we expect errors
-            pass
-
-        allvars[libname] = opts_docs(cli_object, libname)
-
-        for extras in ('ARGUMENTS'):
-            if hasattr(cli_object, extras):
-                allvars[libname][extras.lower()] = getattr(cli_object, extras)
+        # FIXME:
+        allvars[cli_name] = opts_docs(cli_class_name, cli_name)
+        cli_bin_name_list.append(cli_bin_name)
 
     cli_list = allvars.keys()
-    for libname in cli_list:
+
+    doc_name_formats = {'man': '%s.1.asciidoc.in',
+                        'rst': '%s.rst'}
+
+    for cli_name in cli_list:
 
         # template it!
-        env = Environment(loader=FileSystemLoader('../templates'))
-        template = env.get_template('man.j2')
+        env = Environment(loader=FileSystemLoader(template_dir))
+        template = env.get_template(template_basename)
 
         # add rest to vars
-        tvars = allvars[libname]
+        tvars = allvars[cli_name]
         tvars['cli_list'] = cli_list
-        tvars['cli'] = libname
+        tvars['cli_bin_name_list'] = cli_bin_name_list
+        tvars['cli'] = cli_name
         if '-i' in tvars['options']:
             print('uses inventory')
 
         manpage = template.render(tvars)
-        filename = '../man/man1/%s' % output[libname]
+        filename = os.path.join(output_dir, doc_name_formats[output_format] % tvars['cli_name'])
+
         with open(filename, 'wb') as f:
             f.write(to_bytes(manpage))
-            print("Wrote man docs to %s" % filename)
+            print("Wrote doc to %s" % filename)

docs/bin/plugin_formatter.py

@@ -26,6 +26,7 @@ import datetime
 import glob
 import optparse
 import os
+import pprint
 import re
 import sys
 import warnings
@@ -41,7 +42,7 @@ except ImportError:
 
 import yaml
 from jinja2 import Environment, FileSystemLoader
-from six import iteritems
+from six import iteritems, string_types
 
 from ansible.errors import AnsibleError
 from ansible.module_utils._text import to_bytes
@@ -120,6 +121,9 @@ def write_data(text, output_dir, outputname, module=None):
     if output_dir is not None:
         if module:
             outputname = outputname % module
+
+        if not os.path.exists(output_dir):
+            os.makedirs(output_dir)
         fname = os.path.join(output_dir, outputname)
         fname = fname.replace(".py", "")
         with open(fname, 'wb') as f:
@@ -152,6 +156,7 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
             least one key, '_modules' which contains a list of module names in
             that category.  Any other keys in the dict are subcategories with
             the same structure.
+
     '''
 
     categories = dict()
@@ -174,7 +179,7 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
 
         # Do not list blacklisted modules
         module = os.path.splitext(os.path.basename(module_path))[0]
-        if module in plugin_docs.BLACKLIST['MODULE']:
+        if module in plugin_docs.BLACKLIST['MODULE'] or module == 'base':
             continue
 
         # If requested, limit module documentation building only to passed-in
@@ -207,15 +212,21 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
         # Start at the second directory because we don't want the "vendor"
         mod_path_only = os.path.dirname(module_path[len(module_dir):])
 
+        module_categories = []
         # build up the categories that this module belongs to
         for new_cat in mod_path_only.split('/')[1:]:
             if new_cat not in category:
                 category[new_cat] = dict()
                 category[new_cat]['_modules'] = []
+            module_categories.append(new_cat)
             category = category[new_cat]
 
         category['_modules'].append(module)
 
+        # the category we will use in links (so list_of_all_plugins can point to plugins/action_plugins/*'
+        if module_categories:
+            primary_category = module_categories[0]
+
         # use ansible core library to parse out doc metadata YAML and plaintext examples
         doc, examples, returndocs, metadata = plugin_docs.get_docstring(module_path, verbose=verbose)
 
@@ -227,6 +238,8 @@ def get_module_info(module_dir, limit_to_modules=None, verbose=False):
                                'doc': doc,
                                'examples': examples,
                                'returndocs': returndocs,
+                               'categories': module_categories,
+                               'primary_category': primary_category,
                                }
 
     # keep module tests out of becoming module docs
@@ -247,6 +260,7 @@ def generate_parser():
 
     p.add_option("-A", "--ansible-version", action="store", dest="ansible_version", default="unknown", help="Ansible version number")
     p.add_option("-M", "--module-dir", action="store", dest="module_dir", default=MODULEDIR, help="Ansible library path")
+    p.add_option("-P", "--plugin-type", action="store", dest="plugin_type", default='modules', help="The type of plugin (plugins, modules)")
     p.add_option("-T", "--template-dir", action="store", dest="template_dir", default="hacking/templates", help="directory containing Jinja2 templates")
     p.add_option("-t", "--type", action='store', dest='type', choices=['rst'], default='rst', help="Document type")
     p.add_option("-v", "--verbose", action='store_true', default=False, help="Verbose")
@@ -258,7 +272,7 @@ def generate_parser():
     return p
 
 
-def jinja2_environment(template_dir, typ):
+def jinja2_environment(template_dir, typ, plugin_type):
 
     env = Environment(loader=FileSystemLoader(template_dir),
                       variable_start_string="@{",
@@ -273,14 +287,13 @@ def jinja2_environment(template_dir, typ):
         env.filters['fmt'] = rst_fmt
         env.filters['xline'] = rst_xline
         templates['plugin'] = env.get_template('plugin.rst.j2')
-        templates['category_list'] = env.get_template('modules_by_category.rst.j2')
-        templates['support_list'] = env.get_template('modules_by_support.rst.j2')
-        templates['list_of_CATEGORY_modules'] = env.get_template('list_of_CATEGORY_modules.rst.j2')
-        outputname = "%s_module.rst"
+        templates['category_list'] = env.get_template('%s_by_category.rst.j2' % plugin_type)
+        templates['support_list'] = env.get_template('%s_by_support.rst.j2' % plugin_type)
+        templates['list_of_CATEGORY_modules'] = env.get_template('list_of_CATEGORY_%s.rst.j2' % plugin_type)
     else:
         raise Exception("unknown module format type: %s" % typ)
 
-    return templates, outputname
+    return templates
 
 
 def too_old(added):
@@ -296,24 +309,54 @@ def too_old(added):
     return added_float < TO_OLD_TO_BE_NOTABLE
 
 
-def process_modules(module_map, templates, outputname, output_dir, ansible_version):
+def process_modules(module_map, templates, outputname, output_dir, ansible_version, plugin_type):
     for module in module_map:
-        print("rendering: %s" % module)
+        # print("rendering: %s" % module)
+
+        # pprint.pprint(('process_modules module:', module))
 
         fname = module_map[module]['path']
 
+        # pprint.pprint(('process_modules module_info: ', module_map[module]))
+
+        module_categories = module_map[module].get('categories', [])
+
         # crash if module is missing documentation and not explicitly hidden from docs index
         if module_map[module]['doc'] is None:
-            sys.exit("*** ERROR: MODULE MISSING DOCUMENTATION: %s, %s ***\n" % (fname, module))
+            print("%s: ERROR: MODULE MISSING DOCUMENTATION" % (fname,))
+            _doc = {'module': module,
+                    'version_added': '2.4',
+                    'filename': fname}
+            module_map[module]['doc'] = _doc
+            # continue
 
         # Going to reference this heavily so make a short name to reference it by
         doc = module_map[module]['doc']
 
+        # pprint.pprint(('process_modules doc: ', doc))
+
+        # add some defaults for plugins that dont have most of the info
+        doc['module'] = doc.get('module', module)
+        doc['version_added'] = doc.get('version_added', 'historical')
+
+        doc['plugin_type'] = plugin_type
+
         if module_map[module]['deprecated'] and 'deprecated' not in doc:
-            sys.exit("*** ERROR: DEPRECATED MODULE MISSING 'deprecated' DOCUMENTATION: %s, %s ***\n" % (fname, module))
+            print("%s: WARNING: MODULE MISSING DEPRECATION DOCUMENTATION: %s" % (fname, 'deprecated'))
+
+        required_fields = ('short_description',)
+        for field in required_fields:
+            if field not in doc:
+                print("%s: WARNING: MODULE MISSING field '%s'" % (fname, field))
+
+        not_nullable_fields = ('short_description',)
+        for field in not_nullable_fields:
+            if field in doc and doc[field] in (None, ''):
+                print("%s: WARNING: MODULE field '%s' DOCUMENTATION is null/empty value=%s" % (fname, field, doc[field]))
 
         if 'version_added' not in doc:
-            sys.exit("*** ERROR: missing version_added in: %s ***\n" % module)
+            pprint.pprint(doc)
+            # sys.exit("*** ERROR: missing version_added in: %s ***\n" % module)
 
         #
         # The present template gets everything from doc so we spend most of this
@@ -366,25 +409,54 @@ def process_modules(module_map, templates, outputname, output_dir, ansible_versi
         doc['docuri'] = doc['module'].replace('_', '-')
         doc['now_date'] = datetime.date.today().strftime('%Y-%m-%d')
         doc['ansible_version'] = ansible_version
-        doc['plainexamples'] = module_map[module]['examples']  # plain text
+
+        # check the 'deprecated' field in doc. We expect a dict potentially with 'why', 'version', and 'alternative' fields
+        # examples = module_map[module]['examples']
+        # print('\n\n%s: type of examples: %s\n' % (module, type(examples)))
+        # if examples and not isinstance(examples, (str, unicode, list)):
+        #    raise TypeError('module %s examples is wrong type (%s): %s' % (module, type(examples), examples))
+
+        # use 'examples' for 'plainexamples' if 'examples' is a string
+        if isinstance(module_map[module]['examples'], string_types):
+            doc['plainexamples'] = module_map[module]['examples']  # plain text
+        else:
+            doc['plainexamples'] = ''
+
         doc['metadata'] = module_map[module]['metadata']
 
+        # pprint.pprint(module_map[module]
         if module_map[module]['returndocs']:
             try:
                 doc['returndocs'] = yaml.safe_load(module_map[module]['returndocs'])
-            except:
-                print("could not load yaml: %s" % module_map[module]['returndocs'])
-                raise
+            except Exception as e:
+                print("%s:%s:yaml error:%s:returndocs=%s" % (fname, module, e, module_map[module]['returndocs']))
+                doc['returndocs'] = None
         else:
             doc['returndocs'] = None
 
+        doc['author'] = doc.get('author', ['UNKNOWN'])
+        if isinstance(doc['author'], string_types):
+            doc['author'] = [doc['author']]
+
+        # print('about to template')
+        # pprint.pprint(doc)
         text = templates['plugin'].render(doc)
 
-        write_data(text, output_dir, outputname, module)
+        # plugins get namespace dirs but modules do not
+        if plugin_type == 'plugins':
+            for module_category in module_categories:
+                category_output_dir = os.path.join(output_dir, 'plugins', '%s' % module_category)
+                write_data(text, category_output_dir, outputname, module)
+        else:
+                write_data(text, output_dir, outputname, module)
 
 
-def process_categories(mod_info, categories, templates, output_dir, output_name):
+def process_categories(mod_info, categories, templates,
+                       output_dir, output_name, plugin_type):
     for category in sorted(categories.keys()):
+        if (plugin_type, category) == ('plugins', ''):
+            print('skipping unknown cat: %s' % category)
+            continue
         module_map = categories[category]
         category_filename = output_name % category
 
@@ -392,21 +464,23 @@ def process_categories(mod_info, categories, templates, output_dir, output_name)
 
         # start a new category file
 
-        category = category.replace("_", " ")
-        category = category.title()
+        category_name = category.replace("_", " ")
+        category_title = category_name.title()
 
         subcategories = dict((k, v) for k, v in module_map.items() if k != '_modules')
-        template_data = {'title': category,
+        template_data = {'title': category_title,
+                         'category_name': category_name,
                          'category': module_map,
                          'subcategories': subcategories,
                          'module_info': mod_info,
+                         'plugin_type': plugin_type
                          }
 
         text = templates['list_of_CATEGORY_modules'].render(template_data)
         write_data(text, output_dir, category_filename)
 
 
-def process_support_levels(mod_info, templates, output_dir):
+def process_support_levels(mod_info, templates, output_dir, plugin_type):
     supported_by = {'Ansible Core Team': {'slug': 'core_supported',
                                           'modules': [],
                                           'output': 'core_maintained.rst',
@@ -450,8 +524,14 @@ These modules are currently shipped with Ansible, but will most likely be shippe
                                           """},
                     }
 
+    # only gen support pages for modules for now, need to split and namespace templates and generated docs
+    if plugin_type == 'plugins':
+        return
     # Separate the modules by support_level
     for module, info in mod_info.items():
+        if not info.get('metadata', None):
+            print('no metadata for %s' % module)
+            continue
         if info['metadata']['supported_by'] == 'core':
             supported_by['Ansible Core Team']['modules'].append(module)
         elif info['metadata']['supported_by'] == 'network':
@@ -492,34 +572,57 @@ def main():
     (options, args) = p.parse_args()
     validate_options(options)
 
-    templates, outputname = jinja2_environment(options.template_dir, options.type)
+    plugin_type = options.plugin_type
+
+    if plugin_type == 'modules':
+        templates = jinja2_environment(options.template_dir, options.type, plugin_type)
+        output_dir = options.output_dir
+        # trim trailing s off of plugin_type for plugin_type=='modules'. ie 'copy_module.rst'
+        outputname = '%s_' + '%s.rst' % plugin_type[:-1]
+    else:
+        templates = jinja2_environment(options.template_dir, options.type, 'plugins')
+        # for plugins, just use 'ssh.rst' vs 'ssh_module.rst'
+        outputname = '%s.rst'
+        output_dir = '%s/plugins/%s' % (options.output_dir, plugin_type)
 
     # Convert passed-in limit_to_modules to None or list of modules.
     if options.limit_to_modules is not None:
         options.limit_to_modules = [s.lower() for s in options.limit_to_modules.split(",")]
 
-    mod_info, categories = get_module_info(options.module_dir, limit_to_modules=options.limit_to_modules,
-                                           verbose=options.verbose)
+    mod_info, categories = get_module_info(options.module_dir, limit_to_modules=options.limit_to_modules, verbose=options.verbose)
 
     categories['all'] = {'_modules': mod_info.keys()}
 
+    # pprint.pprint(categories)
+    # pprint.pprint(mod_info)
+    # pprint.pprint(dict(mod_info))
     # Transform the data
     if options.type == 'rst':
-        for record in mod_info.values():
-            record['doc']['short_description'] = rst_ify(record['doc']['short_description'])
+        for key, record in mod_info.items():
+            # pprint.pprint(('record', record))
+            if record.get('doc', None):
+                short_desc = record['doc']['short_description']
+                if short_desc is None:
+                    print('WARNING: short_description for %s is None' % key)
+                    short_desc = ''
+                record['doc']['short_description'] = rst_ify(short_desc)
 
-    # Write master category list
-    category_list_text = templates['category_list'].render(categories=sorted(categories.keys()))
-    write_data(category_list_text, options.output_dir, 'modules_by_category.rst')
+    if plugin_type == 'modules':
+        # Write master category list
+        category_list_text = templates['category_list'].render(categories=sorted(categories.keys()))
+        category_index_name = '%s_by_category.rst' % plugin_type
+        write_data(category_list_text, output_dir, category_index_name)
 
     # Render all the individual module pages
-    process_modules(mod_info, templates, outputname, options.output_dir, options.ansible_version)
+    process_modules(mod_info, templates, outputname, output_dir, options.ansible_version, plugin_type)
 
     # Render all the categories for modules
-    process_categories(mod_info, categories, templates, options.output_dir, "list_of_%s_modules.rst")
+    if plugin_type == 'modules':
+        category_list_name_template = 'list_of_%s_' + '%s.rst' % plugin_type
+        process_categories(mod_info, categories, templates, output_dir, category_list_name_template, plugin_type)
 
-    # Render all the categories for modules
-    process_support_levels(mod_info, templates, options.output_dir)
+        # Render all the categories for modules
+        process_support_levels(mod_info, templates, output_dir, plugin_type)
 
 
 if __name__ == '__main__':

docs/docsite/Makefile

@@ -4,12 +4,25 @@ FORMATTER=../bin/plugin_formatter.py
 TESTING_FORMATTER=../bin/testing_formatter.sh
 DUMPER=../bin/dump_keywords.py
 CONFIG_DUMPER=../bin/dump_config.py
+GENERATE_CLI=../bin/generate_man.py
 ifeq ($(shell echo $(OS) | egrep -ic 'Darwin|FreeBSD|OpenBSD|DragonFly'),1)
 CPUS ?= $(shell sysctl hw.ncpu|awk '{print $$2}')
 else
 CPUS ?= $(shell nproc)
 endif
 
+MODULE_ARGS=
+ifdef MODULES
+	MODULE_ARGS = -l $(MODULES)
+endif
+
+PLUGIN_ARGS=
+ifdef PLUGINS
+	PLUGIN_ARGS = -l $(PLUGINS)
+endif
+
+DOC_PLUGINS ?= cache callback connection inventory lookup strategy vars
+
 assertrst:
 ifndef rst
 	$(error specify document or pattern with rst=somefile.rst)
@@ -19,9 +32,14 @@ all: docs
 
 docs: clean htmldocs
 
-htmldocs: testing keywords modules staticmin config
+generate_rst: testing keywords modules plugins staticmin cli config
+
+htmldocs: generate_rst
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx html
 
+singlehtmldocs: generate_rst
+	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx singlehtml
+
 webdocs: docs
 
 #TODO: leaving htmlout removal for those having older versions, should eventually be removed also
@@ -42,12 +60,21 @@ clean:
 	-rm rst/list_of_*.rst
 	-rm rst/*_by_category.rst
 	-rm rst/*_module.rst
+	-rm rst/*_plugin.rst
 	-rm rst/*_maintained.rst
 	-rm rst/playbooks_directives.rst
 	-rm rst/playbooks_keywords.rst
+	-rm rst/plugins/*/*.rst
+	-rm rst/config.rst
+	-rm rst/ansible-*.rst
+	-rm rst/ansible.rst
 
 .PHONEY: docs clean
 
+# TODO: make generate_man output dir cli option
+cli: $(GENERATE_CLI)
+	PYTHONPATH=../../lib $(GENERATE_CLI) --template-file=../templates/cli_rst.j2 --output-dir=rst/ --output-format rst ../../lib/ansible/cli/*.py
+
 keywords: $(FORMATTER) ../templates/playbooks_keywords.rst.j2
 	PYTHONPATH=../../lib $(DUMPER) --template-dir=../templates --output-dir=rst/ -d ./keyword_desc.yml
 
@@ -55,12 +82,14 @@ config:
 	PYTHONPATH=../../lib $(CONFIG_DUMPER) --template-file=../templates/config.rst.j2 --output-dir=rst/ -d ../../lib/ansible/config/base.yml
 
 modules: $(FORMATTER) ../templates/plugin.rst.j2
-# Limit building of module documentation if requested.
-ifdef MODULES
-	PYTHONPATH=../../lib $(FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/ -l $(MODULES)	
-else
-	PYTHONPATH=../../lib $(FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/
-endif
+	PYTHONPATH=../../lib $(FORMATTER) -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/ $(MODULE_ARGS)
+
+plugins: $(FORMATTER) ../templates/plugin.rst.j2
+	@echo "looping over doc plugins"
+	for plugin in $(DOC_PLUGINS); \
+	do \
+		PYTHONPATH=../../lib $(FORMATTER) -t rst --plugin-type $$plugin --template-dir=../templates --module-dir=../../lib/ansible/plugins/$$plugin -o rst/ $(PLUGIN_ARGS); \
+	done
 
 testing:
 	$(TESTING_FORMATTER)

docs/docsite/Makefile.sphinx

@@ -2,7 +2,7 @@
 #
 
 # You can set these variables from the command line.
-SPHINXOPTS    = -j $(CPUS)
+SPHINXOPTS    = -j $(CPUS) -n -w rst_warnings
 SPHINXBUILD   = sphinx-build
 SPHINXPROJ    = sdfsdf
 SOURCEDIR     = rst
@@ -17,4 +17,4 @@ help:
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile.sphinx
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/docsite/keyword_desc.yml

@@ -1,55 +1,55 @@
 accelerate: DEPRECATED, set to True to use accelerate connection plugin.
-accelerate_ipv6: DEPRECATED, set to True to force accelerate plugin to use ipv6 for it's connection.
+accelerate_ipv6: "DEPRECATED, set to True to force accelerate plugin to use ipv6 for it's connection."
 accelerate_port: DEPRECATED, set to override default port use for accelerate connection.
-action: The 'action' to execute for a task, it normally translates into a C(module) or action plugin.
+action: "The 'action' to execute for a task, it normally translates into a C(module) or action plugin."
 args: DEPRECATED, A secondary way to add arguments into a task, it takes a dictionary in which keys map to options and values .. well you get it.
 always: List of tasks, in a block, that execute no matter if there is an error in the block or not.
-always_run: DEPRECATED, forces a task to run even in check mode, use check_mode directive instead.
+always_run: DEPRECATED, forces a task to run even in check mode, use :term:`check_mode` directive instead.
 any_errors_fatal: Force any un-handled task errors on any host to propagate to all hosts and end the play.
 async: Run a task asyncronouslly if the C(action) supports this.
-become: Boolean that controls if privilege escalation is used or not on Task execution.
-become_flags: A string of flag(s) to pass to the privilege escalation program when ``become`` is True.
+become: Boolean that controls if privilege escalation is used or not on :term:`Task` execution.
+become_flags: A string of flag(s) to pass to the privilege escalation program when :term:`become` is True.
 become_method: Which method of privilege escalation to use. i.e. sudo/su/etc.
-become_user: User that you 'become' after using privilege escalation, the remote/login user must have permissions to become this user.
+become_user: "User that you 'become' after using privilege escalation, the remote/login user must have permissions to become this user."
 block: List of tasks in a block.
-changed_when: Conditional expression that overrides the task's normal 'changed' status.
-check_mode: A boolean that controls if a task is executed in 'check' mode
+changed_when: "Conditional expression that overrides the task's normal 'changed' status."
+check_mode: "A boolean that controls if a task is executed in 'check' mode"
 connection: Allows you to change the connection plugin used for tasks to execute on the target.
-delay: Number of seconds to delay between retries, this setting only makes sense when used with 'until'.
+delay: Number of seconds to delay between retries, this setting is only used in combination with :term:`until`.
 delegate_facts: Boolean that allows you to apply facts to delegated host instead of inventory_hostname.
 delegate_to: Host to execute task instead of the target (inventory_hostname), connection vars from the delegated host will also be used for the task.
-diff: Toggle to make tasks return 'diff' information or not.
+diff: "Toggle to make tasks return 'diff' information or not."
 environment: A dictionary that gets converted into environment vars to be provided for the task upon execution.
-fact_path: Set the fact path option for the fact gathering plugin controlled by ``gather_facts``.
-failed_when: Conditional expression that overrides the task's normal 'failed' status.
+fact_path: Set the fact path option for the fact gathering plugin controlled by :term:`gather_facts`.
+failed_when: "Conditional expression that overrides the task's normal 'failed' status."
 force_handlers: Will force notified handler execution for hosts even if they failed during the play, it will not trigger if the play itself fails.
-gather_facts: A boolean that controls if the play will automatically run the 'setup' task to gather facts for the hosts.
-gather_subset: Allows you to pass subset options to the  fact gathering plugin controlled by ``gather_facts``.
-gather_timeout: Allows you to set the timeout for the fact gathering plugin controlled by ``gather_facts``.
-handlers: A section with tasks that are treated as handlers, these won't get executed normally, only when notified. After each section of tasks is complete.
-hosts: A list of groups, hosts or host pattern that translates into a list of hosts that are the play's target.
+gather_facts: "A boolean that controls if the play will automatically run the 'setup' task to gather facts for the hosts."
+gather_subset: Allows you to pass subset options to the  fact gathering plugin controlled by :term:`gather_facts`.
+gather_timeout: Allows you to set the timeout for the fact gathering plugin controlled by :term:`gather_facts`.
+handlers: "A section with tasks that are treated as handlers, these won't get executed normally, only when notified. After each section of tasks is complete."
+hosts: "A list of groups, hosts or host pattern that translates into a list of hosts that are the play's target."
 ignore_errors: Boolean that allows you to ignore task failures and continue with play. It does not affect connection errors.
-loop_control: Several keys here allow you to modify/set loop behaviour in a task see http://docs.ansible.com/ansible/latest/playbooks_loops.html#loop-control for details.
+loop_control: "Several keys here allow you to modify/set loop behaviour in a task see http://docs.ansible.com/ansible/latest/playbooks_loops.html#loop-control for details."
 max_fail_percentage: can be used to abort the run after a given percentage of hosts in the current batch has failed.
-name: It's a name, works mostly for documentation, in the case of tasks/handlers it can be an identifier.
+name: "It's a name, works mostly for documentation, in the case of tasks/handlers it can be an identifier."
 no_log: Boolean that controls information disclosure.
-notify: list of handlers to notify when the task returns a 'changed=True' status.
+notify: "list of handlers to notify when the task returns a 'changed=True' status."
 order: Controls the sorting of hosts as they are used for executing the play. Possible values are inventory (default), sorted, reverse_sorted, reverse_inventory and shuffle.
 poll: Sets the polling interval in seconds for async tasks (default 10s).
 port: Used to override the default port used in a connection.
-post_tasks: A list of tasks to execute after the ``tasks`` section.
-pre_tasks: A list of tasks to execute before ``roles``.
+post_tasks: A list of tasks to execute after the :term:`tasks` section.
+pre_tasks: A list of tasks to execute before :term:`roles`.
 remote_user: User used to log into the target via the connection plugin. AKA login user.
 register: Name of variable that will contain task status and module return data.
-rescue: List of tasks in a block that run if there is a task error in the main ``block`` list.
-retries: Number of retries before giving up in a 'until' loop, this setting is ignored on it's own.
+rescue: List of tasks in a :term:`block` that run if there is a task error in the main :term:`block` list.
+retries: "Number of retries before giving up in a :term:`until` loop. This setting is only used in combination with :term:`until`."
 roles: List of roles to be imported into the play
 run_once: Boolean that will bypass the host loop, forcing the task to execute on the first host available and will also apply any facts to all active hosts.
 serial: Defines the 'batch' of hosts to execute the current play until the end.
 strategy: Allows you to choose the connection plugin to use for the play.
 tags: Tags applied to the task or included tasks, this allows selecting subsets of tasks from the command line.
-tasks: Main list of tasks to execute in the play, they run after ``roles`` and before ``post_tasks``.
-until: This keyword implies a 'retry loop' that will go on until the condition supplied here is met or we hit the retry limit.
+tasks: Main list of tasks to execute in the play, they run after :term:`roles` and before :term:`post_tasks`.
+until: "This keyword implies a ':term:`retries` loop' that will go on until the condition supplied here is met or we hit the :term:`retries` limit."
 vars: Dictionary/map of variables
 vars_files: List of files that contain vars to include in the play.
 vars_prompt: list of variables to prompt for.

docs/docsite/rst/command_line_tools.rst

@@ -0,0 +1,11 @@
+Command Line Tools
+==================
+
+Most users are familiar with `ansible` and `ansible-playbook`, but those are not the only utilities Ansible provides.
+Below you is a complete list of Ansible utilities. Each page contains a description of the utility and a listing of supported parameters.	
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    ansible
+    ansible-*

docs/docsite/rst/committer_guidelines.rst

@@ -3,7 +3,7 @@ Committers Guidelines (for people with commit rights to Ansible on GitHub)
 
 These are the guidelines for people with commit access to Ansible. Committers are essentially acting as members of the Ansible Core team, although not necessarily as an employee of Ansible and Red Hat. Please read the guidelines before you commit.
 
-These guidelines apply to everyone. At the same time, this ISN’T a process document. So just use good judgement. You’ve been given commit access because we trust your judgement.
+These guidelines apply to everyone. At the same time, this ISN'T a process document. So just use good judgement. You've been given commit access because we trust your judgement.
 
 That said, use the trust wisely. 
 
@@ -19,18 +19,18 @@ Any other new features and changes to high level design should go through the pr
 Our Workflow on GitHub
 ======================
 
-As a committer, you may already know this, but our workflow forms a lot of our team policies. Please ensure you’re aware of the following workflow steps:
+As a committer, you may already know this, but our workflow forms a lot of our team policies. Please ensure you're aware of the following workflow steps:
 
 * Fork the repository upon which you want to do some work to your own personal repository
 * Work on the specific branch upon which you need to commit
-* Create a Pull Request back to the Ansible repository and tag the people you would like to review; assign someone as the primary “owner” of your request
+* Create a Pull Request back to the Ansible repository and tag the people you would like to review; assign someone as the primary "owner" of your request
 * Adjust code as necessary based on the Comments provided
 * Ask someone on the Core Team to do a final review and merge
 
 Addendum to workflow for Committers:
 ------------------------------------
 
-The Core Team is aware that this can be a difficult process at times. Sometimes, the team breaks the rules: Direct commits, merging their own PRs. This section is a set of guidelines. If you’re changing a comma in a doc, or making a very minor change, you can use your best judgement. This is another trust thing. The process is critical for any major change, but for little things or getting something done quickly, use your best judgement and make sure people on the team are aware of your work.
+The Core Team is aware that this can be a difficult process at times. Sometimes, the team breaks the rules: Direct commits, merging their own PRs. This section is a set of guidelines. If you're changing a comma in a doc, or making a very minor change, you can use your best judgement. This is another trust thing. The process is critical for any major change, but for little things or getting something done quickly, use your best judgement and make sure people on the team are aware of your work.
 
 Roles on Core
 =============
@@ -41,12 +41,12 @@ General Rules
 =============
 Individuals with direct commit access to ansible/ansible are entrusted with powers that allow them to do a broad variety of things--probably more than we can write down. Rather than rules, treat these as general *guidelines*, individuals with this power are expected to use their best judgement. 
 
-* Don’t
+* Don't
 
   - Commit directly.
   - Merge your own PRs. Someone else should have a chance to review and approve the PR merge. If you are a Core Committer, you have a small amount of leeway here for very minor changes.
   - Forget about alternate environments. Consider the alternatives--yes, people have bad environments, but they are the ones who need us the most.
-  - Drag your community team members down. Always discuss the technical merits, but you should never address the person’s limitations (you can later go for beers and call them idiots, but not in IRC/Github/etc.).
+  - Drag your community team members down. Always discuss the technical merits, but you should never address the person's limitations (you can later go for beers and call them idiots, but not in IRC/Github/etc.).
   - Forget about the maintenance burden. Some things are really cool to have, but they might not be worth shoehorning in if the maintenance burden is too great.
   - Break playbooks. Always keep backwards compatibility in mind.
   - Forget to keep it simple. Complexity breeds all kinds of problems.
@@ -55,7 +55,7 @@ Individuals with direct commit access to ansible/ansible are entrusted with powe
 
   - Squash, avoid merges whenever possible, use github's squash commits or cherry pick if needed (bisect thanks you).
   - Be active. Committers who have no activity on the project (through merges, triage, commits, etc.) will have their permissions suspended.
-  - Consider backwards compatibility (goes back to "don’t break existing playbooks").
+  - Consider backwards compatibility (goes back to "don't break existing playbooks").
   - Write tests. PRs with tests are looked at with more priority than PRs without tests that should have them included. While not all changes require tests, be sure to add them for bug fixes or functionality changes.
   - Discuss with other committers, specially when you are unsure of something.
   - Document! If your PR is a new feature or a change to behavior, make sure you've updated all associated documentation or have notified the right people to do so. It also helps to add the version of Core against which this documentation is compatible (to avoid confusion with stable versus devel docs, for backwards compatibility, etc.).

docs/docsite/rst/community.rst

@@ -2,7 +2,7 @@ Community Information & Contributing
 ````````````````````````````````````
 
 Ansible is an open source project designed to bring together administrators and developers of all kinds to collaborate on building
-IT automation solutions that work well for them.   
+IT automation solutions that work well for them.
 
 Should you wish to get more involved -- whether in terms of just asking a question, helping other users, introducing new people to Ansible, or helping with the software or documentation, we welcome your contributions to the project.
 
@@ -16,7 +16,7 @@ I've Got A Question
 
 We're happy to help!
 
-Ansible questions are best asked on the `Ansible Google Group Mailing List <http://groups.google.com/group/ansible-project>`_.  
+Ansible questions are best asked on the `Ansible Google Group Mailing List <http://groups.google.com/group/ansible-project>`_.
 
 This is a very large high-traffic list for answering questions and sharing tips
 and tricks. Anyone can join, and email delivery is optional if you just want to read the group online.  To cut down on spam, your first post is moderated, though posts are approved quickly.
@@ -42,10 +42,10 @@ This is a low-traffic read-only list, where we'll share release announcements an
 I'd Like To Help Share and Promote Ansible
 ------------------------------------------
 
-You can help share Ansible with others by telling friends and colleagues, writing a blog post, 
-or presenting at user groups (like DevOps groups or the local LUG).  
+You can help share Ansible with others by telling friends and colleagues, writing a blog post,
+or presenting at user groups (like DevOps groups or the local LUG).
 
-You are also welcome to share slides on speakerdeck, sign up for a free account and tag it “Ansible”. On Twitter, 
+You are also welcome to share slides on speakerdeck, sign up for a free account and tag it "Ansible". On Twitter,
 you can also share things with #ansible and may wish to `follow us <https://twitter.com/ansible>`_.
 
 I'd Like To Help Ansible Move Faster
@@ -73,31 +73,31 @@ more quickly.
 Do not use the issue tracker for "how do I do this" type questions.  These are great candidates
 for IRC or the mailing list instead where things are likely to be more of a discussion.
 
-To be respectful of reviewers' time and allow us to help everyone efficiently, please 
+To be respectful of reviewers' time and allow us to help everyone efficiently, please
 provide minimal well-reduced and well-commented examples versus sharing your entire production
-playbook.  Include playbook snippets and output where possible.  
+playbook.  Include playbook snippets and output where possible.
 
 When sharing YAML in playbooks, formatting can be preserved by using `code blocks <https://help.github.com/articles/creating-and-highlighting-code-blocks/>`_.
 
 For multiple-file content, we encourage use of gist.github.com.  Online pastebin content can expire, so it's nice to have things around for a longer term if they
 are referenced in a ticket.
 
-If you are not sure if something is a bug yet, you are welcome to ask about something on 
-the mailing list or IRC first.  
+If you are not sure if something is a bug yet, you are welcome to ask about something on
+the mailing list or IRC first.
 
-As we are a very high volume project, if you determine that 
+As we are a very high volume project, if you determine that
 you do have a bug, please be sure to open the issue yourself to ensure we have a record of
-it. Don’t rely on someone else in the community to file the bug report for you.
+it. Don't rely on someone else in the community to file the bug report for you.
 
 It may take some time to get to your report, see our information about priority flags below.
 
 I'd Like To Help With Documentation
 -----------------------------------
 
-Ansible documentation is a community project too!  
+Ansible documentation is a community project too!
 
-If you would like to help with the 
-documentation, whether correcting a typo or improving a section, or maybe even 
+If you would like to help with the
+documentation, whether correcting a typo or improving a section, or maybe even
 documenting a new feature, submit a GitHub pull request to  the code that
 lives in the ``docsite/rst`` subdirectory of the project for most pages, and there is an "Edit on GitHub"
 link up on those.
@@ -107,9 +107,9 @@ Module documentation is generated from a ``DOCUMENTATION`` structure embedded in
 For more information on module documentation see `How to document modules <http://docs.ansible.com/ansible/dev_guide/developing_modules_documenting.html>`_.
 
 Aside from modules, the main docs are in restructured text
-format.  
+format.
 
-If you aren’t comfortable with restructured text, you can also open a ticket on 
+If you aren't comfortable with restructured text, you can also open a ticket on
 GitHub about any errors you spot or sections you would like to see added. For more information
 on creating pull requests, please refer to the
 `GitHub help guide <https://help.github.com/articles/using-pull-requests>`_.
@@ -155,7 +155,7 @@ If you make a mistake you do not need to close your PR. Instead, create a clean
 with ``--force`` to overwrite the existing branch (permissible in this case as no one else should be using that
 branch as reference). Code comments won't be lost, they just won't be attached to the existing branch.
 
-We’ll then review your contributions and engage with you about questions and  so on.
+We'll then review your contributions and engage with you about questions and  so on.
 
 Because we have a very large and active community it may take awhile to get your contributions
 in!  See the notes about priorities in a later section for understanding our work queue.
@@ -172,14 +172,14 @@ are interested in writing new modules to be included in the core Ansible distrib
 to the `module development documentation <http://docs.ansible.com/developing_modules.html>`_.
 
 Ansible's aesthetic encourages simple, readable code and consistent,
-conservatively extending, backwards-compatible improvements. When contributing code to Ansible, 
+conservatively extending, backwards-compatible improvements. When contributing code to Ansible,
 please observe the following guidelines:
 
 - Code developed for Ansible needs to support Python2-2.6 or higher and Python3-3.5 or higher.
 - Use a 4-space indent, not  tabs.
 - We do not enforce 80 column lines; up to 160 columns are acceptable.
 - We do not accept 'style only' pull requests unless the code is nearly unreadable.
-- We are "PEP8ish", but not strictly compliant, see :doc:`testing_pep8` for more information.
+- We are not strictly compliant with PEP8. See :doc:`dev_guide/testing_pep8` for more information.
 
 You can also contribute by testing and revising other requests, especially if it is one you are interested
 in using. Please keep your comments clear and to the point, courteous and constructive, tickets are not a
@@ -228,7 +228,7 @@ To subscribe to a group from a non-google account, you can send an email to the
 IRC Meetings
 ------------
 
-The Ansible community holds regular IRC meetings on various topics, and anyone who is interested is invited to 
+The Ansible community holds regular IRC meetings on various topics, and anyone who is interested is invited to
 participate. For more information about Ansible meetings, consult the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
 
 Release Numbering
@@ -286,7 +286,7 @@ These labels don't really have a definition - they are a simple ordering.  Howev
 affecting a major module (yum, apt, etc) is likely to be prioritized higher than a module
 affecting a smaller number of users.
 
-Since we place a strong emphasis on testing and code review, it may take a few months for a minor feature to get merged. This doesn't mean that we'll be exhausting all of the higher-priority queues before getting to your ticket; we will also take periodic sweeps through the lower priority queues and give them some attention as well, particularly in the area of new module changes. 
+Since we place a strong emphasis on testing and code review, it may take a few months for a minor feature to get merged. This doesn't mean that we'll be exhausting all of the higher-priority queues before getting to your ticket; we will also take periodic sweeps through the lower priority queues and give them some attention as well, particularly in the area of new module changes.
 
 
 Every bit of effort helps - if you're wishing to expedite the inclusion of a P3 feature pull request for instance, the best thing you can do is help close P2 bug reports.

docs/docsite/rst/community/development_process.rst

@@ -41,9 +41,9 @@ Each module has at least one assigned maintainer, listed in a `maintainer's file
 .. _Ansibullbot: https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md
 .. _maintainer's file: https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml
 
-Some modules have no community maintainers assigned. In this case, the maintainer is listed as ``$team_ansible``. Ultimately, it’s our goal to have at least one community maintainer for every module.
+Some modules have no community maintainers assigned. In this case, the maintainer is listed as ``$team_ansible``. Ultimately, it's our goal to have at least one community maintainer for every module.
 
-The maintainer’s job is to review PRs and decide whether that PR should be merged (``shipit``) or revised (``needs_revision``).
+The maintainer's job is to review PRs and decide whether that PR should be merged (``shipit``) or revised (``needs_revision``).
 
 The ultimate goal of any pull request is to reach **shipit** status, where the Core team then decides whether the PR is ready to be merged. Not every PR that reaches the **shipit** label is actually ready to be merged, but the better our reviewers are, and the better our guidelines are, the more likely it will be that a PR that reaches **shipit** will be mergeable.
 
@@ -54,7 +54,7 @@ Workflow
 
 Ansibullbot runs continuously. You can generally expect to see changes to your issue or pull request within thirty minutes. Ansibullbot examines every open pull request in the repositories, and enforces state roughly according to the following workflow:
 
--  If a pull request has no workflow labels, it’s considered **new**. Files in the pull request are identified, and the maintainers of those files are pinged by the bot, along with instructions on how to review the pull request. (Note: sometimes we strip labels from a pull request to “reboot” this process.)
+-  If a pull request has no workflow labels, it's considered **new**. Files in the pull request are identified, and the maintainers of those files are pinged by the bot, along with instructions on how to review the pull request. (Note: sometimes we strip labels from a pull request to "reboot" this process.)
 -  If the module maintainer is not ``$team_ansible``, the pull request then goes into the **community_review** state.
 -  If the module maintainer is ``$team_ansible``, the pull request then goes into the **core_review** state (and probably sits for a while).
 -  If the pull request is in **community_review** and has received comments from the maintainer:
@@ -110,4 +110,4 @@ Special Labels
 
 -  **new_plugin**: this is for new modules or plugins that are not yet in Ansible.
 
-   **Note:** `new_plugin` kicks off a completely separate process, and frankly it doesn’t work very well at present. We’re working our best to improve this process.
+   **Note:** `new_plugin` kicks off a completely separate process, and frankly it doesn't work very well at present. We're working our best to improve this process.

docs/docsite/rst/community/other_tools_and_programs.rst

@@ -16,3 +16,5 @@ of some of the most popular of these tools.
 - `Ansigenome <https://github.com/nickjj/ansigenome>`_ is a command line tool designed to help you manage your Ansible roles.
 
 - `Awesome Ansible <https://github.com/jdauphant/awesome-ansible>`_ is a collaboratively curated list of awesome Ansible resources.
+
+- `Ansible Inventory Grapher <http://github.com/willthames/ansible-inventory-grapher>_` can be used to visually display inventory inheritance hierarchies and at what level a variable is defined in inventory.

docs/docsite/rst/community/reporting_bugs_and_features.rst

@@ -25,7 +25,7 @@ For multiple-file content, we encourage use of gist.github.com.  Online pastebin
 
 If you are not sure if something is a bug yet, you are welcome to ask about something on the mailing list or IRC first.
 
-As we are a very high volume project, if you determine that you do have a bug, please be sure to open the issue yourself to ensure we have a record of it. Don’t rely on someone else in the community to file the bug report for you.
+As we are a very high volume project, if you determine that you do have a bug, please be sure to open the issue yourself to ensure we have a record of it. Don't rely on someone else in the community to file the bug report for you.
 
 Requesting a feature
 ====================

docs/docsite/rst/dev_guide/developing_api.rst

@@ -44,8 +44,8 @@ In 2.0 things get a bit more complicated to start, but you end up with much more
     import json
     from collections import namedtuple
     from ansible.parsing.dataloader import DataLoader
-    from ansible.vars import VariableManager
-    from ansible.inventory import Inventory
+    from ansible.vars.manager import VariableManager
+    from ansible.inventory.manager import InventoryManager
     from ansible.playbook.play import Play
     from ansible.executor.task_queue_manager import TaskQueueManager
     from ansible.plugins.callback import CallbackBase
@@ -63,21 +63,21 @@ In 2.0 things get a bit more complicated to start, but you end up with much more
             This method could store the result in an instance attribute for retrieval later
             """
             host = result._host
-            print json.dumps({host.name: result._result}, indent=4)
+            print(json.dumps({host.name: result._result}, indent=4))
 
-    Options = namedtuple('Options', ['connection', 'module_path', 'forks', 'become', 'become_method', 'become_user', 'check'])
+    Options = namedtuple('Options', ['connection', 'module_path', 'forks', 'become', 'become_method', 'become_user', 'check', 'diff'])
     # initialize needed objects
-    variable_manager = VariableManager()
     loader = DataLoader()
-    options = Options(connection='local', module_path='/path/to/mymodules', forks=100, become=None, become_method=None, become_user=None, check=False)
+    options = Options(connection='local', module_path='/path/to/mymodules', forks=100, become=None, become_method=None, become_user=None, check=False,
+                      diff=False)
     passwords = dict(vault_pass='secret')
 
     # Instantiate our ResultCallback for handling results as they come in
     results_callback = ResultCallback()
 
     # create inventory and pass to var manager
-    inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list='localhost')
-    variable_manager.set_inventory(inventory)
+    inventory = InventoryManager(loader=loader, sources=['localhost'])
+    variable_manager = VariableManager(loader=loader, inventory=inventory)
 
     # create play with tasks
     play_source =  dict(

docs/docsite/rst/dev_guide/developing_modules_checklist.rst

@@ -55,7 +55,7 @@ The complete module metadata specification is here: `Ansible metadata block <htt
     * Examples--include them whenever possible and make sure they are reproducible.
     * Document the return structure of the module. Refer to :ref:`common_return_values` and :ref:`module_documenting` for additional information.
 * Predictable user interface: This is a particularly important section as it is also an area where we need significant improvements.
-    * Name consistency across modules (we’ve gotten better at this, but we still have many deviations).
+    * Name consistency across modules (we've gotten better at this, but we still have many deviations).
     * Declarative operation (not CRUD)--this makes it easy for a user not to care what the existing state is, just about the final state. ``started/stopped``, ``present/absent``--don't overload options too much. It is preferable to add a new, simple option than to add choices/states that don't fit with existing ones.
     * Keep options small, having them take large data structures might save us a few tasks, but adds a complex requirement that we cannot easily validate before passing on to the module.
     * Allow an "expert mode". This may sound like the absolute opposite of the previous one, but it is always best to let expert users deal with complex data. This requires different modules in some cases, so that you end up having one (1) expert module and several 'piecemeal' ones (ec2_vpc_net?). The reason for this is not, as many users express, because it allows a single task and keeps plays small (which just moves the data complexity into vars files, leaving you with a slightly different structure in another YAML file). It does, however, allow for a more 'atomic' operation against the underlying APIs and services.

docs/docsite/rst/dev_guide/developing_modules_documenting.rst

@@ -41,7 +41,8 @@ code.
 .. code-block:: python
 
     #!/usr/bin/python
-    # Copyright (c) 2017 Ansible Project
+    
+    # Copyright: (c) 2017, Ansible Project
     # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 Every file should have a copyright line with the original copyright holder.
@@ -61,8 +62,9 @@ add the newer line above the older one, like so:
 .. code-block:: python
 
     #!/usr/bin/python
-    # Copyright (c) 2017 [New Contributor(s)]
-    # Copyright (c) 2015 [Original Contributor(s)]
+    
+    # Copyright: (c) 2017, [New Contributor(s)]
+    # Copyright: (c) 2015, [Original Contributor(s)]
     # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 
@@ -84,11 +86,11 @@ For new modules, the following block can be simply added into your module
    * ``metadata_version`` is the version of the ``ANSIBLE_METADATA`` schema, *not* the version of the module.
    * Promoting a module's ``status`` or ``supported_by`` status should only be done by members of the Ansible Core Team.
 
-.. note:: Pre-released metdata version
+.. note:: Pre-released metadata version
 
     During development of Ansible-2.3, modules had an initial version of the
     metadata.  This version was modified slightly after release to fix some
-    points of confusion.  You may occassionally see PRs for modules where the
+    points of confusion.  You may occasionally see PRs for modules where the
     ANSIBLE_METADATA doesn't look quite right because of this.  Module
     metadata should be fixed before checking it into the repository.
 
@@ -109,11 +111,11 @@ Structure
 Fields
 ^^^^^^
 
-:metadata_version: An “X.Y” formatted string. X and Y are integers which
+:metadata_version: An "X.Y" formatted string. X and Y are integers which
    define the metadata format version. Modules shipped with Ansible are
    tied to an Ansible release, so we will only ship with a single version
-   of the metadata. We’ll increment Y if we add fields or legal values
-   to an existing field. We’ll increment X if we remove fields or values
+   of the metadata. We'll increment Y if we add fields or legal values
+   to an existing field. We'll increment X if we remove fields or values
    or change the type or meaning of a field.
    Current metadata_version is "1.1"
 :supported_by: This field records who supports the module.
@@ -130,13 +132,13 @@ Fields
    `Modules Support <http://docs.ansible.com/ansible/modules_support.html>`_.
 
 :status: This field records information about the module that is
-   important to the end user. It’s a list of strings. The default value
-   is a single element list [“preview”]. The following strings are valid
+   important to the end user. It's a list of strings. The default value
+   is a single element list ["preview"]. The following strings are valid
    statuses and have the following meanings:
 
-   :stableinterface: This means that the module’s parameters are
+   :stableinterface: This means that the module's parameters are
       stable. Every effort will be made not to remove parameters or change
-      their meaning. It is not a rating of the module’s code quality.
+      their meaning. It is not a rating of the module's code quality.
    :preview: This module is a tech preview. This means it may be
       unstable, the parameters may change, or it may require libraries or
       web services that are themselves subject to incompatible changes.
@@ -223,6 +225,8 @@ The following fields can be used and are all required unless specified otherwise
     * If `required` is false/missing, `default` may be specified (assumed 'null' if missing).
     * Ensure that the default parameter in the docs matches the default parameter in the code.
     * The default option must not be listed as part of the description.
+    * If the option is a boolean value, you can use any of the boolean values recognized by Ansible:
+      (such as true/false or yes/no).  Choose the one that reads better in the context of the option.
   :choices:
     List of option values. Should be absent if empty.
   :type:
@@ -333,8 +337,30 @@ Example::
         returned: when supported
         type: string
         sample: 2a5aeecc61dc98c4d780b14b330e3282
-    ...
+    '''
 
+    RETURN = '''
+    packages:
+        description: Information about package requirements
+        returned: On success
+        type: complex
+        contains:
+            missing:
+                description: Packages that are missing from the system
+                returned: success
+                type: list
+                sample:
+                    - libmysqlclient-dev
+                    - libxml2-dev
+            badversion:
+                description: Packages that are installed but at bad versions.
+                returned: success
+                type: list
+                sample:
+                    - package: libxml2-dev
+                      version: 2.9.4+dfsg1-2
+                      constraint: ">= 3.0"
+    '''
 
 .. note::
 

docs/docsite/rst/dev_guide/developing_modules_general.rst

@@ -312,5 +312,5 @@ use the ``#ansible`` channel.
 Credit
 ======
 
-Thank you to Thomas Stringer (`@tstring <https://github.com/tstringer>`_) for contributing source
+Thank you to Thomas Stringer (`@tstringer <https://github.com/tstringer>`_) for contributing source
 material for this topic.

docs/docsite/rst/dev_guide/developing_modules_general_windows.rst

@@ -38,52 +38,17 @@ When creating a new module there are a few things to keep in mind:
 - Use the full cmdlet name instead of aliases, e.g. ``Remove-Item`` over ``rm``
 - Use named parameters with cmdlets, e.g. ``Remove-Item -Path C:\temp`` over ``Remove-Item C:\temp``
 
-A very basic powershell module template can be found found below:
+A very basic powershell module `win_environment <https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/windows/win_environment.ps1>`_ is included below. It demonstrates how to implement check-mode and diff-support, and also shows a warning to the user when a specific condition is met.
 
-.. code-block:: powershell
+.. .. include:: ../../../../lib/ansible/modules/windows/win_environment.ps1
+..    :code: powershell
 
-    #!powershell
-    # This file is part of Ansible
+.. literalinclude:: ../../../../lib/ansible/modules/windows/win_environment.ps1
+   :language: powershell
 
-    # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+A slightly more advanced module is `win_uri <https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/windows/win_uri.ps1>`_ which additionally shows how to use different parameter types (bool, str, int, list, dict, path) and a selection of choices for parameters, how to fail a module and how to handle exceptions.
 
-    #Requires -Module Ansible.ModuleUtils.Legacy.psm1
-
-    $ErrorActionPreference = 'Stop'
-
-    $params = Parse-Args -arguments $args -supports_check_mode $true
-    $check_mode = Get-AnsibleParam -obj $params -name "_ansible_check_mode" -type "bool" -default $false
-    $diff_mode = Get-AnsibleParam -obj $params -name "_ansible_diff" -type "bool" -default $false
-
-    # these are your module parameters, there are various types which can be
-    # used to format your parameters. You can also set mandatory parameters
-    # with -failifempty, set defaults with -default and set choices with
-    # -validateset.
-    $string = Get-AnsibleParam -obj $params -name "string" -type "str" -failifempty $true
-    $bool = Get-AnsibleParam -obj $params -name "bool" -type "bool" -default $false
-    $int = Get-AnsibleParam -obj $params -name "int" -type "int"
-    $path = Get-AnsibleParam -obj $params -name "path" -type "path"
-    $list = Get-AnsibleParam -obj $params -name "list" -type "list"
-    $choices = Get-AnsibleParam -obj $params -name "choices" -type "str" -default "present" -validateset "absent","present"
-
-    $result = @{
-        changed = $false
-    }
-
-    if ($diff_mode) {
-        $result.diff = @{}
-    }
-
-    # code goes here
-
-    # you can add/set new result objects with
-    $result.changed = $true
-    $result.new_result = "Hi"
-
-    Exit-Json -obj $result
-
-
-When in doubt, look at some of the core modules and see how things have been
+When in doubt, look at some of the other core modules and see how things have been
 implemented there.
 
 Sometimes there are multiple ways that Windows offers to complete a task; this

docs/docsite/rst/dev_guide/developing_modules_in_groups.rst

@@ -26,7 +26,7 @@ Although it's tempting to get straight into coding, there are a few things to be
 * Shared code can be placed into ``lib/ansible/module_utils/``
 * Shared documentation (for example describing common arguments) can be placed in ``lib/ansible/utils/module_docs_fragments/``.
 * With great power comes great responsibility: Ansible module maintainers have a duty to help keep modules up to date. As with all successful community projects, module maintainers should keep a watchful eye for reported issues and contributions.
-* Although not required, unit and/or integration tests are strongly recommended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :doc:`testing` and the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
+* Although not required, unit and/or integration tests are strongly recommended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :doc:`dev_guide/testing` and the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
   * Starting with Ansible 2.4 all :doc:`../list_of_network_modules` MUST have unit tests.
 
 

docs/docsite/rst/dev_guide/developing_plugins.rst

@@ -3,23 +3,50 @@ Developing Plugins
 
 .. contents:: Topics
 
-Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own.
+Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own. This section describes the various types of Ansible  plugins and how to implement them.
 
-The following types of plugins are available:
+.. _plugin_guidelines:
 
-- *Action* plugins are front ends to modules and can execute actions on the controller before calling the modules themselves.
-- *Cache* plugins are used to keep a cache of 'facts' to avoid costly fact-gathering operations.
-- *Callback* plugins enable you to hook into Ansible events for display or logging purposes.
-- *Connection* plugins define how to communicate with inventory hosts.
-- *Filters* plugins allow you to manipulate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra filter plugins.
-- *Lookup* plugins are used to pull data from an external source. These are implemented using a custom Jinja2 function.
-- *Strategy* plugins control the flow of a play and execution logic.
-- *Shell* plugins deal with low-level commands and formatting for the different shells Ansible can encounter on remote hosts.
-- *Test* plugins allow you to validate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra test plugins.
-- *Vars* plugins inject additional variable data into Ansible runs that did not come from an inventory, playbook, or the command line.
+General Guidelines
+------------------
 
-This section describes the various types of plugins and how to implement them.
+This section lists some things that should apply to any type of plugin you develop.
 
+Raising Errors
+``````````````
+
+In general, errors encountered during execution should be returned by raising AnsibleError() or similar class with a message describing the error. When wrapping other exceptions into error messages, you should always use the `to_text` Ansible function to ensure proper string compatibility across Python versions:
+
+.. code-block:: python
+
+    from ansible.module_utils._text import to_native
+
+    try:
+        cause_an_exception()
+    except Exception as e:
+        AnsibleError('Something happened, this was original exception: %s' % to_native(e))
+
+Check the different AnsibleError objects and see which one applies the best to your situation.
+
+String Encoding
+```````````````
+Any strings returned by your plugin that could ever contain non-ASCII characters must be converted into Python's unicode type because the strings will be run through jinja2.  To do this, you can use:
+
+.. code-block:: python
+
+    from ansible.module_utils._text import to_text
+    result_string = to_text(result_string)
+
+Plugin Configuration
+````````````````````
+
+Starting with Ansible version 2.4, we are unifying how each plugin type is configured and how they get those settings.  Plugins will be able to declare their requirements and have Ansible provide them with a resolved'configuration. Starting with Ansible 2.4 both callback and connection type plugins can use this system. 
+
+Most plugins will be able to use  `self._options[<optionname>]` to access the settings, except callbacks that use `self._plugin_options[<optionname>]`.
+
+Plugins that support embedded documentation (see `ansible-doc` for the list) are now required to provide well-formed doc strings to be considered for merge into the Ansible repo.
+
+If you inherit from a plugin, you must document the options it takes, either via a documentation fragment or as a copy.
 
 .. _developing_callbacks:
 
@@ -28,75 +55,50 @@ Callback Plugins
 
 Callback plugins enable adding new behaviors to Ansible when responding to events. By default, callback plugins control most of the output you see when running the command line programs.
 
-.. _callback_examples:
-
-Example Callback Plugins
-++++++++++++++++++++++++
-
-Ansible comes with a number of callback plugins that you can look at for examples. These can be found in `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_.
-
-The `log_plays
-<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/log_plays.py>`_
-callback is an example of how to intercept playbook events to a log
-file, and the `mail
-<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/mail.py>`_
-callback sends email when playbooks complete.
-
-The `osx_say
-<https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/callback/osx_say.py>`_
-callback provided is particularly entertaining -- it will respond with
-computer synthesized speech on OS X in relation to playbook events,
-and is guaranteed to entertain and/or annoy coworkers.
-
-.. _configuring_callbacks:
-
-Configuring Callback Plugins
-++++++++++++++++++++++++++++
-
-You can activate a custom callback by either dropping it into a callback_plugins directory adjacent to your play or inside a role or by putting it in one of the callback directory sources configured in `ansible.cfg`.
-
-Plugins are loaded in alphanumeric order; for example, a plugin implemented in a file named `1_first.py` would run before a plugin file named `2_second.py`.
-
-Most callbacks shipped with Ansible are disabled by default and need to be whitelisted in your `ansible.cfg` file in order to function. For example::
-
-  #callback_whitelist = timer, mail, mycallbackplugin
-
-
-Managing stdout
-```````````````
-
-You can only have one plugin be the main manager of your console output. If you want to replace the default, you should define CALLBACK_TYPE = stdout in the subclass and then configure the stdout plugin in `ansible.cfg`. For example::
-
-  #stdout_callback = mycallbackplugin
-
-
-
-.. _callback_development:
-
-Developing Callback Plugins
-+++++++++++++++++++++++++++
-
 Callback plugins are created by creating a new class with the Base(Callbacks) class as the parent:
 
 .. code-block:: python
 
   from ansible.plugins.callback import CallbackBase
-  from ansible import constants as C
 
   class CallbackModule(CallbackBase):
       pass
 
-From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`. For a complete list of methods that you can override, please see ``__init__.py`` in the `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_ directory.
+From there, override the specific methods from the CallbackBase that you want to provide a callback for.
+For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`.
+For a complete list of methods that you can override, please see ``__init__.py`` in the
+`lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_ directory.
 
 
-The following example shows how Ansible's timer plugin is implemented:
+The following is a modified example of how Ansible's timer plugin is implemented,
+but with an extra option so you can see how configuration works in Ansible version 2.4 and later:
 
 .. code-block:: python
 
-  # Make coding more python3-ish
+  # Make coding more python3-ish, this is required for contributions to Ansible
   from __future__ import (absolute_import, division, print_function)
   __metaclass__ = type
 
+  # not only visible to ansible-doc, it also 'declares' the options the plugin requires and how to configure them.
+  DOCUMENTATION = '''
+    callback: timer
+    callback_type: aggregate
+    requirements:
+      - whitelist in configuration
+    short_description: Adds time to play stats
+    version_added: "2.0"
+    description:
+        - This callback just adds total play duration to the play stats.
+    options:
+      format_string:
+        description: format of the string shown to user at play end
+        ini:
+          - section: callback_timer
+            key: format_string
+        env:
+          - name: ANSIBLE_CALLBACK_TIMER_FORMAT
+        default: "Playbook run took %s days, %s hours, %s minutes, %s seconds"
+  '''
   from datetime import datetime
 
   from ansible.plugins.callback import CallbackBase
@@ -109,40 +111,86 @@ The following example shows how Ansible's timer plugin is implemented:
       CALLBACK_VERSION = 2.0
       CALLBACK_TYPE = 'aggregate'
       CALLBACK_NAME = 'timer'
+
+      # only needed if you ship it and don't want to enable by default
       CALLBACK_NEEDS_WHITELIST = True
 
       def __init__(self):
 
+          # make sure the expected objects are present, calling the base's __init__
           super(CallbackModule, self).__init__()
 
+          # start the timer when the plugin is loaded, the first play should start a few milliseconds after.
           self.start_time = datetime.now()
 
-      def days_hours_minutes_seconds(self, runtime):
+      def _days_hours_minutes_seconds(self, runtime):
+          ''' internal helper method for this callback '''
           minutes = (runtime.seconds // 60) % 60
           r_seconds = runtime.seconds - (minutes * 60)
           return runtime.days, runtime.seconds // 3600, minutes, r_seconds
 
-      def playbook_on_stats(self, stats):
-          self.v2_playbook_on_stats(stats)
-
+      # this is only event we care about for display, when the play shows its summary stats; the rest are ignored by the base class
       def v2_playbook_on_stats(self, stats):
           end_time = datetime.now()
           runtime = end_time - self.start_time
-          self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds" % (self.days_hours_minutes_seconds(runtime)))
 
-Note that the CALLBACK_VERSION and CALLBACK_NAME definitions are required for properly functioning plugins for Ansible >=2.0.
+          # Shows the usage of a config option declared in the DOCUMENTATION variable. Ansible will have set it when it loads the plugin.
+          # Also note the use of the display object to print to screen. This is available to all callbacks, and you should use this over printing yourself
+          self._display.display(self._plugin_options['format_string'] % (self._days_hours_minutes_seconds(runtime)))
+
+Note that the CALLBACK_VERSION and CALLBACK_NAME definitions are required for properly functioning plugins for Ansible version 2.0 and later. CALLBACK_TYPE is mostly needed to distinguish 'stdout' plugins from the rest, since you can only load one plugin that writes to stdout.
 
 .. _developing_connection_plugins:
 
 Connection Plugins
 ------------------
 
-By default, Ansible ships with a 'paramiko' SSH, native ssh (just called 'ssh'), 'local' connection type, and there are also some minor players like 'chroot' and 'jail'.  All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines.  The basics of these connection types
-are covered in the :doc:`../intro_getting_started` section.  Should you want to extend Ansible to support other transports (SNMP, Message bus, etc) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins
-directory.   The value of 'smart' for a connection allows selection of paramiko or openssh based on system capabilities, and chooses
-'ssh' if OpenSSH supports ControlPersist, in Ansible 1.2.1 and later.  Previous versions did not support 'smart'.
+Connection plugins allow Ansible to connect to the target hosts so it can execute tasks on them. Ansible ships with many connection plugins, but only one can be used per host at a time.
 
-More documentation on writing connection plugins is pending, though you can jump into `lib/ansible/plugins/connection <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/connection>`_ and figure things out pretty easily.
+By default, Ansible ships with several plugins. The most commonly used are the 'paramiko' SSH, native ssh (just called 'ssh'), and 'local' connection types.  All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines.  
+
+The basics of these connection types are covered in the :doc:`../intro_getting_started` section.  
+
+Should you want to extend Ansible to support other transports (SNMP, Message bus, etc) it's as simple as copying the format of one of the existing modules and dropping it into the connection plugins directory.  
+
+Ansible version 2.1 introduced the 'smart' connection plugin. The 'smart' connection type allows Ansible to automatically select either the 'paramiko' or 'openssh' connection plugin based on system capabilities, or the 'ssh' connection plugin if OpenSSH supports ControlPersist. 
+
+For examples on how to implement a connection plug in, see the source code here: 
+`lib/ansible/plugins/connection <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/connection>`_.
+
+.. _developing_inventory_plugins:
+
+Inventory Plugins
+-----------------
+
+Inventory plugins were added in Ansible version 2.4. Inventory plugins parse inventory sources and form an in memory representation of the inventory.
+
+Inventory plugins are invoked via the InventoryManager and are given access to any existing inventory data. They are given an 'inventory source' as supplied to Ansible (via config/options/defaults/etc), which they can either ignore
+by returning false from the `verify_file` method, or attempting to parse (with the `parse` method) and return an `AnsibleParserError` on failure.
+
+.. code-block:: python
+
+   def parse(self, inventory, loader, path, cache=True):
+        pass # your code goes here
+
+Inventory plugins take the following parameters:
+
+ * inventory: inventory object with existing data and the methods to add hosts/groups/variables to inventory
+ * loader: Ansible's DataLoader. The DataLoader can read files, auto load JSON/YAML and decrypt vaulted data, and cache read files.
+ * path: string with inventory source (this is usually a path, but is not required)
+ * cache: indicates whether the plugin should use or avoid caches (cache plugin and/or loader)
+
+Inventory sources are strings. They usually correspond to a file path, but they can also be a comma separated list,
+a URI, or anything your plugin can use as input.
+The 'inventory source' provided can be either a string (`host_list` plugin), a data file (like consumed by the `yaml` and `ini` plugins), a configuration file (see `virtualbox` and `constructed`) or even a script or executable (the `script` uses those).
+
+When using the 'persistent' cache, inventory plugins can also use the configured cache plugin to store and retrieve data to avoid costly external calls. 
+
+Inventory plugins normally only execute at the start of a run, before playbooks/plays and roles are found,
+but they can be 're-executed' via the `meta: refresh_inventory` task, which will clear out the existing inventory and rebuild it.
+
+For examples on how to implement a connection plug in, see the source code here:
+`lib/ansible/plugins/inventory <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/inventory>`_.
 
 .. _developing_lookup_plugins:
 
@@ -155,6 +203,25 @@ Here's a simple lookup plugin implementation - this lookup returns the contents
 
 .. code-block:: python
 
+  # python 3 headers, required if submitting to Ansible
+  from __future__ import (absolute_import, division, print_function)
+  __metaclass__ = type
+
+  DOCUMENTATION = """
+        lookup: file
+          author: Daniel Hokka Zakrisson <daniel@hozac.com>
+          version_added: "0.9"
+          short_description: read file contents
+          description:
+              - This lookup returns the contents from a file on the Ansible controller's file system.
+          options:
+            _terms:
+              description: path(s) of files to read
+              required: True
+          notes:
+            - if read in variable context, the file can be interpreted as YAML if the content is valid to the parser.
+            - this lookup does not understand 'globing' - use the fileglob lookup instead.
+  """
   from ansible.errors import AnsibleError, AnsibleParserError
   from ansible.plugins.lookup import LookupBase
 
@@ -169,26 +236,35 @@ Here's a simple lookup plugin implementation - this lookup returns the contents
 
       def run(self, terms, variables=None, **kwargs):
 
-          ret = []
 
+          # lookups in general are expected to both take a list as input and output a list
+          # this is done so they work with the looping construct `with_`.
+          ret = []
           for term in terms:
               display.debug("File lookup term: %s" % term)
 
-              # Find the file in the expected search path
+              # Find the file in the expected search path, using a class method
+              # that implements the 'expected' search path for Ansible plugins.
               lookupfile = self.find_file_in_search_path(variables, 'files', term)
+
+              # Don't use print or your own logging, the display class
+              # takes care of it in a unified way.
               display.vvvv(u"File lookup using %s as file" % lookupfile)
               try:
                   if lookupfile:
                       contents, show_data = self._loader._get_file_contents(lookupfile)
                       ret.append(contents.rstrip())
                   else:
+                      # Always use ansible error classes to throw 'final' exceptions,
+                      # so the Ansible engine will know how to deal with them.
+                      # The Parser error indicates invalid options passed
                       raise AnsibleParserError()
               except AnsibleParserError:
                   raise AnsibleError("could not locate file in lookup: %s" % term)
 
           return ret
 
-An example of how this lookup is called::
+The following is an example of how this lookup is called::
 
   ---
   - hosts: all
@@ -199,30 +275,48 @@ An example of how this lookup is called::
 
        - debug: msg="the value of foo.txt is {{ contents }} as seen today {{ lookup('pipe', 'date +"%Y-%m-%d"') }}"
 
-Errors encountered during execution should be returned by raising AnsibleError() with a message describing the error. Any strings returned by your lookup plugin implementation that could ever contain non-ASCII characters must be converted into Python's unicode type because the strings will be run through jinja2.  To do this, you can use:
-
-.. code-block:: python
-
-    from ansible.module_utils._text import to_text
-    result_string = to_text(result_string)
-
 For more example lookup plugins, check out the source code for the lookup plugins that are included with Ansible here: `lib/ansible/plugins/lookup <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/lookup>`_.
 
-For usage examples of lookup plugins, see `Using Lookups <http://docs.ansible.com/ansible/playbooks_lookups.html>`_.
+For more usage examples of lookup plugins, see `Using Lookups <http://docs.ansible.com/ansible/playbooks_lookups.html>`_.
 
 .. _developing_vars_plugins:
 
 Vars Plugins
 ------------
 
-Playbook constructs like 'host_vars' and 'group_vars' work via 'vars' plugins.  They inject additional variable
-data into ansible runs that did not come from an inventory, playbook, or command line.  Note that variables
-can also be returned from inventory, so in most cases, you won't need to write or understand vars_plugins.
+Vars plugins inject additional variable data into Ansible runs that did not come from an inventory source, playbook, or command line. Playbook constructs like 'host_vars' and 'group_vars' work using vars plugins.
 
-More documentation on writing vars plugins is pending, though you can jump into `lib/ansible/plugins <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/>`_ and figure
-things out pretty easily.
+Vars plugins were partially implemented in Ansible 2.0 and rewritten to be fully implemented starting with Ansible 2.4.
+
+Older plugins used a `run` method as their main body/work:
+
+.. code-block:: python
+
+    def run(self, name, vault_password=None):
+        pass # your code goes here
+
+
+Ansible 2.0 did not pass passwords to older plugins, so vaults were unavailable.
+Most of the work now  happens in the `get_vars` method which is called from the VariableManager when needed.
+
+.. code-block:: python
+
+    def get_vars(self, loader, path, entities):
+        pass # your code goes here
+
+The parameters are:
+
+ * loader: Ansible's DataLoader. The DataLoader can read files, auto load JSON/YAML and decrypt vaulted data, and cache read files.
+ * path: this is 'directory data' for every inventory source and the current play's playbook directory, so they can search for data in reference to them. `get_vars` will be called at least once per available path.
+ * entities: these are host or group names that are pertinent to the variables needed. The plugin will get called once for hosts and again for groups.
+
+This `get vars` method just needs to return a dictionary structure with the variables.
+
+Since Ansible version 2.4, vars plugins only execute as needed when preparing to execute a task. This avoids the costly 'always execute' behavior that occurred during inventory construction in older versions of Ansible.
+
+For implementation examples of vars plugins, check out the source code for the vars plugins that are included with Ansible:
+`lib/ansible/plugins/vars <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/vars>`_  .
 
-If you find yourself wanting to write a vars_plugin, it's more likely you should write an inventory script instead.
 
 .. _developing_filter_plugins:
 
@@ -255,11 +349,14 @@ Plugins are automatically loaded when you have one of the following subfolders a
     * lookup_plugins
     * callback_plugins
     * connection_plugins
+    * inventory_plugins
     * filter_plugins
     * strategy_plugins
     * cache_plugins
     * test_plugins
     * shell_plugins
+    * vars_plugins
+
 
 When shipped as part of a role, the plugin will be available as soon as the role is called in the play.
 

docs/docsite/rst/dev_guide/developing_python3.rst

@@ -295,7 +295,7 @@ new exception-catching syntax which uses the ``as`` keyword:
 Do **not** use the following syntax as it will fail on every version of Python-3:
 
 .. This code block won't highlight because python2 isn't recognized. This is necessary to pass tests under python 3.
-.. code-block:: python2
+.. code-block:: none
 
     try:
         a = 2/0

docs/docsite/rst/dev_guide/index.rst

@@ -30,7 +30,6 @@ To get started, select one of the following topics.
    developing_core
    developing_python3
    developing_api
-   developing_test_pr
    developing_rebasing
    testing
    repomerge

docs/docsite/rst/dev_guide/overview_architecture.rst

@@ -25,7 +25,7 @@ Inventory
 
 By default, Ansible represents what machines it manages using a very simple INI file that puts all of your managed machines in groups of your own choosing.
 
-To add new machines, there is no additional SSL signing server involved, so there's never any hassle deciding why a particular machine didn’t get linked up due to obscure NTP or DNS issues.
+To add new machines, there is no additional SSL signing server involved, so there's never any hassle deciding why a particular machine didn't get linked up due to obscure NTP or DNS issues.
 
 If there's another source of truth in your infrastructure, Ansible can also plugin to that, such as drawing inventory, group, and variable information from sources like EC2, Rackspace, OpenStack, and more.
 
@@ -69,4 +69,4 @@ Here's what a simple playbook looks like::
 Extending Ansible with Plug-ins and the API
 ````````````````````````````````````````````
 
-Should you want to write your own, Ansible modules can be written in any language that can return JSON (Ruby, Python, bash, etc). Inventory can also plug in to any datasource by writing a program that speaks to that datasource and returns JSON. There's also various Python APIs for extending Ansible’s connection types (SSH is not the only transport possible), callbacks (how Ansible logs, etc), and even for adding new server side behaviors.
+Should you want to write your own, Ansible modules can be written in any language that can return JSON (Ruby, Python, bash, etc). Inventory can also plug in to any datasource by writing a program that speaks to that datasource and returns JSON. There's also various Python APIs for extending Ansible's connection types (SSH is not the only transport possible), callbacks (how Ansible logs, etc), and even for adding new server side behaviors.

docs/docsite/rst/dev_guide/repomerge.rst

@@ -27,19 +27,3 @@ As part of this move, we will be introducing module metadata, which will contain
 
 
 The documentation pages for modules will be updated to reflect the above information as well, so that users can evaluate the status of a module before committing to using it in playbooks and roles.
-
-
-.. _PRMover:
-
-Move Issues and PRs to new Repo
--------------------------------
-A tool has been developed to move a PR from the old repos to `ansible/ansible` this can be found at `prmover tool <https://prmover.pythonanywhere.com/>`_ 
-
-Before using prmover please ensure you have a fork of the Ansible repo.
-
-To move issues please use `GitHub Issue Mover <https://github-issue-mover.appspot.com/>`_
-
-If you have *any* issues with updating your PR please ask for support in `#ansible-devel`
-
-For support please use `#ansible-devel` on Freenode IRC
-

docs/docsite/rst/dev_guide/style_guide/grammar_punctuation.rst

@@ -55,7 +55,7 @@ Commas are always important, considering the vast difference in meanings of the
 - Let's eat, Grandma
 - Let's eat Grandma.
 
-Correct punctation could save Grandma's life.
+Correct punctuation could save Grandma's life.
 
 If that does not convince you, maybe this will:
 
@@ -110,7 +110,7 @@ Never use "we" when writing. "We" aren't doing anything on the user side. Ansibl
 
 Hyphen
 ~~~~~~~~~~~~~~
-The hyphen’s primary function is the formation of certain compound terms. Do not use a hyphen unless it serves a purpose. If a compound adjective cannot be misread or, as with many psychological terms, its meaning is established, a hyphen is not necessary.
+The hyphen's primary function is the formation of certain compound terms. Do not use a hyphen unless it serves a purpose. If a compound adjective cannot be misread or, as with many psychological terms, its meaning is established, a hyphen is not necessary.
 
 Use hyphens to avoid ambiguity or confusion:
 

docs/docsite/rst/dev_guide/style_guide/resources.rst

@@ -2,17 +2,15 @@ Resources
 ````````````````
 Internal resources
 -------------------------
-- http://docs.ansible.com/
+- https://docs.ansible.com/
 - https://sites.google.com/a/ansibleworks.com/ansible-intranet/
 - ??? intranet Engineering doc???
 
 
 External Resources
 -------------------------
-- www.apstylebook.com
-- www.chicagomanualofstyle.org—home.html
-- www.crockford.com—style.html
-- orwell.ru—index.htm
-- www.sun.com—sun_tech_pub.xml
-- webopedia.internet.com
-- www.computeruser.com—index.html
+- https://www.apstylebook.com
+- http://www.chicagomanualofstyle.org/home.html
+- https://www.crockford.com/wrrrld/style.html
+- http://orwell.ru
+- http://www.computeruser.com

docs/docsite/rst/dev_guide/style_guide/spelling_word_choice.rst

@@ -78,7 +78,7 @@ Use to describes where to place options for a command, but not where to type the
 Daylight saving time (DST)
 +++++++++++++++++++++++++++++++
 
-Correct. Do not use daylight savings time. Daylight Saving Time (DST) is often misspelled “Daylight Savings”, with an “s” at the end. Other common variations are “Summer Time”and “Daylight-Saving Time”. (http://www.timeanddate.com/time/dst/daylight-savings-time.html)
+Correct. Do not use daylight savings time. Daylight Saving Time (DST) is often misspelled "Daylight Savings", with an "s" at the end. Other common variations are "Summer Time"and "Daylight-Saving Time". (http://www.timeanddate.com/time/dst/daylight-savings-time.html)
 
 
 Download
@@ -99,7 +99,7 @@ When used as a verb, fail over is two words since there can be different tenses
 
 Fewer
 +++++++++++++++++++
-Fewer is used with plural nouns. Think things you could count.  Time, money, distance, and weight are often listed as exceptions to the traditional “can you count it” rule, often thought of a singular amounts (the work will take less than 5 hours, for example).
+Fewer is used with plural nouns. Think things you could count.  Time, money, distance, and weight are often listed as exceptions to the traditional "can you count it" rule, often thought of a singular amounts (the work will take less than 5 hours, for example).
 
 File name
 +++++++++++++
@@ -250,7 +250,7 @@ Correct. Do not use "trouble shoot" or "trouble-shoot." To isolate the source of
 
 UK
 ++++++++++++++
-Correst as is, no periods.
+Correct as is, no periods.
 
 UNIX®
 ++++++++++++++
@@ -262,7 +262,7 @@ Don't use. Use Clear.
 
 US
 ++++++++++++++
-Correst as is, no periods.
+Correct as is, no periods.
 
 User
 ++++++++++++++

docs/docsite/rst/dev_guide/style_guide/trademarks.rst

@@ -87,10 +87,10 @@ Other Common Trademarks and Resource Sites:
 ++++++++++++++++++++++++++++++++++++++++++++++++
 - Linux is a registered trademark of Linus Torvalds.
 - UNIX® is a registered trademark of The Open Group.
-- Microsoft, Windows, Vista, XP, and NT are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. http://members.microsoft.com—en-us.mspx
-- Apple, Mac, Mac OS, Macintosh, Pages and TrueType are either registered trademarks or trademarks of Apple Computer, Inc. in the United States and/or other countries. http://www.apple.com—appletmlist.html
-- Adobe, Acrobat, GoLive, InDesign, Illustrator, PostScript , PhotoShop and the OpenType logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. htto:// www.adobe.com—trade.html
-- Macromedia and Macromedia Flash are trademarks of Macromedia, Inc. http://www.adobe.com—trademarkguideline.html
-- IBM is a registered trademark of International Business Machines Corporation. http://www.ibm.com—copytrade.shtml
-- Celeron, Celeron Inside, Centrino, Centrino logo, Core Inside, Intel Core, Intel Inside, Intel Inside logo, Itanium, Itanium Inside, Pentium, Pentium Inside,VTune, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. http://www.intel.com—tradmarx.htm
+- Microsoft, Windows, Vista, XP, and NT are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/en-us.aspx
+- Apple, Mac, Mac OS, Macintosh, Pages and TrueType are either registered trademarks or trademarks of Apple Computer, Inc. in the United States and/or other countries. https://www.apple.com/legal/intellectual-property/trademark/appletmlist.html
+- Adobe, Acrobat, GoLive, InDesign, Illustrator, PostScript , PhotoShop and the OpenType logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. http://www.adobe.com/legal/permissions/trademarks.html
+- Macromedia and Macromedia Flash are trademarks of Macromedia, Inc. http://www.adobe.com/legal/permissions/trademarks.html
+- IBM is a registered trademark of International Business Machines Corporation. https://www.ibm.com/legal/us/en/copytrade.shtml
+- Celeron, Celeron Inside, Centrino, Centrino logo, Core Inside, Intel Core, Intel Inside, Intel Inside logo, Itanium, Itanium Inside, Pentium, Pentium Inside,VTune, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. https://www.intel.com/content/www/us/en/legal/trademarks.html
 

docs/docsite/rst/dev_guide/style_guide/why_use.rst

@@ -13,8 +13,8 @@ This style guide incorporates current Ansible resources and information so that
 
    <blockquote class="note info">
 
-   “If you don't find it in the index, look very carefully through the entire catalogue.” 
-    ― Sears, Roebuck and Co., 1897 Sears Roebuck & Co. Catalogue 
+   "If you don't find it in the index, look very carefully through the entire catalogue."
+    ― Sears, Roebuck and Co., 1897 Sears Roebuck & Co. Catalogue
  
 .. raw:: html
 

docs/docsite/rst/dev_guide/testing/sanity/ansible-var-precedence-check.rst

@@ -1,4 +1,4 @@
 Sanity Tests » ansible-var-precedence-check
 ===========================================
 
-Check the order of precedence for Ansible variables against :ref:`variable_precedence`.
+Check the order of precedence for Ansible variables against :ref:`ansible_variable_precedence`.

docs/docsite/rst/dev_guide/testing/sanity/no-smart-quotes.rst

@@ -0,0 +1,4 @@
+Sanity Tests » no-smart-quotes
+==============================
+
+Smart quotes (``”“‘’``) should not be used.  Use plain ascii quotes (``"'``) instead.

docs/docsite/rst/dev_guide/testing/sanity/no-unicode-literals.rst

@@ -0,0 +1,16 @@
+Sanity Tests » no-unicode_literals
+==================================
+
+The use of :code:`from __future__ import unicode_literals` has been deemed an anti-pattern.  The
+problems with it are:
+
+* It makes it so one can't jump into the middle of a file and know whether a bare literal string is
+  a byte string or text string.  The programmer has to first check the top of the file to see if the
+  import is there.
+* It removes the ability to define native strings (a string which should be a byte string on python2
+  and a text string on python3) via a string literal.
+* It makes for more context switching.  A programmer could be reading one file which has
+  `unicode_literals` and know that bare string literals are text strings but then switch to another
+  file (perhaps tracing program execution into a third party library) and have to switch their
+  understanding of what bare string literals are.
+

docs/docsite/rst/faq.rst

@@ -56,7 +56,7 @@ by default if OpenSSH is new enough to support ControlPersist as an option.
 
 Paramiko is great for starting out, but the OpenSSH type offers many advanced options.  You will want to run Ansible
 from a machine new enough to support ControlPersist, if you are using this connection type.  You can still manage
-older clients.  If you are using RHEL 6, CentOS 6, SLES 10 or SLES 11 the version of OpenSSH is still a bit old, so 
+older clients.  If you are using RHEL 6, CentOS 6, SLES 10 or SLES 11 the version of OpenSSH is still a bit old, so
 consider managing from a Fedora or openSUSE client even though you are managing older nodes, or just use paramiko.
 
 We keep paramiko as the default as if you are first installing Ansible on an EL box, it offers a better experience
@@ -116,7 +116,7 @@ By default, Ansible assumes it can find a /usr/bin/python on your remote system
 
 Setting the inventory variable 'ansible_python_interpreter' on any host will allow Ansible to auto-replace the interpreter
 used when executing python modules.   Thus, you can point to any python you want on the system if /usr/bin/python on your
-system does not point to a Python 2.X interpreter.  
+system does not point to a Python 2.X interpreter.
 
 Some Linux operating systems, such as Arch, may only have Python 3 installed by default.  This is not sufficient and you will
 get syntax errors trying to run modules with Python 3.  Python 3 is essentially not the same language as Python 2.  Python 3
@@ -305,6 +305,20 @@ You shouldn't put plaintext passwords in your playbook or host_vars; instead, us
 
 .. _commercial_support:
 
+Ansible supports dot notation and array notation for variables. Which notation should I use?
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+The dot notation comes from Jinja and works fine for variables without special
+characters. If your variable contains dots (.), colons (:), or dashes (-) it is
+safer to use the array notation for variables.
+
+.. code-block:: jinja
+
+    item[0]['checksum:md5']
+    item['section']['2.1']
+    item['region']['Mid-Atlantic']
+    It is {{ temperature['Celsius']['-3'] }} outside.
+
 Can I get training on Ansible?
 ++++++++++++++++++++++++++++++
 
@@ -407,6 +421,3 @@ Please see the section below for a link to IRC and the Google Group, where you c
        Have a question?  Stop by the google group!
    `irc.freenode.net <http://irc.freenode.net>`_
        #ansible IRC chat channel
-
-
-

docs/docsite/rst/galaxy.rst

@@ -152,7 +152,7 @@ Content of the *requirements.yml* file:
     # from galaxy
     - src: yatesr.timezone
 
-    - include: <path_to_requirements>/webserver.yml
+    - import_tasks: <path_to_requirements>/webserver.yml
 
 
 Content of the *webserver.yml* file:

docs/docsite/rst/glossary.rst

@@ -436,7 +436,7 @@ when a term comes up on the mailing list.
         labeled ``ntp``, and then run just the ``ntp`` steps to reconfigure
         the time server information on a remote host.
 
-    Tasks
+    Task
         :term:`Playbooks` exist to run tasks.  Tasks combine an :term:`action`
         (a module and its arguments) with a name and optionally some other
         keywords (like :term:`looping directives <loops>`).   :term:`Handlers`
@@ -444,6 +444,9 @@ when a term comes up on the mailing list.
         unless they are notified by name when a task reports an underlying
         change on a remote system.
 
+    Tasks
+        A list of :term:`Task`.
+
     Templates
         Ansible can easily transfer files to remote systems but often it is
         desirable to substitute variables in other files.  Variables may come

docs/docsite/rst/guide_azure.rst

@@ -40,11 +40,11 @@ There is now a detailed official tutorial describing `how to create a service pr
 
 After stepping through the tutorial you will have:
 
-* Your Client ID, which is found in the “client id” box in the “Configure” page of your application in the Azure portal
+* Your Client ID, which is found in the "client id" box in the "Configure" page of your application in the Azure portal
 * Your Secret key, generated when you created the application. You cannot show the key after creation.
-  If you lost the key, you must create a new one in the “Configure” page of your application.
-* And finally, a tenant ID. It’s a UUID (e.g. ABCDEFGH-1234-ABCD-1234-ABCDEFGHIJKL) pointing to the AD containing your
-  application. You will find it in the URL from within the Azure portal, or in the “view endpoints” of any given URL.
+  If you lost the key, you must create a new one in the "Configure" page of your application.
+* And finally, a tenant ID. It's a UUID (e.g. ABCDEFGH-1234-ABCD-1234-ABCDEFGHIJKL) pointing to the AD containing your
+  application. You will find it in the URL from within the Azure portal, or in the "view endpoints" of any given URL.
 
 
 Using Active Directory Username/Password

docs/docsite/rst/index.rst

@@ -9,7 +9,7 @@ Welcome to the Ansible documentation!
 Ansible is an IT automation tool.  It can configure systems, deploy software, and orchestrate more advanced IT tasks
 such as continuous deployments or zero downtime rolling updates.
 
-Ansible's main goals are simplicity and ease-of-use. It also has a strong focus on security and reliability, featuring a minimum of moving parts, usage of OpenSSH for transport (with an accelerated socket mode and pull modes as alternatives), and a language that is designed around auditability by humans--even those not familiar with the program.
+Ansible's main goals are simplicity and ease-of-use. It also has a strong focus on security and reliability, featuring a minimum of moving parts, usage of OpenSSH for transport (with other transports and pull modes as alternatives), and a language that is designed around auditability by humans--even those not familiar with the program.
 
 We believe simplicity is relevant to all sizes of environments, so we design for busy users of all types: developers, sysadmins, release engineers, IT managers, and everyone in between. Ansible is appropriate for managing all environments, from small setups with a handful of instances to enterprise environments with many thousands of instances.
 
@@ -32,6 +32,8 @@ Ansible, Inc. releases a new major release of Ansible approximately every two mo
    modules
    modules_by_category
    vault
+   command_line_tools
+   plugins_by_category
    guides
    dev_guide/index
    tower

docs/docsite/rst/intro_adhoc.rst

@@ -3,8 +3,6 @@ Introduction To Ad-Hoc Commands
 
 .. contents:: Topics
 
-.. highlight:: bash
-
 The following examples show how to use `/usr/bin/ansible` for running
 ad hoc tasks. 
 

docs/docsite/rst/intro_bsd.rst

@@ -27,7 +27,7 @@ Bootstrapping BSD
 As mentioned above, you can bootstrap Ansible with the ``raw`` module and remotely install Python on targets. The following example installs Python 2.7 which includes the json library required for full functionality of Ansible.
 On your control machine you can simply execute the following for most versions of FreeBSD::
 
-    ansible -m raw -a “pkg install -y python27” mybsdhost1
+    ansible -m raw -a "pkg install -y python27" mybsdhost1
 
 Once this is done you can now use other Ansible modules apart from the ``raw`` module.
 

docs/docsite/rst/intro_getting_started.rst

@@ -26,7 +26,7 @@ machines over SSH.
 
 By default, Ansible 1.3 and later will try to use native 
 OpenSSH for remote communication when possible.  This enables ControlPersist (a performance feature), Kerberos, and options in ``~/.ssh/config`` such as Jump Host setup.  However, when using Enterprise Linux 6 operating systems as the control machine (Red Hat Enterprise Linux and derivatives such as CentOS), the version of OpenSSH may be too old to support ControlPersist. On these operating systems, Ansible will fallback into using a high-quality Python implementation of
-OpenSSH called 'paramiko'.  If you wish to use features like Kerberized SSH and more, consider using Fedora, OS X, or Ubuntu as your control machine until a newer version of OpenSSH is available for your platform -- or engage 'accelerated mode' in Ansible.  See :doc:`playbooks_acceleration`.
+OpenSSH called 'paramiko'.  If you wish to use features like Kerberized SSH and more, consider using Fedora, OS X, or Ubuntu as your control machine until a newer version of OpenSSH is available for your platform.
 
 In releases up to and including Ansible 1.2, the default was strictly paramiko.  Native SSH had to be explicitly selected with the ``-c`` ssh option or set in the configuration file.
 

docs/docsite/rst/intro_installation.rst

@@ -104,10 +104,10 @@ Installing the Control Machine
 Latest Release Via Yum
 ++++++++++++++++++++++
 
-.. note:: We’ve changed how the Ansible community packages are distributed. 
+.. note:: We've changed how the Ansible community packages are distributed.
   For users of RHEL/CentOS/Scientific Linux version 7, the Ansible community RPM
   package will transition from the EPEL repository to the Extras channel.  There will be no
-  change for version 6 of RHEL/CentOS/Scientific Linux since Extras is not a part of version 6.   
+  change for version 6 of RHEL/CentOS/Scientific Linux since Extras is not a part of version 6.
 
 RPMs for RHEL7 are available from `the Extras channel <https://access.redhat.com/solutions/912213>`_.
 
@@ -115,16 +115,16 @@ RPMs for RHEL6 are available from yum for `EPEL
 <http://fedoraproject.org/wiki/EPEL>`_ 6 and currently supported
 Fedora distributions.
 
-Ansible will also have RPMs/YUM-repo available at `<https://releases.ansible.com/ansible/rpms/`.
+Ansible will also have RPMs/YUM-repo available `here <https://releases.ansible.com/ansible/rpm/>`_.
 
 Ansible version 2.4 can manage earlier operating
 systems that contain Python 2.6 or higher.
 
-You can also build an RPM yourself.  From the root of a checkout or tarball, use the ``make rpm`` command to build an RPM you can distribute and install. 
+You can also build an RPM yourself.  From the root of a checkout or tarball, use the ``make rpm`` command to build an RPM you can distribute and install.
 
 .. code-block:: bash
 
-    $ git clone git://github.com/ansible/ansible.git 
+    $ git clone git://github.com/ansible/ansible.git
     $ cd ./ansible
     $ make rpm
     $ sudo rpm -Uvh ./rpm-build/ansible-*.noarch.rpm

docs/docsite/rst/intro_inventory.rst

@@ -11,7 +11,7 @@ which defaults to being saved in the location ``/etc/ansible/hosts``.
 You can specify a different inventory file using the ``-i <path>`` option on the command line.
 
 Not only is this inventory configurable, but you can also use multiple inventory files at the same time and
-pull inventory from dynamic or cloud sources, as described in :doc:`intro_dynamic_inventory`.
+pull inventory from dynamic or cloud sources or different formats (YAML, ini, etc), as described in :doc:`intro_dynamic_inventory`.
 Introduced in version 2.4, Ansible has inventory plugins to make this flexible and customizable.
 
 .. _inventoryformat:
@@ -38,12 +38,30 @@ For this example, the format for ``/etc/ansible/hosts`` is an INI-like (one of A
 The headings in brackets are group names, which are used in classifying systems
 and deciding what systems you are controlling at what times and for what purpose.
 
+A YAML version would look like:
+
+.. code-block:: yaml
+
+  all:
+    hosts:
+      mail.example.com
+    children:
+      webservers:
+        hosts:
+          foo.example.com:
+          bar.example.com:
+      dbservers:
+        hosts:
+          one.example.com:
+          two.example.com:
+          three.example.com:
+
+
 It is ok to put systems in more than one group, for instance a server could be both a webserver and a dbserver.
 If you do, note that variables will come from all of the groups they are a member of. Variable precedence is detailed in a later chapter.
 
-If you have hosts that run on non-standard SSH ports you can put the port number
-after the hostname with a colon.  Ports listed in your SSH config file won't be used with the `paramiko`
-connection but will be used with the `openssh` connection.
+If you have hosts that run on non-standard SSH ports you can put the port number after the hostname with a colon.
+Ports listed in your SSH config file won't be used with the `paramiko` connection but will be used with the `openssh` connection.
 
 To make things explicit, it is suggested that you set them if things are not running on the default port:
 
@@ -51,20 +69,33 @@ To make things explicit, it is suggested that you set them if things are not run
 
     badwolf.example.com:5309
 
-Suppose you have just static IPs and want to set up some aliases that live in your host file, or you are connecting through tunnels.  You can also describe hosts like this:
+Suppose you have just static IPs and want to set up some aliases that live in your host file, or you are connecting through tunnels.
+You can also describe hosts via variables:
+
+In INI:
 
 .. code-block:: ini
 
     jumper ansible_port=5555 ansible_host=192.0.2.50
 
-In the above example, trying to ansible against the host alias "jumper" (which may not even be a real hostname) will contact 192.0.2.50 on port 5555.  Note that this is using a feature of the inventory file to define some special variables.  Generally speaking this is not the best
-way to define variables that describe your system policy, but we'll share suggestions on doing this later.  We're just getting started.
+In YAML:
 
-.. note:: Values passed in using the ``key=value`` syntax are interpreted as Python literal structure (strings, numbers, tuples, lists, dicts,
-          booleans, None), alternatively as string. For example ``var=FALSE`` would create a string equal to 'FALSE'. Do not rely on types set
-          during definition, always make sure you specify type with a filter when needed when consuming the variable.
+.. code-block:: yaml
 
-Adding a lot of hosts?  If you have a lot of hosts following similar patterns you can do this rather than listing each hostname:
+    hosts:
+      jumper:
+        ansible_port: 5555
+        ansible_host: 192.0.2.50
+
+In the above example, trying to ansible against the host alias "jumper" (which may not even be a real hostname) will contact 192.0.2.50 on port 5555.
+Note that this is using a feature of the inventory file to define some special variables.
+Generally speaking, this is not the best way to define variables that describe your system policy, but we'll share suggestions on doing this later.
+
+.. note:: Values passed in the INI format using the ``key=value`` syntax are not interpreted as Python literal structure
+          (strings, numbers, tuples, lists, dicts, booleans, None), but as a string. For example ``var=FALSE`` would create a string equal to 'FALSE'.
+          Do not rely on types set during definition, always make sure you specify type with a filter when needed when consuming the variable.
+
+If you are adding a lot of hosts following similar patterns, you can do this rather than listing each hostname:
 
 .. code-block:: ini
 
@@ -91,15 +122,14 @@ You can also select the connection type and user on a per host basis:
    other1.example.com     ansible_connection=ssh        ansible_user=mpdehaan
    other2.example.com     ansible_connection=ssh        ansible_user=mdehaan
 
-As mentioned above, setting these in the inventory file is only a shorthand, and we'll discuss how to store them in individual files
-in the 'host_vars' directory a bit later on.
+As mentioned above, setting these in the inventory file is only a shorthand, and we'll discuss how to store them in individual files in the 'host_vars' directory a bit later on.
 
 .. _host_variables:
 
 Host Variables
 ++++++++++++++
 
-As alluded to above, it is easy to assign variables to hosts that will be used later in playbooks:
+As described above, it is easy to assign variables to hosts that will be used later in playbooks:
 
 .. code-block:: ini
 
@@ -112,7 +142,11 @@ As alluded to above, it is easy to assign variables to hosts that will be used l
 Group Variables
 +++++++++++++++
 
-Variables can also be applied to an entire group at once::
+Variables can also be applied to an entire group at once:
+
+The INI way:
+
+.. code-block:: ini
 
    [atlanta]
    host1
@@ -122,14 +156,30 @@ Variables can also be applied to an entire group at once::
    ntp_server=ntp.atlanta.example.com
    proxy=proxy.atlanta.example.com
 
-Be aware that this is only a convenient way to apply variables to multiple hosts at once; even though you can target hosts by group, variables are always flattened to the host level before a play is executed.
+The YAML version:
+
+.. code-block:: yaml
+
+    atlanta:
+      hosts:
+        host1:
+        host2:
+      vars:
+        ntp_server: ntp.atlanta.example.com
+        proxy: proxy.atlanta.example.com
+
+Be aware that this is only a convenient way to apply variables to multiple hosts at once; even though you can target hosts by group, **variables are always flattened to the host level** before a play is executed.
 
 .. _subgroups:
 
 Groups of Groups, and Group Variables
 +++++++++++++++++++++++++++++++++++++
 
-It is also possible to make groups of groups using the ``:children`` suffix. Just like above, you can apply variables using ``:vars``::
+It is also possible to make groups of groups using the ``:children`` suffix in INI or the ``children:`` entry in YAML.
+You can apply variables using ``:vars`` or ``vars:``:
+
+
+.. code-block:: ini
 
    [atlanta]
    host1
@@ -155,11 +205,38 @@ It is also possible to make groups of groups using the ``:children`` suffix. Jus
    southwest
    northwest
 
+.. code-block:: yaml
+
+  all:
+    children:
+      usa:
+        children:
+          southeast:
+            children:
+              atlanta:
+                hosts:
+                  host1:
+                  host2:
+              raleigh:
+                hosts:
+                  host2:
+                  host3:
+            vars:
+              some_server: foo.southeast.example.com
+              halon_system_timeout: 30
+              self_destruct_countdown: 60
+              escape_pods: 2
+         northeast:
+         northwest:
+         southwest:
+
 If you need to store lists or hash data, or prefer to keep host and group specific variables separate from the inventory file, see the next section.
 Child groups have a couple of properties to note:
 
- - First, any host that is member of a child group is automatically a member of the parent group.
- - Second, a child group's variables will have higher precedence (override) a parent group's variables.
+ - Any host that is member of a child group is automatically a member of the parent group.
+ - A child group's variables will have higher precedence (override) a parent group's variables.
+ - Groups can have multiple parents and children, but not circular relationships.
+ - Hosts can also be in multiple groups, but there will only be **one** instance of a host, merging the data from the multiple groups.
 
 .. _default_groups:
 
@@ -168,20 +245,20 @@ Default groups
 
 There are two default groups: ``all`` and ``ungrouped``. ``all`` contains every host.
 ``ungrouped`` contains all hosts that don't have another group aside from ``all``.
+Every host will always belong to at least 2 groups.
+Though ``all`` and ``ungrouped`` are always present, they can be implicit and not appear in group listings like ``group_names``.
 
 .. _splitting_out_vars:
 
 Splitting Out Host and Group Specific Data
 ++++++++++++++++++++++++++++++++++++++++++
 
-The preferred practice in Ansible is actually not to store variables in the main inventory file.
+The preferred practice in Ansible is to not store variables in the main inventory file.
 
-In addition to storing variables directly in the INI file, host
-and group variables can be stored in individual files relative to the
-inventory file.
+In addition to storing variables directly in the inventory file, host and group variables can be stored in individual files relative to the inventory file (not directory, it is always the file).
 
-These variable files are in YAML format. Valid file extensions include '.yml', '.yaml', '.json',
-or no file extension. See :doc:`YAMLSyntax` if you are new to YAML.
+These variable files are in YAML format. Valid file extensions include '.yml', '.yaml', '.json', or no file extension.
+See :doc:`YAMLSyntax` if you are new to YAML.
 
 Assuming the inventory file path is::
 
@@ -202,9 +279,9 @@ the 'raleigh' group might look like::
     ntp_server: acme.example.org
     database_server: storage.example.org
 
-It is ok if these files do not exist, as this is an optional feature.
+It is okay if these files do not exist, as this is an optional feature.
 
-As an advanced use-case, you can create *directories* named after your groups or hosts, and
+As an advanced use case, you can create *directories* named after your groups or hosts, and
 Ansible will read all the files in these directories. An example with the 'raleigh' group::
 
     /etc/ansible/group_vars/raleigh/db_settings
@@ -222,12 +299,40 @@ directory will override variables set in the inventory directory.
 Tip: Keeping your inventory file and variables in a git repo (or other version control)
 is an excellent way to track changes to your inventory and host variables.
 
+.. _how_we_merge:
+
+How Variables Are Merged
+++++++++++++++++++++++++
+
+By default variables are merged/flattened to the specific host before a play is run. This keeps Ansible focused on the Host and Task, so groups don't really surive outside of inventory and host matching. By default, Ansible overwrites variables including the ones defined for a group and/or host (see the `hash_merge` setting to change this) . The order/precedence is (from lowest to highest):
+
+- all group (becauseit is the 'parent' of all other groups)
+- parent group
+- child group
+- host
+
+When groups of the same parent/child level are merged, it is done alphabetically, and the last group loaded overwrites the previous groups. For example, an a_group will be merged with b_group and b_group vars that match will overwrite the ones in a_group.
+
+.. versionadded:: 2.4
+
+Starting in Ansible version 2.4, users can use the group variable ``ansible_group_priority`` to change the merge order for groups of the same level (after the parent/child order is resolved). The larger the number, the later it will be merged, giving it higher priority. This variable defaults to ``1`` if not set. For example:
+
+.. code-block:: yaml
+
+    a_group:
+        testvar: a
+        ansible_group_priority: 10
+    b_group
+        testvar: b
+
+In this example, if both groups have the same priority, the result would normally have been ``testvar == b``, but since we are giving the ``a_group`` a higher priority the result will be ``testvar == a``.
+
 .. _behavioral_parameters:
 
 List of Behavioral Inventory Parameters
 +++++++++++++++++++++++++++++++++++++++
 
-As alluded to above, setting the following variables controls how ansible interacts with remote hosts.
+As described above, setting the following variables control how Ansible interacts with remote hosts.
 
 Host connection:
 
@@ -372,7 +477,7 @@ Here is an example of how to instantly deploy to created containers::
    :doc:`intro_adhoc`
        Examples of basic commands
    :doc:`playbooks`
-       Learning Ansible’s configuration, deployment, and orchestration language.
+       Learning Ansible's configuration, deployment, and orchestration language.
    `Mailing List <http://groups.google.com/group/ansible-project>`_
        Questions? Help? Ideas?  Stop by the list on Google Groups
    `irc.freenode.net <http://irc.freenode.net>`_

docs/docsite/rst/intro_windows.rst

@@ -94,8 +94,12 @@ You can specify which authentication option you wish to use by setting it to the
 Certificate
 +++++++++++
 
-Certificate authentication is similar to SSH where a certificate is assigned to a local user and instead of using a password to authenticate a certificate is used instead.
+Certificate authentication is similar to SSH because it uses both a public and private key to access a local Windows user account. This removes the need for using a password.
 
+This requires two variables that specify the path to both the PEM file containing the certificate (cert_pem) and the related private key file (cert_key_pem).
+
+    ansible_winrm_cert_pem:
+    ansible_winrm_cert_key_pem:
 
 Kerberos
 ++++++++

docs/docsite/rst/modules_support.rst

@@ -1,51 +1,50 @@
-Module Support
---------------
+Module Maintenance & Support
+----------------------------
 
 .. toctree:: :maxdepth: 1
 
-Ansible has many modules, but not all of them are maintained by the core project committers. Each module should have associated metadata that indicates which of the following categories they fall into. This should be visible in each module's documentation.
-
-Documentation updates for each module can also be edited directly in the module and by submitting a pull request to the module source code; just look for the "DOCUMENTATION" block in the source tree.
-
-If you believe you have found a bug in a module and are already running the latest stable or development version of Ansible, first look in the `issue tracker at github.com/ansible/ansible <https://github.com/ansible/ansible/issues>`_ to see if a bug has already been filed.  If not, we would be grateful if you would file one.
-
-Should you have a question rather than a bug report, inquiries are welcome on the `ansible-project google group <https://groups.google.com/forum/#!forum/ansible-project>`_ or on Ansible's "#ansible" channel, located on irc.freenode.net.
-
-For development-oriented topics, use the `ansible-devel google group <https://groups.google.com/forum/#!forum/ansible-devel>`_  or Ansible's ``#ansible`` and ``#ansible-devel`` channels, located on irc.freenode.net.  You should also read :doc:`community`, :doc:`dev_guide/testing` and :doc:`dev_guide/developing_modules`.
-
-The modules are hosted on GitHub in a subdirectory of the `ansible <https://github.com/ansible/ansible/tree/devel/lib/ansible/modules>`_ repo.
+To help identify maintainers and understand how the included modules are officially supported, each module now has associated metadata that provides additional clarity for maintenance and support.
 
 Core
 ````
 
-These are :doc:`modules maintained by the Ansible Core Team<core_maintained>`
-and will always ship with Ansible itself.
+:doc:`Core modules are maintained by the Ansible Engineering Team<core_maintained>`.
+These modules are integral to the basic foundations of the Ansible distribution.
 
 Network
 ```````
 
-These are :doc:`modules maintained by the Ansible Network Team<network_maintained>` in a relationship
-similar to how the Ansible Core Team maintains the Core modules.
+:doc:`Network modules are maintained by the Ansible Network Team<network_maintained>`. Please note there are :doc:`additional networking modules<list_of_network_modules>` that are categorized as Certified or Community not maintained by Ansible.
+
 
 Certified
 `````````
 
-Some examples of :doc:`Certified Modules<partner_maintained>` are those submitted by other companies. Maintainers of these types of modules must watch for any issues reported or pull requests raised against the module.
-
-The Ansible Core Team will review all modules becoming certified.  Core committers will review proposed changes to existing Certified Modules once the community maintainers of the module have approved the changes. Core committers will also ensure that any issues that arise due to Ansible engine changes will be remediated.
-Also, it is strongly recommended (but not presently required) for these types of modules to have unit tests.
-
-These modules are currently shipped with Ansible, but might be shipped separately in the future.
+Certified modules are part of a future planned program currently in development.
 
 Community
 `````````
 
-These are :doc:`modules maintained by the Ansible Community<community_maintained>`.  They **are not** supported by the Ansible Core Team or by companies/partners associated to the module.
+:doc:`Community modules are submitted and maintained by the Ansible community<community_maintained>`.  These modules are not maintained by Ansible, and are included as a convenience.
 
-They are still fully usable, but the response rate to issues is purely up to the community.  Best effort support will be provided but is not covered under any support contracts.
+Issue Reporting
+```````````````
 
-These modules are currently shipped with Ansible, but will most likely be shipped separately in the future.
+If you believe you have found a bug in a module and are already running the latest stable or development version of Ansible, first look at the `issue tracker in the Ansible repo <https://github.com/ansible/ansible/issues>`_ to see if an issue has already been filed. If not, please file one.
 
+Should you have a question rather than a bug report, inquiries are welcome on the `ansible-project Google group <https://groups.google.com/forum/#%21forum/ansible-project>`_ or on Ansible's "#ansible" channel, located on irc.freenode.net.
+
+For development-oriented topics, use the `ansible-devel Google group <https://groups.google.com/forum/#%21forum/ansible-devel>`_ or Ansible's #ansible and #ansible-devel channels, located on irc.freenode.net. You should also read :doc:`Community Information & Contributing <community>`, :doc:`Testing Ansible <dev_guide/testing>`, and :doc:`Developing Modules <dev_guide/developing_modules>`.
+
+The modules are hosted on GitHub in a subdirectory of the `Ansible <https://github.com/ansible/ansible/tree/devel/lib/ansible/modules>`_ repo.
+
+NOTE: If you have a Red Hat Ansible Engine product subscription, please follow the standard issue reporting process via the Red Hat Customer Portal.
+
+Support
+```````
+
+For more information on how included Ansible modules are supported by Red Hat,
+please refer to the following `knowledgebase article <https://access.redhat.com/articles/3166901>`_ as well as other resources on the `Red Hat Customer Portal. <https://access.redhat.com/>`_
 
 .. seealso::
 

docs/docsite/rst/network_debug_troubleshooting.rst

@@ -51,7 +51,7 @@ Enabling Networking logging and how to read the logfile
 
 Ansible 2.3 features improved logging to help diagnose and troubleshoot issues regarding Ansible Networking modules.
 
-Because logging is very verbose it is disabled by default. It can be enabled via the :envvar:`ANSIBLE_LOG_PATH` and :envvar:`ANISBLE_DEBUG` options::
+Because logging is very verbose it is disabled by default. It can be enabled via the :envvar:`ANSIBLE_LOG_PATH` and :envvar:`ANSIBLE_DEBUG` options::
 
    # Specify the location for the log file
    export ANSIBLE_LOG_PATH=~/ansible.log

docs/docsite/rst/playbooks_acceleration.rst

@@ -1,82 +0,0 @@
-Accelerated Mode
-================
-
-.. versionadded:: 1.3
-
-.. note::
-
-     Accelerated mode is deprecated. Consider using SSH with ControlPersist and pipelining enabled instead. This feature will be removed in a future release. Deprecation warnings can be disabled by setting :code:`deprecation_warnings=False` in :code:`ansible.cfg`.
-
-You Might Not Need This!
-````````````````````````
-
-Are you running Ansible 1.5 or later? If so, you may not need accelerated mode due to a new feature called "SSH pipelining" and should read the :ref:`pipelining` section of the documentation.
-
-For users on 1.5 and later, accelerated mode only makes sense if you (A) are managing from an Enterprise Linux 6 or earlier host and still are on paramiko, or (B) can't enable TTYs with sudo as described in the pipelining docs.
-
-If you can use pipelining, Ansible will reduce the amount of files transferred over the wire, 
-making everything much more efficient, and performance will be on par with accelerated mode in nearly all cases, possibly excluding very large file transfer. Because less moving parts are involved, pipelining is better than accelerated mode for nearly all use cases.
-
-Accelerated mode remains around in support of EL6
-control machines and other constrained environments.
-
-Accelerated Mode Details
-````````````````````````
-
-While OpenSSH using the ControlPersist feature is quite fast and scalable, there is a certain small amount of overhead involved in
-using SSH connections. While many people will not encounter a need, if you are running on a platform that doesn't have ControlPersist support (such as an EL6 control machine), you'll probably be even more interested in tuning options.
-
-Accelerated mode is there to help connections work faster, but still uses SSH for initial secure key exchange. There is no
-additional public key infrastructure to manage, and this does not require things like NTP or even DNS. 
-
-Accelerated mode can be anywhere from 2-6x faster than SSH with ControlPersist enabled, and 10x faster than paramiko.
-
-Accelerated mode works by launching a temporary daemon over SSH. Once the daemon is running, Ansible will connect directly
-to it via a socket connection. Ansible secures this communication by using a temporary AES key that is exchanged during
-the SSH connection (this key is different for every host, and is also regenerated periodically). 
-
-By default, Ansible will use port 5099 for the accelerated connection, though this is configurable. Once running, the daemon will accept connections for 30 minutes, after which time it will terminate itself and need to be restarted over SSH.
-
-In order to use accelerated mode, simply add `accelerate: true` to your play::
-
-    ---
-
-    - hosts: all
-      accelerate: true
-
-      tasks:
-
-      - name: some task
-        command: echo {{ item }}
-        with_items:
-        - foo
-        - bar
-        - baz
-
-If you wish to change the port Ansible will use for the accelerated connection, just add the `accelerate_port` option::
-
-    ---
-
-    - hosts: all
-      accelerate: true
-      # default port is 5099
-      accelerate_port: 10000
-
-The `accelerate_port` option can also be specified in the environment variable :envvar:`ACCELERATE_PORT`, or in your `ansible.cfg` configuration::
-
-    [accelerate]
-    accelerate_port = 5099
-
-As noted above, accelerated mode also supports running tasks via sudo, however there are two important caveats:
-
-* You must remove requiretty from your sudoers options.
-* Prompting for the sudo password is not yet supported, so the NOPASSWD option is required for sudo'ed commands.
-
-As of Ansible version `1.6`, you can also allow the use of multiple keys for connections from multiple Ansible management nodes. To do so, add the following option
-to your `ansible.cfg` configuration::
-
-    accelerate_multi_key = yes
-
-When enabled, the daemon will open a UNIX socket file (by default `$ANSIBLE_REMOTE_TEMP/.ansible-accelerate/.local.socket`). New connections over SSH can
-use this socket file to upload new keys to the daemon.
-

docs/docsite/rst/playbooks_async.rst

@@ -94,7 +94,7 @@ of tasks running concurrently, you can do it this way::
           - 4
           - 5
         durations: "{{ item }}"
-      include: execute_batch.yml
+      include_tasks: execute_batch.yml
       with_items:
         - "{{ sleep_durations | batch(2) | list }}"
 

docs/docsite/rst/playbooks_best_practices.rst

@@ -216,15 +216,15 @@ variables from the file "group_vars/ec2_tag_class_webserver" automatically.
 Top Level Playbooks Are Separated By Role
 `````````````````````````````````````````
 
-In site.yml, we include a playbook that defines our entire infrastructure.  Note this is SUPER short, because it's just including
-some other playbooks.  Remember, playbooks are nothing more than lists of plays::
+In site.yml, we import a playbook that defines our entire infrastructure.  This is a very short example, because it's just importing
+some other playbooks::
 
     ---
     # file: site.yml
-    - include: webservers.yml
-    - include: dbservers.yml
+    - import_playbook: webservers.yml
+    - import_playbook: dbservers.yml
 
-In a file like webservers.yml (also at the top level), we simply map the configuration of the webservers group to the roles performed by the webservers group.  Also notice this is incredibly short.  For example::
+In a file like webservers.yml (also at the top level), we map the configuration of the webservers group to the roles performed by the webservers group::
 
     ---
     # file: webservers.yml

docs/docsite/rst/playbooks_blocks.rst

@@ -61,7 +61,8 @@ Blocks also introduce the ability to handle errors in a way similar to exception
 
 The tasks in the ``block`` would execute normally, if there is any error the ``rescue`` section would get executed
 with whatever you need to do to recover from the previous error. The ``always`` section runs no matter what previous
-error did or did not occur in the ``block`` and ``rescue`` sections.
+error did or did not occur in the ``block`` and ``rescue`` sections. It should be noted that the play continues if a
+``rescue`` section completes successfully as it 'erases' the error status (but not the reporting), this means it won't trigger ``max_fail_percentage`` nor ``any_errors_fatal`` configurations but will appear in the playbook statistics.
 
 
 Another example is how to run handlers after an error occurred :

docs/docsite/rst/playbooks_conditionals.rst

@@ -154,13 +154,13 @@ there will be accessible to future tasks::
 
 .. _when_roles_and_includes:
 
-Applying 'when' to roles and includes
-`````````````````````````````````````
+Applying 'when' to roles, imports, and includes
+```````````````````````````````````````````````
 
 Note that if you have several tasks that all share the same conditional statement, you can affix the conditional
 to a task include statement as below.  All the tasks get evaluated, but the conditional is applied to each and every task::
 
-    - include: tasks/sometasks.yml
+    - import_tasks: tasks/sometasks.yml
       when: "'reticulating splines' in output"
 
 .. note:: In versions prior to 2.0 this worked with task includes but not playbook includes.  2.0 allows it to work with both.
@@ -174,6 +174,24 @@ Or with a role::
 You will note a lot of 'skipped' output by default in Ansible when using this approach on systems that don't match the criteria.
 Read up on the 'group_by' module in the :doc:`modules` docs for a more streamlined way to accomplish the same thing.
 
+When used with `include_*` tasks instead of imports, the conditional is applied _only_ to the include task itself and not any other
+tasks within the included file(s). A common situation where this distinction is important is as follows::
+
+    # include a file to define a variable when it is not already defined
+
+    # main.yml
+    - include_tasks: other_tasks.yml
+      when: x is not defined
+
+    # other_tasks.yml
+    - set_fact:
+        x: foo
+    - debug:
+        var: x
+
+In the above example, if ``import_tasks`` had been used instead both included tasks would have also been skipped. With ``include_tasks``
+instead, the tasks are executed as expected because the conditional is not applied to them.
+
 .. _conditional_imports:
 
 Conditional Imports

docs/docsite/rst/playbooks_filters.rst

@@ -286,6 +286,15 @@ This example shows ports from cluster1::
 
 .. note:: You can use a variable to make the query more readable.
 
+Or, alternatively::
+
+    - name: "Display all server names from cluster1"
+      debug:
+        var: item
+      with_items: "{{domain_definition|json_query('domain.server[?cluster=`cluster1`].port')}}"
+
+.. note:: Here, quoting literals using backticks avoids escaping quotes and maintains readability.
+
 In this example, we get a hash map with all ports and names of a cluster::
 
     - name: "Display all server ports and names from cluster1"
@@ -442,7 +451,10 @@ To get a sha512 password hash (random salt)::
 To get a sha256 password hash with a specific salt::
 
     {{ 'secretpassword'|password_hash('sha256', 'mysecretsalt') }}
+    
+An idempotent method to generate unique hashes per system is to use a salt that is consistent between runs::
 
+    {{ 'secretpassword'|password_hash('sha512', 65534|random(seed=inventory_hostname)|string) }}
 
 Hash types available depend on the master system running ansible,
 'hash' depends on hashlib password_hash depends on passlib (http://passlib.readthedocs.io/en/stable/lib/passlib.hash.html).
@@ -601,34 +613,34 @@ URL Split Filter
 
 The ``urlsplit`` filter extracts the fragment, hostname, netloc, password, path, port, query, scheme, and username from an URL. With no arguments, returns a dictionary of all the fields::
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('hostname') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('hostname') }}
     # => 'www.acme.com'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('netloc') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('netloc') }}
     # => 'user:password@www.acme.com:9000'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('username') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('username') }}
     # => 'user'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('password') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('password') }}
     # => 'password'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('path') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('path') }}
     # => '/dir/index.html'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('port') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('port') }}
     # => '9000'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('scheme') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('scheme') }}
     # => 'http'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('query') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('query') }}
     # => 'query=term'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit('fragment') }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit('fragment') }}
     # => 'fragment'
 
-    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#frament" | urlsplit }}
+    {{ "http://user:password@www.acme.com:9000/dir/index.html?query=term#fragment" | urlsplit }}
     # =>
     #   {
     #       "fragment": "fragment",

docs/docsite/rst/playbooks_intro.rst

@@ -114,18 +114,26 @@ the web servers, and then the database servers. For example::
 
       tasks:
       - name: ensure apache is at the latest version
-        yum: name=httpd state=latest
+        yum:
+          name: httpd
+          state: latest
       - name: write the apache config file
-        template: src=/srv/httpd.j2 dest=/etc/httpd.conf
+        template:
+          src: /srv/httpd.j2
+          dest: /etc/httpd.conf
 
     - hosts: databases
       remote_user: root
 
       tasks:
       - name: ensure postgresql is at the latest version
-        yum: name=postgresql state=latest
+        yum:
+          name: postgresql
+          state: latest
       - name: ensure that postgresql is started
-        service: name=postgresql state=started
+        service:
+          name: postgresql
+          state: started
 
 You can use this method to switch between the host group you're targeting,
 the username logging into the remote servers, whether to sudo or not, and so
@@ -187,7 +195,9 @@ You can also use become on a particular task instead of the whole play::
     - hosts: webservers
       remote_user: yourname
       tasks:
-        - service: name=nginx state=started
+        - service: 
+            name: nginx
+            state: started
           become: yes
           become_method: sudo
 
@@ -302,7 +312,9 @@ the service module takes ``key=value`` arguments::
 
    tasks:
      - name: make sure apache is running
-       service: name=httpd state=started
+       service:
+         name: httpd
+         state: started
 
 The **command** and **shell** modules are the only modules that just take a list
 of arguments and don't use the ``key=value`` form.  This makes
@@ -340,12 +352,14 @@ a variable called ``vhost`` in the ``vars`` section, you could do this::
 
    tasks:
      - name: create a virtual host file for {{ vhost }}
-       template: src=somefile.j2 dest=/etc/httpd/conf.d/{{ vhost }}
+       template:
+         src: somefile.j2
+         dest: /etc/httpd/conf.d/{{ vhost }}
 
 Those same variables are usable in templates, which we'll get to later.
 
 Now in a very basic playbook all the tasks will be listed directly in that play, though it will usually
-make more sense to break up tasks using the ``include:`` directive.  We'll show that a bit later.
+make more sense to break up tasks as described in :doc:`playbooks_reuse`.
 
 .. _action_shorthand:
 
@@ -384,7 +398,9 @@ Here's an example of restarting two services when the contents of a file
 change, but only if the file changes::
 
    - name: template configuration file
-     template: src=template.j2 dest=/etc/foo.conf
+     template:
+       src: template.j2
+       dest: /etc/foo.conf
      notify:
         - restart memcached
         - restart apache
@@ -402,18 +418,26 @@ Here's an example handlers section::
 
     handlers:
         - name: restart memcached
-          service: name=memcached state=restarted
+          service:
+            name: memcached
+            state: restarted
         - name: restart apache
-          service: name=apache state=restarted
+          service:
+            name: apache
+            state: restarted
 
 As of Ansible 2.2, handlers can also "listen" to generic topics, and tasks can notify those topics as follows::
 
     handlers:
         - name: restart memcached
-          service: name=memcached state=restarted
+          service:
+            name: memcached
+            state: restarted
           listen: "restart web services"
         - name: restart apache
-          service: name=apache state=restarted
+          service:
+            name: apache
+            state:restarted
           listen: "restart web services"
 
     tasks:
@@ -458,7 +482,7 @@ Let's run a playbook using a parallelism level of 10::
 
     ansible-playbook playbook.yml -f 10
 
-.. _ansible-pull:
+.. _playbook_ansible-pull:
 
 Ansible-Pull
 ````````````

docs/docsite/rst/playbooks_lookups.rst

@@ -1,17 +1,13 @@
 Lookups
 -------
 
-Lookup plugins allow access of data in Ansible from outside sources. Like all templating, these plugins are evaluated on the Ansible control
-machine, and can include reading the filesystem but also contacting external datastores and services.
-These values are then made available using the standard templating system in Ansible, and are typically used to load variables or templates with information from those systems.
+Lookup plugins allow access to outside data sources. Like all templating, these plugins are evaluated on the Ansible control machine, and can include reading the filesystem as well as contacting external datastores and services. This data is then made available using the standard templating system in Ansible. 
 
-.. note:: This is considered an advanced feature, and many users will probably not rely on these features.
-
-.. note:: Lookups occur on the local computer, not on the remote computer.
-
-.. note:: Lookups are executed with a cwd relative to the role or play, as opposed to local tasks which are executed with the cwd of the executed script.
-
-.. note:: Since 1.9 you can pass wantlist=True to lookups to use in jinja2 template "for" loops.
+.. note::
+    - Lookups occur on the local computer, not on the remote computer.
+    - They are executed with in the directory containing the role or play, as opposed to local tasks which are executed with the directory of the executed script.
+    - Since Ansible version 1.9, you can pass wantlist=True to lookups to use in jinja2 template "for" loops.
+    - Lookups are an advanced feature. You should have a good working knowledge of Ansible plays before incorporating them.
 
 .. warning:: Some lookups pass arguments to a shell. When using variables from a remote/untrusted source, use the `|quote` filter to ensure safe usage.
 
@@ -24,7 +20,7 @@ Intro to Lookups: Getting File Contents
 
 The file lookup is the most basic lookup type.
 
-Contents can be read off the filesystem as follows::
+Contents can be read from the filesystem as follows::
 
     ---
     - hosts: all
@@ -42,18 +38,14 @@ The Password Lookup
 
 .. note::
 
-    A great alternative to the password lookup plugin, if you don't need to generate random passwords on a per-host basis, would be to use :doc:`playbooks_vault`.  Read the documentation there and consider using it first, it will be more desirable for most applications.
+    We recommend using :doc:`playbooks_vault` instead of the password lookup plugin if you don't need to generate random passwords on a per-host basis.
 
-``password`` generates a random plaintext password and stores it in
+The ``password lookup`` plugin generates a random plaintext password and stores it in
 a file at a given filepath.
 
-(Docs about crypted save modes are pending)
+If the file already exists, it will retrieve its contents, behaving just like with_file. Usage of variables like "{{ inventory_hostname }}" in the filepath can be used to set up random passwords per host (which simplifies password management in 'host_vars' variables).
 
-If the file exists previously, it will retrieve its contents, behaving just like with_file. Usage of variables like "{{ inventory_hostname }}" in the filepath can be used to set
-up random passwords per host (which simplifies password management in 'host_vars' variables).
-
-A special case is using ``/dev/null`` as a path. The password lookup will generate a new random password each time, but will not write it to ``/dev/null``. This can be used when you need a password
-without storing it on the controller.
+A special case is using ``/dev/null`` as a path. The password lookup will generate a new random password each time, but will not write it to ``/dev/null``. This can be used when you need a password without storing it on the controller.
 
 Generated passwords contain a random mix of upper and lowercase ASCII letters, the
 numbers 0-9 and punctuation (". , : - _"). The default length of a generated password is 20 characters.
@@ -74,9 +66,9 @@ This length can be changed by passing an extra parameter::
 
 .. note:: If the file already exists, no data will be written to it. If the file has contents, those contents will be read in as the password. Empty files cause the password to return as an empty string.
 
-Caution: Since this runs on the ansible host as the user running the playbook, and "become" does not apply, the target file must be readable by the playbook user, or, if it does not exist, the playbook user must have sufficient privileges to create it. (So, for example, attempts to write into areas such as /etc will fail unless the entire playbook is being run as root).
+.. warning:: Since the password lookup runs on the Ansible host as the user running the playbook, and "become" does not apply, the target file must be readable by the playbook user, or, if it does not exist, the playbook user must have sufficient privileges to create it. Attempts to write into areas such as /etc will fail unless the entire playbook is being run as root.
 
-Starting in version 1.4, password accepts a "chars" parameter to allow defining a custom character set in the generated passwords. It accepts comma separated list of names that are either string module attributes (ascii_letters,digits, etc) or are used literally::
+Starting in Ansible version 1.4, password accepts a "chars" parameter to allow defining a custom character set in the generated passwords. It accepts comma separated list of names that are either string module attributes (such as ascii_letters or digits) or are used literally::
 
     ---
     - hosts: all
@@ -100,7 +92,7 @@ Starting in version 1.4, password accepts a "chars" parameter to allow defining
 
         # (...)
 
-To enter comma use two commas ',,' somewhere - preferably at the end. Quotes and double quotes are not supported.
+To enter a comma use two commas ',,' somewhere - preferably at the end. Quotes and double quotes are not supported.
 
 .. _passwordstore_lookup:
 
@@ -458,7 +450,7 @@ Please check https://api.mongodb.org/python/current/api/pymongo/collection.html?
 
 Since there are too many parameters for this lookup method, below is a sample playbook which shows its usage and a nice way to feed the parameters:
 
-.. code-block:: YAML+Jinja
+.. code-block:: yaml
 
     ---
     - hosts: all

docs/docsite/rst/playbooks_loops.rst

@@ -408,7 +408,7 @@ Negative numbers are not supported.  This works as follows::
             state: present
           with_sequence: count=4
 
-.. _random_choice:
+.. _playbooks_loops_random_choice:
 
 Random Choices
 ``````````````
@@ -743,7 +743,7 @@ Ansible by default sets the loop variable `item` for each loop, which causes the
 As of Ansible 2.1, the `loop_control` option can be used to specify the name of the variable to be used for the loop::
 
     # main.yml
-    - include: inner.yml
+    - include_tasks: inner.yml
       with_items:
         - 1
         - 2
@@ -807,7 +807,7 @@ Because `loop_control` is not available in Ansible 2.0, when using an include wi
 for `item`::
 
     # main.yml
-    - include: inner.yml
+    - include_tasks: inner.yml
       with_items:
         - 1
         - 2

docs/docsite/rst/playbooks_prompts.rst

@@ -27,8 +27,9 @@ Here is a most basic example::
         - name: "favcolor"
           prompt: "what is your favorite color?"
 
+
 .. note::
-   Prompts for individual ``vars_prompt`` variables will be skipped for any variable that is already defined through the command line ``--extra-vars`` option, or when running from a non-interactive session (such as cron or Ansible Tower). See :ref:`_passing_variables_on_the_command_line` in the /Variables/ chapter.
+    Prompts for individual ``vars_prompt`` variables will be skipped for any variable that is already defined through the command line ``--extra-vars`` option, or when running from a non-interactive session (such as cron or Ansible Tower). See :ref:`passing_variables_on_the_command_line` in the /Variables/ chapter.
 
 If you have a variable that changes infrequently, it might make sense to
 provide a default value that can be overridden.  This can be accomplished using
@@ -77,13 +78,13 @@ You can use any crypt scheme supported by 'Passlib':
 - *sun_md5_crypt* - Sun MD5 Crypt
 - *sha256_crypt* - SHA-256 Crypt
 - *sha512_crypt* - SHA-512 Crypt
-- *apr_md5_crypt* - Apache’s MD5-Crypt variant
-- *phpass* - PHPass’ Portable Hash
+- *apr_md5_crypt* - Apache's MD5-Crypt variant
+- *phpass* - PHPass' Portable Hash
 - *pbkdf2_digest* - Generic PBKDF2 Hashes
-- *cta_pbkdf2_sha1* - Cryptacular’s PBKDF2 hash
-- *dlitz_pbkdf2_sha1* - Dwayne Litzenberger’s PBKDF2 hash
+- *cta_pbkdf2_sha1* - Cryptacular's PBKDF2 hash
+- *dlitz_pbkdf2_sha1* - Dwayne Litzenberger's PBKDF2 hash
 - *scram* - SCRAM Hash
-- *bsd_nthash* - FreeBSD’s MCF-compatible nthash encoding
+- *bsd_nthash* - FreeBSD's MCF-compatible nthash encoding
 
 However, the only parameters accepted are 'salt' or 'salt_size'. You can use your own salt using
 'salt', or have one generated automatically using 'salt_size'. If nothing is specified, a salt

docs/docsite/rst/playbooks_reuse.rst

@@ -9,10 +9,12 @@ Creating Reusable Playbooks
 
 While it is possible to write a playbook in one very large file (and you might start out learning playbooks this way), eventually you'll want to reuse files and start to organize things. In Ansible, there are three ways to do this: includes, imports, and roles.
 
-Includes and imports allow users to break up large playbooks into smaller files, which can be used across multiple parent playbooks or even multiple times within the same Playbook.
+Includes and imports (added in 2.4) allow users to break up large playbooks into smaller files, which can be used across multiple parent playbooks or even multiple times within the same Playbook.
 
 Roles allow more than just tasks to be packaged together and can include variables, handlers, or even modules and other plugins. Unlike includes and imports, roles can also be uploaded and shared via Ansible Galaxy.
 
+.. _dynamic_vs_static:
+
 Dynamic vs. Static
 ``````````````````
 

docs/docsite/rst/playbooks_reuse_includes.rst

@@ -13,17 +13,25 @@ As noted in :doc:`playbooks_reuse`, include and import statements are very simil
 
 Please refer to  :doc:`playbooks_reuse` for documentation concerning the trade-offs one may encounter when using each type.
 
+Also be aware that this behaviour changed in 2.4; prior to that Ansible version only ``include`` was available, and it behaved differently depending on context.
+
+.. versionadded:: 2.4
+
 Importing Playbooks
 ```````````````````
 
 It is possible to include playbooks inside a master playbook. For example::
 
     ---
-    import_playbook: webservers.yml
-    import_playbook: databases.yml
+    - import_playbook: webservers.yml
+    - import_playbook: databases.yml
 
-Each playbook listed will be run in the order they are listed.
+The plays and tasks in each playbook listed will be run in the order they are listed, just as if they had been defined here directly.
 
+Prior to 2.4 only ``include`` was available and worked for both playbooks and tasks as both import and include.
+
+
+.. versionadded:: 2.4
 
 Including and Importing Task Files
 ``````````````````````````````````
@@ -61,7 +69,7 @@ Variables can also be passed to include files using an alternative syntax, which
         - "{{ lookup('file', 'keys/one.pub') }}"
         - "{{ lookup('file', 'keys/two.pub') }}"
 
-Using either syntax, variables passed in can then be used in the included files. These variables will only be available to tasks within the included file. See :doc:`variable_precedence` for more details on variable inheritance and precedence.
+Using either syntax, variables passed in can then be used in the included files. These variables will only be available to tasks within the included file. See :ref:`ansible_variable_precedence` for more details on variable inheritance and precedence.
 
 Task include statements can be used at arbitrary depth.
 

docs/docsite/rst/playbooks_reuse_roles.rst

@@ -33,8 +33,8 @@ Roles expect files to be in certain directory names. Roles must include at least
 
 - ``tasks`` - contains the main list of tasks to be executed by the role.
 - ``handlers`` - contains handlers, which may be used by this role or even anywhere outside this role.
-- ``defaults`` - default variables for the role (see :doc:`Variables` for more information).
-- ``vars`` - other variables for the role (see :doc:`Variables` for more information).
+- ``defaults`` - default variables for the role (see :doc:`playbooks_variables` for more information).
+- ``vars`` - other variables for the role (see :doc:`playbooks_variables` for more information).
 - ``files`` - contains files which can be deployed via this role.
 - ``templates`` - contains templates which can be deployed via this role.
 - ``meta`` - defines some meta data for this role. See below for more details.
@@ -42,7 +42,8 @@ Roles expect files to be in certain directory names. Roles must include at least
 Other YAML files may be included in certain directories. For example, it is common practice to have platform-specific tasks included from the ``tasks/main.yml`` file::
 
     # roles/example/tasks/main.yml
-    - import_tasks: redhat.yml
+    - name: added in 2.4, previouslly you used 'include'
+      import_tasks: redhat.yml
       when: ansible_os_platform|lower == 'redhat'
     - import_tasks: debian.yml
       when: ansible_os_platform|lower == 'debian'
@@ -57,7 +58,7 @@ Other YAML files may be included in certain directories. For example, it is comm
         name: "apache2"
         state: present
 
-Roles may also include modules and other plugin types. For more information, please refer to the :doc:`Embedding Modules and Plugins In Roles` section below.
+Roles may also include modules and other plugin types. For more information, please refer to the :ref:`embedding_modules_and_plugins_in_roles` section below.
 
 Using Roles
 ```````````
@@ -113,9 +114,9 @@ As of Ansible 2.4, you can now use roles inline with any other tasks using ``imp
 When roles are defined in the classic manner, they are treated as static imports and processed during playbook parsing.
 
 .. note::
-    The ``include_role`` option was introduced in Ansible 2.3. The usage has changed slightly as of Ansible 2.4 to match the include (dynamic) vs. import (static) usage. See :doc:`Dynamic vs. Static` for more details.
+    The ``include_role`` option was introduced in Ansible 2.3. The usage has changed slightly as of Ansible 2.4 to match the include (dynamic) vs. import (static) usage. See :ref:`dynamic_vs_static` for more details.
 
-The name used for the role can be a simple name (see :doc:`Role Search Path` below), or it can be a fully qualified path::
+The name used for the role can be a simple name (see :ref:`role_search_path` below), or it can be a fully qualified path::
 
     ---
 
@@ -290,6 +291,8 @@ Note that we did not have to use ``allow_duplicates: true`` for ``wheel``, becau
 .. note::
    Variable inheritance and scope are detailed in the :doc:`playbooks_variables`.
 
+.. _embedding_modules_and_plugins_in_roles:
+
 Embedding Modules and Plugins In Roles
 ``````````````````````````````````````
 
@@ -332,6 +335,8 @@ The same mechanism can be used to embed and distribute plugins in a role, using
 
 They can then be used in a template or a jinja template in any role called after 'my_custom_filter'
 
+.. _role_search_path:
+
 Role Search Path
 ````````````````
 

docs/docsite/rst/playbooks_special_topics.rst

@@ -16,9 +16,10 @@ and adopt these only if they seem relevant or useful to your environment.
    playbooks_environment
    playbooks_error_handling
    playbooks_advanced_syntax
-   playbooks_lookups
+   plugins
    playbooks_prompts
    playbooks_tags
    playbooks_vault
    playbooks_startnstep
    playbooks_keywords
+   playbooks_lookups

docs/docsite/rst/playbooks_tags.rst

@@ -80,9 +80,14 @@ You may also apply tags to roles::
     roles:
       - { role: webserver, port: 5000, tags: [ 'web', 'foo' ] }
 
-And include statements::
+And import/include statements::
 
-    - include: foo.yml
+    - import_tasks: foo.yml
+      tags: [web,foo]
+
+or::
+
+    - include_tasks: foo.yml
       tags: [web,foo]
 
 All of these apply the specified tags to EACH task inside the play, included

docs/docsite/rst/playbooks_variables.rst

@@ -798,7 +798,7 @@ As of Ansible 1.3, extra vars can be loaded from a JSON file with the ``@`` synt
 Also as of Ansible 1.3, extra vars can be formatted as YAML, either on the command line
 or in a file as above.
 
-.. _variable_precedence:
+.. _ansible_variable_precedence:
 
 Variable Precedence: Where Should I Put A Variable?
 ````````````````````````````````````````````````````
@@ -867,6 +867,8 @@ Basically, anything that goes into "role defaults" (the defaults folder inside t
           If multiple groups have the same variable, the last one loaded wins.
           If you define a variable twice in a play's vars: section, the 2nd one wins.
 .. note:: the previous describes the default config `hash_behavior=replace`, switch to 'merge' to only partially overwrite.
+.. note:: Group loading follows parent/child relationships. Groups of the same 'patent/child' level are then merged following alphabetical order.
+          This last one can be superceeded by the user via `ansible_group_priority`, which defaults to 0 for all groups.
 
 
 Another important thing to consider (for all versions) is that connection variables override config, command line and play/role/task specific options and directives.  For example::
@@ -1000,7 +1002,7 @@ can set variables in there and make use of them in other roles and elsewhere in
         - { role: something, foo: 12 }
         - { role: something_else }
 
-.. note:: There are some protections in place to avoid the need to namespace variables.  
+.. note:: There are some protections in place to avoid the need to namespace variables.
           In the above, variables defined in common_settings are most definitely available to 'something' and 'something_else' tasks, but if
           "something's" guaranteed to have foo set at 12, even if somewhere deep in common settings it set foo to 20.
 

docs/docsite/rst/playbooks_vault.rst

@@ -55,7 +55,7 @@ As of version 2.3, Ansible can now use a vaulted variable that lives in an other
               34623731376664623134383463316265643436343438623266623965636363326136
     other_plain_text: othervalue
 
-To create a vaulted variable, use the :ref:`ansible-vault encrypt_string` command. See :ref:`encrypt_string` for details.
+To create a vaulted variable, use the :ref:`ansible-vault encrypt_string <ansible_vault_encrypt_string>` command. See :ref:`encrypt_string` for details.
 
 This vaulted variable will be decrypted with the supplied vault secret and used as a normal variable. The ``ansible-vault`` command line supports stdin and stdout for encrypting data on the fly, which can be used from your favorite editor to create these vaulted variables; you just have to be sure to add the ``!vault`` tag so both Ansible and YAML are aware of the need to decrypt. The ``|`` is also required, as vault encryption results in a multi-line string.
 

docs/docsite/rst/plugins.rst

@@ -0,0 +1,29 @@
+Plugins
+=======
+
+Plugins are pieces of code that augment Ansible's core functionality. Ansible uses a plugin architecture to enable a rich, flexible and expandible feature set.
+
+Ansible ships with a number of handy plugins, and you can easily write your own.
+
+This section covers the various types of plugins that are included with Ansible:
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    plugins/*
+    playbooks_filters
+    playbooks_tests
+
+
+.. seealso::
+
+   :doc:`playbooks`
+       An introduction to playbooks
+   :doc:`config`
+       Ansible configuration documentation and settings
+   :doc:`command_line_tools`
+       Ansible tools, description and options
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/action.rst

@@ -0,0 +1,33 @@
+Action Plugins
+---------------
+
+Action plugins act in conjunction with :doc:`modules <../modules>` to execute the actions required by playbook tasks.
+They usually execute automatically in the background doing prerequisite work before modules execute.
+
+The 'normal' action plugin is used for modules that do not already have an action plugin.
+
+Enabling Action Plugins
++++++++++++++++++++++++
+
+You can enable a custom action plugin by either dropping it into the ``action_plugins`` directory adjacent to your play, inside a role, or by putting it in one of the action plugin directory sources configured in :doc:`ansible.cfg <../config>`.
+
+.. seealso::
+
+   :doc:`cache`
+       Ansible Cache plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`shell`
+       Ansible Shell plugins
+   :doc:`strategy`
+       Ansible Strategy plugins
+   :doc:`vars`
+       Ansible Vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/cache.rst

@@ -0,0 +1,64 @@
+.. contents:: Topics
+
+
+Cache Plugins
+-------------
+
+Cache plugin implement a backend caching mechanism that allows Ansible to store gathered facts or inventory source data
+without the performance hit of retrieving them from source.
+
+The default cache plugin is the :doc:`memory <cache/memory>` plugin, which only caches the data for the current execution of Ansible. Other plugins with persistent storage are available to allow caching the data across runs.
+
+
+Enabling Cache Plugins
+++++++++++++++++++++++
+
+Only one cache plugin can be active at a time.
+
+You can enable a cache plugin in the Ansible configuration, either via environment variable:
+
+.. code-block:: shell
+
+    export ANSIBLE_CACHE_PLUGIN=jsonfile
+
+or in the ``ansible.cfg`` file:
+
+.. code-block:: ini
+
+    [defaults]
+    fact_caching=redis
+
+You will also need to configure other settings specific to each plugin. Consult the individual plugin documentation
+or the Ansible :doc:`configuration <../config>` for more details.
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t cache -l`` to see the list of available plugins. 
+Use ``ansible-doc -t cache <plugin name>`` to see specific documentation and examples.
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    cache/*
+
+.. seealso::
+
+   :doc:`action`
+       Ansible Action plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`shell`
+       Ansible Shell plugins
+   :doc:`strategy`
+       Ansible Strategy plugins
+   :doc:`vars`
+       Ansible Vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/callback.rst

@@ -0,0 +1,103 @@
+.. contents:: Topics
+
+
+Callback Plugins
+----------------
+
+Callback plugins enable adding new behaviors to Ansible when responding to events.
+By default, callback plugins control most of the output you see when running the command line programs,
+but can also be used to add additional output, integrate with other tools and marshall the events to a storage backend.
+
+.. _callback_examples:
+
+Example Callback Plugins
+++++++++++++++++++++++++
+
+The :doc:`log_plays <callback/log_plays>` callback is an example of how to record playbook events to a log file,
+and the :doc:`mail callback/mail` callback sends email on playbook failures.
+
+The :doc:`osx_say <callback/oxs_say>` callback responds with computer synthesized speech on OS X in relation to playbook events.
+
+
+.. _enabling_callbacks:
+
+Enabling Callback Plugins
+++++++++++++++++++++++++++
+
+You can activate a custom callback by either dropping it into a ``callback_plugins`` directory adjacent to your play,  inside a role, or by putting it in one of the callback directory sources configured in :doc:`ansible.cfg <../config>`.
+
+Plugins are loaded in alphanumeric order. For example, a plugin implemented in a file named `1_first.py` would run before a plugin file named `2_second.py`.
+
+Most callbacks shipped with Ansible are disabled by default and need to be whitelisted in your :doc:`ansible.cfg <../config>` file in order to function. For example:
+
+.. code-block:: ini
+
+  #callback_whitelist = timer, mail, profile_roles
+
+
+Managing stdout
+```````````````
+
+You can only have one plugin be the main manager of your console output. If you want to replace the default, you should define CALLBACK_TYPE = stdout in the subclass and then configure the stdout plugin in :doc:`ansible.cfg <../config>`. For example:
+
+.. code-block:: ini
+
+  stdout_callback = dense
+
+or for my custom callback:
+
+.. code-block:: ini
+
+  stdout_callback = mycallback
+
+This only affects :doc:`../ansible-playbook` by default.
+
+Managing AdHoc
+``````````````
+
+The :doc:`ansible <../ansible>` AdHoc command specifically uses a different callback plugin for stdout,
+so there is an extra setting in :doc:`ansible.cfg <../config>` you need to add to use the stdout callback defined above:
+
+.. code-block:: ini
+
+    [defaults]
+    bin_ansible_callbacks=True
+
+You can also set this as an environment variable:
+
+.. code-block:: shell
+
+    export ANSIBLE_LOAD_CALLBACK_PLUGINS=1
+
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t callback -l`` to see the list of available plugins. 
+Use ``ansible-doc -t callback <plugin name>`` to see specific documents and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    callback/*
+
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   :doc:`vars`
+       Ansible vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/connection.rst

@@ -0,0 +1,82 @@
+.. contents:: Topics
+
+
+Connection Plugins
+------------------
+
+Connection plugins allow Ansible to connect to the target hosts so it can execute tasks on them. Ansible ships with many connection plugins, but only one can be used per host at a time.
+
+By default, Ansible ships with several plugins. The most commonly used are the 'paramiko' SSH, native ssh (just called 'ssh'), and 'local' connection types.  All of these can be used in playbooks and with /usr/bin/ansible to decide how you want to talk to remote machines.  
+
+The basics of these connection types are covered in the :doc:`../intro_getting_started` section.  
+
+
+.. contents:: Topics
+
+.. _ssh_plugins:
+
+ssh Plugins
++++++++++++
+
+Because ssh is the default protocol used in system administration and the protocol most used in Ansible, ssh options are included in the command line tools. See :doc:`../ansible-playbook` for more details.
+
+
+.. _using_connection_plugins:
+
+Using Connection Plugins
+++++++++++++++++++++++++
+
+The transport can be changed via :doc:`configuration <../config>`, in the command line (``-c``, ``--connection``), as a keyword (:ref:`connection`)
+in your play or by setting the a connection variable (:ref:`ansible_connection`), most often, in your inventory.
+For example, for windows machines you might want o use the :doc:`winrm <connection/winrm>` plugin instead.
+
+Most connection plugins can operate with a minimum configuration. By default they use the :ref:`inventory_hostname` and defaults to find the target host.
+
+Plugins are self-documenting. Each plugin should document its configuration options. The following are connection variables common to most connection plugins:
+
+:ref:ansible_host
+    The name of the host to connect to, if different from the :ref:`inventory_hostname`.
+:ref:ansible_port
+    The ssh port number. For :doc:`ssh <connection/ssh>` and :doc:`paramiko <connection/paramiko>` the default value is 22.
+:ref:ansible_user
+    The default user name to use for log in. Most plugins default to the current user running Ansible.
+
+Each plugin might also have a specific version of a variable that overrides the general version. For example, :ref:`ansible_ssh_host` for the :doc:`ssh <connection/ssh>` plugin.
+
+Enabling Connection Plugins
++++++++++++++++++++++++++++
+
+You can extend Ansible to support other transports (such as SNMP or message bus) by dropping a custom plugin
+into the ``connection_plugins`` directory.
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t connection -l`` to see the list of available plugins.
+Use ``ansible-doc -t connection <plugin name>`` to see detailed documentation and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    connection/*
+
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   :doc:`vars`
+       Ansible vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/inventory.rst

@@ -0,0 +1,61 @@
+.. contents:: Topics
+
+
+Inventory Plugins
+-----------------
+
+Inventory plugins allow users to point at data sources to compile the inventory of hosts that Ansible uses to target tasks, either via the ``-i /path/to/file`` and/or ``-i 'host1, host2`` command line parameters or from other configuration sources.
+
+.. _enabling_inventory_plugins:
+
+Enabling Inventory Plugins
+++++++++++++++++++++++++++
+
+Most inventory plugins shipped with Ansible are disabled by default and need to be whitelisted in your
+:doc:`ansible.cfg <../config>` file in order to function.  This is how the default whitelist looks in the
+config file that ships with Ansible:
+
+.. code-block:: ini
+
+   [inventory]
+   enable_plugins = host_list, script, yaml, ini
+
+This list also establishes the order in which each plugin tries to parse an inventory source. Any plugins left out of the list will not be considered, so you can 'optimize' your inventory loading by minimizing it to what you actually use. For example:
+
+.. code-block:: ini
+
+   [inventory]
+   enable_plugins = advanced_host_list, constructed, yaml
+
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t inventory -l`` to see the list of available plugins. 
+Use ``ansible-doc -t inventory <plugin name>`` to see plugin-specific documentation and examples.
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    inventory/*
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   :doc:`vars`
+       Ansible vars plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/shell.rst

@@ -0,0 +1,31 @@
+Shell Plugins
+-------------
+
+Shell plugins work transparently to ensure that the basic commands Ansible runs are properly formated to work with the target machine.
+
+Enabling Shell Plugins
+++++++++++++++++++++++
+
+.. warning:: These plugins should not be reconfigured unless you have a restricted or exotic setup
+             in which the default ``/bin/sh`` is not a POSIX compatible shell or not availble for execution. 
+
+In addition to modifying the default configuration settings in :doc:`../config`, you can use a 'connection variable' :ref:`ansible_shell_type` to select a shell plugin, and update the :ref:`ansible_executable` to match.
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/strategy.rst

@@ -0,0 +1,71 @@
+.. contents:: Topics
+
+
+Strategy Plugins
+----------------
+
+Strategy plugins control the flow of play execution by handling task and host scheduling.
+
+
+Enabling Strategy Plugins
++++++++++++++++++++++++++
+
+Only one strategy plugin can be used in a play, but you can use different ones for each play in a playbook or Ansible run.
+The default is the :doc:`linear <strategy/linear>` plugin. You can change this default in Ansible :doc:`configuration <../config>` using an environment variable:
+
+.. code-block:: shell
+
+    export ANSIBLE_STRATEGY=free
+
+or in the `ansible.cfg` file:
+
+.. code-block:: ini
+
+    [defaults]
+    strategy=linear
+
+You can also specify the strategy plugin in the play via the :ref:`strategy` keyword::
+
+  - hosts: all
+    strategy: debug
+    tasks:
+      - copy: src=myhosts dest=/etc/hosts
+        notify: restart_tomcat
+
+      - package: name=tomcat state=present
+
+    handlers:
+      - name: restart_tomcat
+        service: name=tomcat state=restarted
+
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t strategy -l`` to see the list of available plugins. 
+Use ``ansible-doc -t strategy <plugin name>`` to see plugin-specific specific documentation and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    strategy/*
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/vars.rst

@@ -0,0 +1,50 @@
+.. contents:: Topics
+
+
+Vars Plugins
+------------
+
+Vars plugins inject additional variable data into Ansible runs that did not come from an inventory source, playbook, or command line. Playbook constructs like 'host_vars' and 'group_vars' work using vars plugins.
+
+Vars plugins were partially implented in Ansible 2.0 and rewritten to be fully implemented starting with Ansible 2.4.
+
+The :doc:`host_group_vars <vars/host_group_vars>` plugin shipped with Ansible enables reading variables from :ref:`host_vars` and :ref:`group_vars`.
+
+
+Enabling Vars Plugins
++++++++++++++++++++++
+
+You can activate a custom vars plugins by either dropping it into a ``vars_plugins`` directory adjacent to your play,  inside a role, or by putting it in one of the directory sources configured in :doc:`ansible.cfg <../config>`.
+
+Plugin Lists
+++++++++++++
+
+You can use ``ansible-doc -t vars -l`` to see the list of available plugins. 
+Use ``ansible-doc -t vars <plugin name>`` to see specific plugin-specific documentation and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    vars/*
+
+.. seealso::
+
+   :doc:`action`
+       Ansible Action plugins
+   :doc:`cache`
+       Ansible Cache plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`connection`
+       Ansible connection plugins
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`shell`
+       Ansible Shell plugins
+   :doc:`strategy`
+       Ansible Strategy plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/porting_guide_2.0.rst

@@ -109,7 +109,7 @@ While all items listed here will show a deprecation warning message, they still
 
 * Bare variables in ``with_`` loops should instead use the ``"{ {var }}"`` syntax, which helps eliminate ambiguity.
 * The ansible-galaxy text format requirements file. Users should use the YAML format for requirements instead.
-* Undefined variables within a ``with_`` loop’s list currently do not interrupt the loop, but they do issue a warning; in the future, they will issue an error.
+* Undefined variables within a ``with_`` loop's list currently do not interrupt the loop, but they do issue a warning; in the future, they will issue an error.
 * Using dictionary variables to set all task parameters is unsafe and will be removed in a future version. For example::
 
     - hosts: localhost
@@ -129,8 +129,8 @@ While all items listed here will show a deprecation warning message, they still
 
 * Host patterns should use a comma (,) or colon (:) instead of a semicolon (;) to separate hosts/groups in the pattern.
 * Ranges specified in host patterns should use the [x:y] syntax, instead of [x-y].
-* Playbooks using privilege escalation should always use “become*” options rather than the old su*/sudo* options.
-* The “short form” for vars_prompt is no longer supported.
+* Playbooks using privilege escalation should always use "become*" options rather than the old su*/sudo* options.
+* The "short form" for vars_prompt is no longer supported.
   For example::
 
     vars_prompt:
@@ -138,28 +138,28 @@ While all items listed here will show a deprecation warning message, they still
 
 * Specifying variables at the top level of a task include statement is no longer supported. For example::
 
-    - include: foo.yml
+    - include_tasks: foo.yml
         a: 1
 
 Should now be::
 
-    - include: foo.yml
+    - include_tasks: foo.yml
       vars:
         a: 1
 
 * Setting any_errors_fatal on a task is no longer supported. This should be set at the play level only.
-* Bare variables in the `environment` dictionary (for plays/tasks/etc.) are no longer supported. Variables specified there should use the full variable syntax: ‘{{foo}}’.
+* Bare variables in the `environment` dictionary (for plays/tasks/etc.) are no longer supported. Variables specified there should use the full variable syntax: '{{foo}}'.
 * Tags (or any directive) should no longer be specified with other parameters in a task include. Instead, they should be specified as an option on the task.
   For example::
 
-    - include: foo.yml tags=a,b,c
+    - include_tasks: foo.yml tags=a,b,c
 
   Should be::
 
-    - include: foo.yml
+    - include_tasks: foo.yml
       tags: [a, b, c]
 
-* The first_available_file option on tasks has been deprecated. Users should use the with_first_found option or lookup (‘first_found’, …) plugin.
+* The first_available_file option on tasks has been deprecated. Users should use the with_first_found option or lookup ('first_found', …) plugin.
 
 
 Other caveats

docs/docsite/rst/porting_guide_2.3.rst

@@ -213,10 +213,10 @@ Will result in:
 delegate_to vs ProxyCommand
 ---------------------------
 
-The new connection framework for Network Modules in Ansible 2.3 no longer supports the use of the
-``delegate_to`` directive.  In order to use a bastion or intermediate jump host
-to connect to network devices, network modules now support the use of
-``ProxyCommand``.
+The new connection framework for Network Modules in Ansible 2.3 that uses ``cli`` transport
+no longer supports the use of the ``delegate_to`` directive.
+In order to use a bastion or intermediate jump host to connect to network devices over ``cli``
+transport, network modules now support the use of ``ProxyCommand``.
 
 To use ``ProxyCommand`` configure the proxy settings in the Ansible inventory
 file to specify the proxy host.

docs/docsite/rst/porting_guide_2.4.rst

@@ -0,0 +1,156 @@
+.. _porting_2.4_guide:
+
+*************************
+Ansible 2.4 Porting Guide
+*************************
+
+This section discusses the behavioral changes between Ansible 2.3 and Ansible 2.4.
+
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
+
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/stable-2.4/CHANGELOG.md#2.4>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
+
+Python version
+==============
+
+Ansible will not support Python 2.4 nor 2.5 on the target hosts anymore. Going forward, Python 2.6+ will be required on targets, as already is the case on the controller.
+
+
+Inventory
+=========
+
+Inventory has been refactored to be implemented via plugins and now allows for multiple sources. This change is mostly transparent to users. 
+
+One exception is the ``inventory_dir``, which is now a host variable; previously it could only have one value so it was set globally. This means you cannot use it early in plays anymore to determine ``hosts:`` or similar keywords.
+
+The ``inventory_file`` remains unchaged, as it was always host specific.
+
+
+Deprecated
+==========
+
+Specifying Inventory sources
+-----------------------------
+
+Use of ``--inventory-file`` on the command line is now deprecated. Use ``--inventory`` or ``-i``.
+The associated ini configuration key, ``hostfile``, and environment variable, :envvar:`ANSIBLE_HOSTS`,
+are also deprecated.  Replace them with the configuration key ``inventory`` and environment variable :envvar:        `ANSIBLE_INVENTORY`.
+
+Use of multiple tags
+--------------------
+
+Specifying ``--tags`` (or ``--skip-tags``) multiple times on the command line currently leads to the last one overriding all the previous ones. This behavior is deprecated. In the future, if you specify --tags multiple times the tags will be merged together. From now on, using ``--tags`` multiple times on one command line will emit a deprecation warning. Setting the ``merge_multiple_cli_tags`` option to True in the ``ansible.cfg`` file will enable the new behavior.
+
+In 2.4, the default has change to merge the tags. You can enable the old overwriting behavior via the config option.
+
+In 2.5, multiple ``--tags`` options will be merged with no way to go back to the old behavior.
+
+
+Other caveats
+-------------
+
+No major changes in this version.
+
+Modules
+=======
+
+Major changes in popular modules are detailed here
+
+* The :ref:`win_shell <win_shell>` and :ref:`win_command <win_command>` modules now properly preserve quoted arguments in the command-line. Tasks that attempted to work around the issue by adding extra quotes/escaping may need to be reworked to remove the superfluous escaping. See `Issue 23019 <https://github.com/ansible/ansible/issues/23019>`_ for additional detail.
+
+Modules removed
+---------------
+
+The following modules no longer exist:
+
+* None
+
+Deprecation notices
+-------------------
+
+The following modules will be removed in Ansible 2.8. Please update update your playbooks accordingly.
+
+* :ref:`azure <azure>`, use :ref:`azure_rm_virtualmachine <azure_rm_virtualmachine>`, which uses the new Resource Manager SDK.
+* :ref:`win_msi <win_msi>`, use :ref:`win_package <win_package>` instead
+
+Noteworthy module changes
+-------------------------
+
+* The :ref:`win_get_url <win_get_url>`  module has the dictionary ``win_get_url`` in its results deprecated, its content is now also available directly in the resulting output, like other modules. This dictionary will be removed in Ansible 2.8.
+* The :ref:`win_unzip <win_unzip>` module no longer includes the dictionary ``win_unzip`` in its results; the contents are now included directly in the resulting output, like other modules.
+* The :ref:`win_package <win_package>` module return values ``exit_code`` and ``restart_required`` have been deprecated in favour of ``rc`` and ``reboot_required`` respectively. The deprecated return values will be removed in Ansible 2.6.
+
+
+Plugins
+=======
+
+A new way to configure and document plugins has been introduced.  This does not require changes to existing setups but developers should start adapting to the new infrastructure now. More details will be available in the developer documentation for each plugin type.
+
+Vars plugin changes
+-------------------
+
+There have been many changes to the implementation of vars plugins, but both users and developers should not need to change anything to keep current setups working. Developers should consider changing their plugins take advantage of new features.
+
+The most notable difference to users is that vars plugins now get invoked on demand instead of at inventory build time.  This should make them more efficient for large inventories, especially when using a subset of the hosts.
+
+.. note:: This also creates a difference with group/host_vars when using them adjacent to playbooks. Before, the 'first' playbook loaded determined the variables; now the 'current' playbook does. We are looking to fix this soon, since 'all playbooks' in the path should be considered for variable loading.
+
+
+Inventory plugins
+-----------------
+
+Developers should start migrating from hardcoded inventory with dynamic inventory scripts to the new Inventory Plugins. The scripts will still work via the ``script`` inventory plugin but Ansible development efforts will now concentrate on writing plugins rather than enhancing existing scripts.
+
+Both users and developers should look into the new plugins because they are intended to alleviate the need for many of the hacks and workarounds found in the dynamic inventory scripts.
+
+Callback plugins
+----------------
+
+Users:
+
+* Callbacks are now using the new configuration system.  Users should not need to change anything as the old system still works,
+  but you might see a deprecation notice if any callbacks used are not inheriting from the built in classes. Developers need to update them as stated below.
+
+Developers:
+
+* If your callback does not inherit from ``CallbackBase`` (directly or indirectly via another callback), it will still work, but issue a deprecation notice.
+  To avoid this and ensure it works in the future change it to inherit from ``callbackBase`` so it has the new options handling methods and properties.
+  You can also implement the new options handling methods and properties but that won't automatically inherit changes added in the future.  You can look at ``CallbackBase`` itself and/or ``AnsiblePlugin`` for details.
+* Any callbacks inheriting from other callbacks might need to also be updated to contain the same documented options
+  as the parent or the options won't be available.  This is noted in the developer guide.
+
+Networking
+==========
+
+There have been a number of changes to how Networking Modules operate.
+
+Playbooks should still use ``connection: local``.
+
+Persistent Connection
+---------------------
+
+The configuration variables ``connection_retries`` and ``connect_interval`` which were added in Ansible 2.3 are now deprecated. For Ansible 2.4 and later use ``connection_retry_timeout``.
+
+To control timeouts use ``command_timeout`` rather than the previous top level ``timeout`` variable under ``[default]``
+
+See :ref:`Ansible Network debug guide <network_debug_troubleshooting>` for more information.
+
+
+Configuration
+=============
+
+
+The configuration system has had some major changes. Users should be unaffected except for the following:
+
+* All relative paths defined are relative to the `ansible.cfg` file itself. Previously they varied by setting. The new behavior should be more predictable.
+* A new macro ``{{CWD}}`` is available for paths, which will make paths relative to the 'current working directory',
+  this is unsafe but some users really want to rely on this behaviour.
+
+Developers that were working directly with the previous API should revisit their usage as some methods (for example, ``get_config``) were  kept for backwards compatibility but will warn users that the function has been deprecated.
+
+The new configuration has been designed to minimize the need for code changes in core for new plugins.  The plugins just need to document their settings and the configuration system will use the documentation to provide what they need. This is still a work in progress; currently only 'callback' and 'connection' plugins support this.  More  details will be added to the specific plugin developer guides.

docs/docsite/rst/porting_guide_2.5.rst

@@ -0,0 +1,102 @@
+.. _porting_2.5_guide:
+
+*************************
+Ansible 2.5 Porting Guide
+*************************
+
+This section discusses the behavioral changes between Ansible 2.4 and Ansible 2.5.
+
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#2.5>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
+
+Playbook
+========
+
+No notable changes.
+
+Deprecated
+==========
+
+No notable changes.
+
+Modules
+=======
+
+Major changes in popular modules are detailed here
+
+No notable changes.
+
+Modules removed
+---------------
+
+The following modules no longer exist:
+
+* None
+
+Deprecation notices
+-------------------
+
+The following modules will be removed in Ansible 2.8. Please update update your playbooks accordingly.
+
+* :ref:`fixme <fixme>`
+
+Noteworthy module changes
+-------------------------
+
+No notable changes.
+
+Plugins
+=======
+
+No notable changes.
+
+Porting custom scripts
+======================
+
+No notable changes.
+
+Networking
+==========
+
+
+Change in deprecation notice of top-level connection arguments
+--------------------------------------------------------------
+.. code-block:: yaml
+
+    - name: example of using top-level options for connection properties
+      ios_command:
+        commands: show version
+        host: "{{ inventory_hostname }}"
+        username: cisco
+        password: cisco
+        authorize: yes
+        auth_pass: cisco
+
+**OLD** In Ansible 2.4:
+
+Will result in:
+
+.. code-block:: yaml
+
+   [WARNING]: argument username has been deprecated and will be removed in a future version
+   [WARNING]: argument host has been deprecated and will be removed in a future version
+   [WARNING]: argument password has been deprecated and will be removed in a future version
+
+
+**NEW** In Ansible 2.5:
+
+
+.. code-block:: yaml
+
+   [DEPRECATION WARNING]: Param 'username' is deprecated. See the module docs for more information. This feature will be removed in version
+   2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
+   [DEPRECATION WARNING]: Param 'password' is deprecated. See the module docs for more information. This feature will be removed in version
+   2.9. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
+   [DEPRECATION WARNING]: Param 'host' is deprecated. See the module docs for more information. This feature will be removed in version 2.9.
+   Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
+

docs/docsite/rst/porting_guides.rst

@@ -10,3 +10,4 @@ This section lists porting guides that can help you playbooks, plugins and other
 * :ref:`Ansible 1.x to 2.0 porting guide <porting_2.0_guide>`
 * :ref:`Ansible 2.2 to 2.3 porting guide <porting_2.3_guide>`
 * :ref:`Ansible 2.3 to 2.4 porting guide <porting_2.4_guide>`
+* :ref:`Ansible 2.4 to 2.5 porting guide <porting_2.5_guide>`

docs/docsite/rst/python_3_support.rst

@@ -2,7 +2,7 @@
 Python 3 Support
 ================
 
-Ansible 2.2 features a tech preview of Python 3 support. This topic discusses how you can test to make sure your modules and playbooks work with Python 3.
+Ansible 2.2, 2.3, and 2.4 feature a tech preview of Python 3 support. This topic discusses how you can test to make sure your modules and playbooks work with Python 3.
 
 .. note:: Ansible supports Python version 3.5 and above only.
 
@@ -15,25 +15,32 @@ Ansible 2.2 features a tech preview of Python 3 support. This topic discusses ho
 Testing Python 3 with commands and playbooks
 --------------------------------------------
 
-* `Run Ansible 2.2+ :ref:`from_source`
+* Run Ansible 2.2+ :ref:`from source <from_source>`
 * To test Python 3 on the controller, run your ansible command via
   ``python3``. For example:
 
 .. code-block:: shell
 
-    python3 /usr/bin/ansible localhost -m ping
-    python3 /usr/bin/ansible-playbook sample-playbook.yml
+    $ python3 /usr/bin/ansible localhost -m ping
+    $ python3 /usr/bin/ansible-playbook sample-playbook.yml
 
 You can also install Ansible using :program:`pip` for Python3 which will make the default
 :command:`/usr/bin/ansible` run with Python3:
 
 .. code-block:: shell
 
-    $ virtualenv py3-ansible
-    $ source ./bin/activate
+    $ virtualenv --python=python3 py3-ansible
+    $ source ./py3-ansible/bin/activate
     $ pip3 install ansible
-    $ ansible --version|grep python
-    python version = 3.5.3 (default, May 10 2017, 15:05:55) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
+    $ ansible --version | grep "python version"
+      python version = 3.6.2 (default, Sep 22 2017, 08:28:09) [GCC 7.2.1 20170915 (Red Hat 7.2.1-2)]
+
+On systems with SELinux installed, such as Red Hat Enterprise Linux or Fedora, the SELinux Python libraries also need to be copied over.
+
+.. code-block:: shell
+
+    $ cp -r -v /usr/lib64/python3.*/site-packages/selinux/ ./py3-ansible/lib64/python3.*/site-packages/
+    $ cp -v /usr/lib64/python3.*/site-packages/*selinux*.so ./py3-ansible/lib64/python3.*/site-packages/
 
 .. note:: Individual Linux distribution packages may be packaged for Python2 or Python3.  When running from
     distro packages you'll only be able to use Ansible with the Python version for which it was
@@ -58,14 +65,14 @@ Testing Python 3 module support
     [py3-hosts:vars]
     ansible_python_interpreter=/usr/bin/python3
 
-  See the :ref:`inventory documentation <inventory>` for more information.
+See the :ref:`inventory documentation <inventory>` for more information.
 
 * Run your command or playbook:
 
 .. code-block:: shell
 
-    ansible localhost-py3 -m ping
-    ansible-playbook sample-playbook.yml
+    $ ansible localhost-py3 -m ping
+    $ ansible-playbook sample-playbook.yml
 
 
 Note that you can also use the :option:`-e` command line option to manually
@@ -73,8 +80,8 @@ set the python interpreter when you run a command. For example:
 
 .. code-block:: shell
 
-  ansible localhost -m ping -e 'ansible_python_interpreter=/usr/bin/python3'
-  ansible-playbook sample-playbook.yml -e 'ansible_python_interpreter=/usr/bin/python3'
+    $ ansible localhost -m ping -e 'ansible_python_interpreter=/usr/bin/python3'
+    $ ansible-playbook sample-playbook.yml -e 'ansible_python_interpreter=/usr/bin/python3'
 
 What to do if an incompatibility is found
 -----------------------------------------

docs/docsite/rst/release_and_maintenance.rst

@@ -24,18 +24,20 @@ latest features and security fixes.
 Older unsupported versions of Ansible can contain unfixed security
 vulnerabilities (*CVE*).
 
-You can refer to the `porting guide`_ for tips on updating your Ansible
+You can refer to the `porting guides`_ for tips on updating your Ansible
 playbooks to run on newer versions.
 
-.. _porting guide: https://docs.ansible.com/ansible/porting_guide_2.0.html
+.. _porting guides: https://docs.ansible.com/ansible/porting_guides.html
 
 Release status
 ``````````````
 
 +-----------------+----------------------------+----------------------------------------+
-| Ansible release | Latest version             | Status                                 |
+| Ansible Release | Latest Version             | Status                                 |
 +=================+============================+========================================+
-| devel           | `2.4`_ (unreleased, trunk) | In development                         |
+| devel           | `2.5`_ (unreleased, trunk) | In development                         |
++-----------------+----------------------------+----------------------------------------+
+| 2.4             | `2.4.x`_ (2017-09-19)      | Supported (bug **and** security fixes) |
 +-----------------+----------------------------+----------------------------------------+
 | 2.3             | `2.3.2`_ (2017-08-08)      | Supported (bug **and** security fixes) |
 +-----------------+----------------------------+----------------------------------------+
@@ -50,7 +52,8 @@ Release status
 | <1.9            | n/a                        | Unsupported (end of life)              |
 +-----------------+----------------------------+----------------------------------------+
 
-.. _2.4: https://github.com/ansible/ansible/blob/devel/CHANGELOG.md
+.. _2.5: https://github.com/ansible/ansible/blob/devel/CHANGELOG.md
+.. _2.4.x: https://github.com/ansible/ansible/blob/stable-2.4/CHANGELOG.md
 .. _2.3.2: https://github.com/ansible/ansible/blob/stable-2.3/CHANGELOG.md
 .. _2.2.3: https://github.com/ansible/ansible/blob/stable-2.2/CHANGELOG.md
 .. _2.1.6: https://github.com/ansible/ansible/blob/stable-2.1/CHANGELOG.md

docs/docsite/rst/roadmap/ROADMAP_2_1.rst

@@ -95,11 +95,11 @@ Expand module diff support (already in progress in devel)
 
 Other
 -----
-Things being kicking down the road that we said we’d do
+Things being kicking down the road that we said we'd do
 
 - NOT remerging core with ansible/ansible this release cycle
 
 Community
 ---------
-- Define the process/ETA for reviewing PR’s from community
+- Define the process/ETA for reviewing PR's from community
 - Publish better docs and how-tos for submitting code/features/fixes

docs/docsite/rst/roadmap/ROADMAP_2_2.rst

@@ -19,7 +19,7 @@ Extras split from Core
 ----------------------
 Lead by Jason M and Jimi-c (Targeting 2.2, could move into 2.3).
 
-Targeted towards the 2.2 release or shortly after, we are planning on splitting Extras out of the “Ansible Core” project.  That means that modules that are shipped with Ansible by default are **only** the modules in ansibl-modules-core.  Ansible extras will become a separate project, managed by the community standard.  Over the next few months we’re going to have a lot of work to do on getting all of the modules in the right places for this to work.
+Targeted towards the 2.2 release or shortly after, we are planning on splitting Extras out of the "Ansible Core" project.  That means that modules that are shipped with Ansible by default are **only** the modules in ansibl-modules-core.  Ansible extras will become a separate project, managed by the community standard.  Over the next few months we're going to have a lot of work to do on getting all of the modules in the right places for this to work.
 
 - Create proposal (Jason or Jimi)
 - Review modules for correct location (extras v core)
@@ -40,7 +40,7 @@ AWS
 ---
 Lead by Ryan Brown
 
-- Pagination for all AWS modules (generic pagination exists, but isn’t used everywhere) (bumped to 2.3)
+- Pagination for all AWS modules (generic pagination exists, but isn't used everywhere) (bumped to 2.3)
 - Refactoring ec2.py to be more digestible (bumped to 2.3)
 - Fix inconsistencies with different authentication methods (STS, environment creds, ~/.aws/credentials) (done)
 - AWS Lambda modules (lambda_execute done, others pending)
@@ -85,7 +85,7 @@ Lead by Matt D
 
 - Feature parity
 
-  - PS module API (mirror Python module API where appropriate). Note: We don’t necessarily like the current python module API (AnsibleModule is a huge class with many unrelated utility functions.  Maybe we should redesign both at the same time?) (bumped to 2.3+ due to "moving target" uncertainty)
+  - PS module API (mirror Python module API where appropriate). Note: We don't necessarily like the current python module API (AnsibleModule is a huge class with many unrelated utility functions.  Maybe we should redesign both at the same time?) (bumped to 2.3+ due to "moving target" uncertainty)
   - Environment keyword support (done)
   - win_shell/win_command (done)
   - Async support (done)
@@ -117,7 +117,7 @@ Lead by Nate C, Peter S
 
 Role revamp
 -----------
-- Implement ‘role revamp’ proposal to give users more control on role/task execution (Brian)
+- Implement 'role revamp' proposal to give users more control on role/task execution (Brian)
 
   - **https://github.com/ansible/proposals/blob/master/roles_revamp.md**
 
@@ -125,13 +125,13 @@ Vault
 -----
 Lead by Jtanner, Adrian
 
-- *Extend ‘transparent vault file usage’ to other action plugins other than 'copy'(https://github.com/ansible/ansible/issues/7298)*
+- *Extend 'transparent vault file usage' to other action plugins other than 'copy'(https://github.com/ansible/ansible/issues/7298)*
   **done:** https://github.com/ansible/ansible/pull/16957
-- Add ‘per variable’ vault support (!vault YAML directive, existing PR already) https://github.com/ansible/ansible/issues/13287 https://github.com/ansible/ansible/issues/14721
+- Add 'per variable' vault support (!vault YAML directive, existing PR already) https://github.com/ansible/ansible/issues/13287 https://github.com/ansible/ansible/issues/14721
 - Add vault/unvault filters https://github.com/ansible/ansible/issues/12087 (deferred to 2.3)
 - Add vault support to lookups (likely deferred to 2.3 or until lookup plugins are revamped)
 - Allow for multiple vault secrets https://github.com/ansible/ansible/issues/13243
-- Config option to turn ‘unvaulting’ failures into warnings https://github.com/ansible/ansible/issues/13244
+- Config option to turn 'unvaulting' failures into warnings https://github.com/ansible/ansible/issues/13244
 
 Python3
 -------
@@ -139,7 +139,7 @@ Lead by Toshio
 
 A note here from Jason M: Getting to complete, tested Python 3 is both
 a critical task and one that has so much work and so many moving parts
-that we don’t expect this to be complete by the 2.2 release.  Toshio will
+that we don't expect this to be complete by the 2.2 release.  Toshio will
 lead this overall effort.
 
 - Motivation:
@@ -173,7 +173,7 @@ lead this overall effort.
     - Update: copy of the six library (v1.4.1 for python2.4 compat) and unicode helpers are here (ansible.module_utils._text.{to_bytes,to_text,to_native})
   - A few basic modules ported to python3
 
-    - Stat module best example module since it’s essential.
+    - Stat module best example module since it's essential.
     - Update:
 
       - A handful of modules like stat have been line-by-line ported.  They should work reliably with few python3-specific bugs.  All but three integration tests pass which means that most essential modules are working to some extent on Python3.
@@ -186,7 +186,7 @@ lead this overall effort.
     - lib/ansible/* and all modules now compile under Python-3.5
 
   - Side work to do:
-    - Figure out best ways to run unit-tests on modules.  Start unit-testing modules.  This is going to become important so we don’t regress python3 or python2.4 support in modules  (Going to largely punt on this for 2.2.  Matt Clay is working on building us a testing foundation for the first half of 2.2 development so we’ll re-evaluate towards the middle of the dev cycle).
+    - Figure out best ways to run unit-tests on modules.  Start unit-testing modules.  This is going to become important so we don't regress python3 or python2.4 support in modules  (Going to largely punt on this for 2.2.  Matt Clay is working on building us a testing foundation for the first half of 2.2 development so we'll re-evaluate towards the middle of the dev cycle).
     - More unit tests of module_utils
     - More integration tests.  Currently integration tests are the best way to test ansible modules so we have to rely on those.
 
@@ -202,7 +202,7 @@ Infrastructure Buildout and Changes
 -----------------------------------
 Lead by Matt Clay
 
-Another note from Jason M: A lot of this work is to ease the burden of CI, CI performance, increase our testing coverage and all of that sort of thing.  It’s not necessarily feature work, but it’s \*\*critical\*\* to growing our product and our ability to get community changes in more securely and quickly.
+Another note from Jason M: A lot of this work is to ease the burden of CI, CI performance, increase our testing coverage and all of that sort of thing.  It's not necessarily feature work, but it's \*\*critical\*\* to growing our product and our ability to get community changes in more securely and quickly.
 
 - **CI Performance**
   Reduce time spent waiting on CI for PRs. Combination of optimizing existing Travis setup and offloading work to other services. Will be impacted by available budget.

docs/docsite/rst/roadmap/ROADMAP_2_3.rst

@@ -46,7 +46,7 @@ Lead by nitzmahone
   - Become support **(done/experimental)**
   - Integrated kerberos ticket management (via ansible_user/ansible_password) **(done)**
   - Switch PS input encoding to BOM-less UTF8 **(done)**
-  - Server 2016 support/testing (now RTM’d) **(partial)**
+  - Server 2016 support/testing (now RTM'd) **(partial)**
   - Modularize Windows module_utils (allow N files) **(partial)**
   - Declarative argspec for PS / .NET **(bumped to 2.4)**
   - Kerberos encryption (via notting, pywinrm/requests_kerberos/pykerberos) **(in progress, available in pywinrm post 2.3 release)**
@@ -98,8 +98,8 @@ Python3
 
   - If users report bugs on python3, these should be fixed and will prioritize our work on porting other modules.
 
-- Still have to solve the python3-only and python2-only modules.  Thinking of doing this via metadata.  Will mean we have to use metadata at the module_common level.  Will also mean we don’t support py2-only or py3-only old style python modules.
-- Note: Most of the currently tested ansible features now run.  But there’s still a lot of code that’s untested.
+- Still have to solve the python3-only and python2-only modules.  Thinking of doing this via metadata.  Will mean we have to use metadata at the module_common level.  Will also mean we don't support py2-only or py3-only old style python modules.
+- Note: Most of the currently tested ansible features now run.  But there's still a lot of code that's untested.
 
 Testing and CI
 --------------
@@ -157,7 +157,7 @@ Plugin Loader
 
 ansible-ssh
 -----------
-- Add a ‘ansible-ssh’ convenience and debugging tool (will slip to 2.4)
+- Add a 'ansible-ssh' convenience and debugging tool (will slip to 2.4)
 - Tool to invoke an interactive ssh to a host with the same args/env/config that ansible would.
 - There are at least three external versions