If we're not delegating then we change _connection into a local
connection midway through the file but we don't change
_play_context.connection (no need to alter that). When we later check
it in process_remote() we need to know the actual connection, not the
connection that we thought we were going to use at the start of run().
So we have to use _connection.transport in process_remote(). The rest
of the places could use either one (because we have not yet changed to
a local connection) but we go ahead and switch those to
_connection.transport as well to avoid confusion in the future.
Fixes https://github.com/ansible/ansible-modules-core/issues/3136
Previously, split_args() was not taking print/block/comment depth into account
when splitting things, meaning that if there was a quote character inside an
un-quoted variable (ie. {{ foo | some_filter(' ') }}), it was incorrectly
splitting on the quotes instead of continuing to append to the previous param.
Fixes#13630
* Fix the way task_include fields were created and copied
* Have blocks get_dep_chain() look at task_include's blocks for proper
dep chain inheritance
* Fix the way task_include fields are copied to prevent a recursive
degradation
Fixes#14460
* Make role param resolution follow the role dependency chain, rather
than using all roles within the play
* Also move params to be merged in after role vars in general, to match
our variable precedence rules
* Changes to the way var lookup is done in role helper methods for
get_vars(), get_inherited_vars(), and get_role_params() to make the
above possible without trampling on previously merged vars
Fixes#14472Fixes#14475
On some switches, starting an SSH connection and immediately calling
recv() will result in a connection timeout. The switch requires some
input on the channel before it provides any prompt.
As such, this patch sends an empty command immediately upon connection,
triggering the switch to send a prompt which the shell can then
interpret.
Signed-off-by: Devananda van der Veen <devananda.vdv@gmail.com>
* Unit tests exposed a problem where nested blocks did not correctly
hit rescue/always portions of parent blocks
* Cleaned up logic in PlayIterator
* Unfortunately fixing the above exposed a potential problem in the
block integration tests, where a failure in an "always" section may
always lead to a failed state and the termination of execution
beyond that point, so certain parts of the block integration test
were disabled.
* fetch_url shouldn't both accept follow_redirects and support follow_redircts via module.params
* Default follow_redirects for open_url should be 'urllib2'
* Add redirect test for get_url
main_q is not used anywhere in the codebase.
It is created in TaskQueueManager._initialize_processes, bundled with rslt_q
into TaskQueueManger._workers, later unwrapped in StrategyBase but not used.
This queue is closed in TaskQueueManger._cleanup_processes.
Historically, it is passed as a init parameter into WorkerProcess,
introduced in 62d7956, but this behavior is changed in 120b9a7.
Signed-off-by: 夏恺(Xia Kai) <xiaket@gmail.com>
This is the same fix we applied to v1.9 in PR #14565, however it does not fix#14678 completely !
The dictionaries are not being merged as tey are on v1.9.
On none-Linux systems `get_distribution()` returns `None`, which fails in `fetch_url`, because the return value of `get_distribution()` is not checked before calling `lower()` on the result.
Note that this will break if we deal with non-utf8 paths. Fixing this
way because converting everythig to byte strings instead is a very
invasive task so it should be done as a specific feature to provide
support for non-utf8 paths at some point in the future (if needed).
The use of realpath means when following symlinks the actual path is
used when loading these files in the VariableManager, which may not
line up with the host or group name specified.
Fixes#14545
- now workers passes queue to task_executor so it can send back events per item and on retry attempt
- updated result class to pass along events to strategy
- base strategy updated to forward new events to callback
- callbacks now remove 'items' on final result but process them directly when invoked per item
- new callback method to deal with retry attempt messages (also now obeys nolog)
- updated tests to match new signature of task_executor
fixes#14558fixes#14072
* Fixes bug where the task was not marked as failed if the number of
retries were exceeded (#14461)
* Reorganizing logic to be a bit cleaner, and so retrie messages are
shown before sleeping (which makes way more sense)
Fixes#14461Fixes#14580
Prior to 75b6f61, we strictly limited variables we re-injected. After that
patch however, we re-injected everything which causes problems under certain
circumstances. For now, we'll continue to filter out some properties of
PlayContext for re-injection.
Fixes#14352
This is related to #14559, but only the part for Ansible v2.0
This commit makes merging empty dicts, or equal dicts more efficient.
I noticed that while debugging merge_hash a lot of merges related to empty dictionaries and sometimes also identical dictionaries.
will display on certain verbosity levels, both playbook/file info
and non empty options with which it's running.
avoid errors when not using CLI classes
The setup module calls /bin/lsblk once for each device appearing in the /etc/mtab file. However, the same device appears there mutliple times when the system uses bind-mounts. As a result, /bin/lsblk is being called repeatedly to get the uuid of the same device.
On a system with many mounts, this leads to a TimeoutError in the get_mount_facts function of the setup module as described in #14551.
Fixes#14551
ansible_os_family on openSUSE Leap has the wrong value:
"ansible_os_family": "openSUSE Leap",
It should be:
"ansible_os_family": "Suse",
This change fixes that by adding the relevant key and ensuring that dict
lookups replace ' ' with '_' so the key does not contain a space.
- added new function for action plugins this avoids the very fragile checksum code that is shell dependant.
- ported copy module to it
- converted assemble to new stat function
- some corrections and ported temlpate
- updated old checksum function to use new stat one under the hood
- documented revamped remote checksum method
* Make sure dep chains are checked recursively for nested blocks
* Fixing iterator is_failed() check to make sure we're not in a
rescue block before returning True
* Use is_failed() to test whether a host should be added to the TQM
failed_hosts list
* Use is_failed() when compiling the list of hosts left to iterate
over in both the linear and free strategies
Fixes#14222
* Raise an error if the action is using BYPASS_HOST_LOOP, to prevent
unexpected behavior from those actions
* Show a warning regarding tasks marked as run_once, as the free strategy
does not yet support that behavior
* Minor tweak to linear strategies run_once code to make sure we don't
raise an error if an action isn't found
* If the internal value is None, do not add the variable
* Make sure all aliases for a given variable name are set (if they're
not already set in the dictionary)
Fixes#14310
* Fixed a bug in PlayIterator when ITERATING_ALWAYS, where the block
was advanced but the incorrect data structure elements were cleared
* Cleaned up the logic of is_failed() in PlayIterator
* Fixed a bug in the free strategy which had not been updated to use
the base strategy _execute_meta() method
* Stopped strategies from using is_failed() to determine if tasks should
still be fetched for a host
Fixes#14040
The correct default options for sudo_flags can be found at: https://github.com/ansible/ansible/blob/devel/lib/ansible/constants.py#L181
Slightly alter explanation of '-H' so as not to confuse it with -E, --preserve-env (which preserves existing environment variables).
When adding the two other options, include short explanations of those options.
Add note about '-n', which did not appear in 1.x I believe, and which bit me.
* Fix the way task_include fields were created and copied
* Have blocks get_dep_chain() look at task_include's blocks for proper
dep chain inheritance
* Fix the way task_include fields are copied to prevent a recursive
degradation
Fixes#14460
* Make sure dep chains are checked recursively for nested blocks
* Fixing iterator is_failed() check to make sure we're not in a
rescue block before returning True
* Use is_failed() to test whether a host should be added to the TQM
failed_hosts list
* Use is_failed() when compiling the list of hosts left to iterate
over in both the linear and free strategies
Fixes#14222
This reverts commit 9fae8501e2.
Reverting for 2.0.1 -- this has been broken in the 2.0 tree for long
enough that some modules are relying on the broken behaviour. This fix
will definitely return for 2.1.0 and may return in 2.0.2 (impact needs
to be evaluated).
- moved to base cli class to handle centrally and duplicate less code
- now avoids duplication and reiteration of signal handler by reassigning it
- left note on how to do non-graceful in case we add in future
as I won't remember everything i did here and don't want to 'relearn' it.
- adhoc now terminates gracefully
- avoid race condition on terminations by ignoring errors if
worker might have been reaped between checking if active and termination call
- ansible-playbook now properly exits on sigint/term
- adhoc and playbook now give exceptions that we should not normally capture
and rely on top level finally to reap children
- handle systemexit breaks in workers
- added debug to see at which frame we exit
partial fix for #14346
This reverts commit 2126e9fee1.
we judge this to be too big a change for stable, keeping in devel
we may still add part or all back to stable in the future.
now deprecation message appears with variable name in all spots where this occurs
debug's var= option is excluded as this is only place where bare variables shold actually
be accepted.
it was assumed it could only be a dict or string (it starts out as a list)
also a 2nd assumption that bare vars only would appear in one of the dict keys.
removed deprecation warnings from here as they should be signaled in the bare conversion itself.
So far, when a 'diff' dict is returned with module results, it is
checked for 'before' and 'after' texts, which are processed in
_get_diff() by python difflib. This generates the changes to display
when CLI users specify --diff.
However, some modules will generate changes that cannot easily be
expressed in a conventional diff. One example is the output of the
synchronize module, which presents changed files in a common log format
as in `rsync --itemize-changes`.
Add a check for a diff['prepared'] key, which can contain prepared diff text
from modules.
Role definitions typically require params to be different from those
which are specified as FieldAttributes on the playbook classes used
for roles, however a certain subset should be allowed (typically those
used for connection stuff).
Fixes#14095
The dep chain for roles created during the compile step had bugs, in
which the dep chain was overwriten and the original tasks in the role
were not assigned a dep chain. This lead to problems in determining
whether roles had already run when in a "diamond" structure, and in
some cases roles were not correctly getting variables from parents.
Fixes#14046
The module docs and vault changes solve issues where tracebacks can
happen. The galaxy changes are mostly refactoring to be more pythonic
with a small chance that a unicode traceback could have occurred there
without the changes. The change in __init__.py when we actually call
the pager makes things more robust but could hide places where we had
bytes coming in already so I didn't want to change that without auditing
where the text was coming from.
Fixes#14178
* In 2.0.0.x become was reversed for synchronize. It was happening on
the local machine instead of the remote machine. This restores the
ansible-1.9.x behaviour of doing become on the remote machine.
However, there's aspects of this that are hacky (no hackier than
ansible-1.9 but not using 2.0 features). The big problem is that it
does not understand any become method except sudo. I'm willing to use
a partial fix now because we don't want people to get used to the
reversed semantics in their playbooks.
* synchronize copying to the wrong host when inventory_hostname is
localhost
* Fix problem with unicode arguments (first seen as a bug on synchronize)
Fixes#14041Fixes#13825
This prevents a bug where the existing cache outside of the class
is not cleared when creating a new Inventory object. This only really
affects people using the API directly right now, but wanted to fix it
to prevent weird errors from popping up.
Instead of bombing out of the strategy, we now properly mark hosts failed
so that the play iterator can handle block rescue/always properly.
Fixes#14024
When using a playbook-level include, we now catch any errors raised during
the conditional evaluation step and set a flag to indicate we need to pass
those conditionals on to the included play (most likely because they contain
inventory variables for evaluation).
Fixes#14003
This causes problems when fetching parent attributes, as the include
was being skipped because the parent block would fetch the attribute
from the parent play first.
Fixes#13872
this was taken out in an effort to default to the user's shell but creates issues as this is not known ahead of time
and its painful to set executable and shell_type for all servers, it should only be needed for those that restrict the user
to specific shells and when /bin/sh is not available. raw and command may still bypass this by explicitly passing None.
fixes#13882
still conditional
* Don't re-use the existing connection if the remote_addr field of
the play context has changed
* When overriding variables in PlayContext (from task/variables),
don't set the same attribute based on a different variable name
if we had already previously set it from another variable name
Fixes#13880
by moving to en-bloc unicode conversion to act on scripts stdout
Both python-json and simplejson always return unicode strings when using
their loads() method on unicode strings. This is true at least since
2009. This makes checking each substring unnecessary, because we do not
need to recursively check the strings contained in the inventory dict
later one-by-one
This commit makes parsing of large dynamic inventory at least 2 seconds
faster.
cf: https://github.com/towolf/ansible-large-inventory-testcase
Letting it pass would just cause an error later on (no such file found)
so it's better to catch it here and know that we have users dealing with
non-utf8 pathnames than to have to track it down from later on.
Note that the fix for display normalizing to unicode is correct but the
fix for pathnames is probably not. Changing pathnames to unicode type
means that we will handle utf8 pathnames fine but pathnames can be any
sequence of bytes that do not contain null. We do not handle sequences
of bytes that are not valid utf8 here. To do that we need to revamp the
handling of basedir and paths to transform to bytes instead of unicode.
Didn't want to do that in 2.0.x as it will potentially introduce other
bugs as we find all the places that we combine basedir with other path
elements. Since no one has raised that as an issue thus far so it's not
something we need to handle yet. But it's something to keep in mind for
the future.
To test utf8 handling, create a utf8 directory and run a playbook from
within there.
To test non-utf8 handling (currently doesn't work as stated above), create
a directory with non-utf8 chars an run a playbook from there. In bash,
create that directory like this: mkdir $'\377'
Fixes#13937
* Relocate the assignment of the host address to the remote_addr field
in the play context, which was only done when the connection was created
(it's now done after the post_validate() is called on the play context)
* Make the assignment of the play context to the connection an else, since
it's not required if the connection is not reused
If this isnt updated, the _connection is reused, and thus has an outdated _play_context
This results in outdated `success_key` and `prompt` causing issues if sudo is run in a loop
Refer to the issue #13763 for more debugging and details
This is because we pass arguments to non-newstyle modules via an
external file. If we pipeline, then the interpreter thinks it has to
run the arguments as the script instead of what is piped in via stdin.
* Added additional methods to the iterator code to assess host failures
while also taking into account the block rescue/always states
* Fixed bugs in the free strategy, where results were not always being
processed after being collected
* Added some prettier printing to the state output from iterator
Fixes#13699
and without hosts and vars
Without this patch, the simplified syntax is triggered when a group
is defined like this:
"platforms": {
"children": [
"cloudstack"
]
}
Which results in a group 'platforms' with 1 host 'platforms'.
more details in https://github.com/ansible/ansible/issues/13655
The nxapi module has been superseded by the nxos shared module and is not longer needed. This commit removes (deletes) nxapi from module_utils. All custom modules that have used nxapi should be using nxos instead.
This commit adds a new shared module that parses network device configuration
files. It is used to build modules that work with the various supported
network device operating systems
This commit add a new shared module shell that is used to build connections
to network devices that operate in a CLI environment. This commit supercedes
the issh.py and cli.py commits and removes them from module_utils.
This commit adds a new shared module openswitch for building modules that
work with OpenSwitch. This shared module supports connectivity to
OpenSwitch devices over SSH, CLI or REST. It also adds an openswitch
documentation fragment for use in modules
This commit refactors the nxapi into a new shared module nxos that supports
connectivity over both ssh (cli) and nxapi. It supercedes the nxapi shared
module and removes it from module_utils. This commit also adds a
documentation fragement supporting the nxos shared module
This commit adds a new shared module for working with Cisco IOS XR devices over
CLI (SSH). It also provides a documentation fragement for the commmon arguments
provided by the iosxr module.
This update refactor the ios shared module to use the new shell shared
library instead of issh and cli. It also adds the ios documentation
fragment to be used when building ios based modules.
This adds a shared module for communicating with Arista EOS devices over
SSH (cli) or JSON-RPC (eapi). This modules replaces the eapi.py module
previously added to module_utils. This commit includes a documentation
fragment that describes the eos common arguments
pushed it to use the existing propmpt from display and moved the vars prompt code there also for uniformity
changed vars_prompt to check extra vars vs the empty play.vars to restore 1.9 behaviour
sipmlified the code as it didn't need to check for syntax again (tqm is made none prior based on that)
fixes#13770
We were logging the command to be executed many times, which made debug
logs very hard to read. Now we do it only once.
Also makes the logged ssh command line cut-and-paste-able (the lack of
which has confused a number of people by now; the problem being that we
pass the command as a single argument to execve(), so it doesn't need an
extra level of quoting as it does when you try to run it by hand).
* Fix to error if validate_cert is True and python doesn't support it.
* Only globally disable certificate checking if really needed. Use
bigip verify parameter if available instead.
* Remove public disable certificate function to make it less likely
people will attempt to reuse that
* now module errors clearly state msg=MODULE FAILURE
* module's stdout and stderr go into module_stdout and module_stderr keys
which only appear during parsing failure
* invocation module_args are deleted from results provided by action
plugin as errors can keep us from overwriting and then disclosing info that
was meant to be kept hidden due to no_log
* fixed invocation module_args set by basic.py as it was creating different
keys as the invocation in action plugin base.
* results now merge
Also adds a new var, used by the prepare_tests role, to prevent it from
deleting the temp test directory at the start of each play to avoid any
potential race conditions
* Changed parse_addresses to throw exceptions instead of passing None
* Switched callers to trap and pass through the original values.
* Added very verbose notice
* Look at deprecating this and possibly validate at plugin instead
fixes#13608
Because the fail_state is potentially non-zero in these block sections,
the prior logic led to included tasks not being inserted at all.
Related issue: #13605
This was added in 1.9 and 2.0 tried to copy, but since it cannot
obey no_log restrictions I commented it out. I did not remove as
it is still very useful for module invocation debugging.
For the deps setup of integration tests, as we sometimes see odd
errors we can't reproduce, which may be related to slightly out of
date package dependencies.
* Saving of the registered variable was occuring after the tests for
changed/failed_when.
* Each of the above fields and until were being post_validated too early,
so variables which were not defined at that time were causing task
failures.
Fixes#13591
Environments were not being templated individually, so a variable environment
value was causing the exception regarding dicts to be hit. Also, environments
as inherited were coming through with the tasks listed first, followed by the
parents, so they were being merged backwards. Reversing the list of environments
fixed this.
Also fixes a bug where we were passing an incorrect number of parameters to
_do_handler_run() when processing an include file in a handler task/block.
Fixes#13560
Otherwise, each relative include path is checked on its own, rather
than in relation to the (possibly relative) path of its parent, meaning
includes multiple level deep may fail to find the correct (or any) file.
Fixes#13472
The current ssh shared module forces only password based authentication. This
change will allow the ssh module to use keys if a password is not provided.
The OpenStack inventory lists hostnames as the UUIDs because hostsnames
are not guarnateed to be unique on OpenStack. However, for the common
case, this is just confusing.
The new behavior is a visible change, so make it an opt-in via config.
Only turn the hostnames to UUIDs if there are duplicate hostnames.
moved from the field attribute declaration and created a placeholder
which then is resolved in the field attribute class.
this is to avoid unwanted persistent of the defaults across objects which introduces
stealth bugs when multiple objects of the same kind are used in succession while
not overriding the default values.
OS X El Capitan moved the /etc/ssh_* files into /etc/ssh/. This fix
adds a distribution version check for Darwin to set the keydir
appropriately on El Capitan and later.
The `webdocs` make target fails under Python 3. It fails due to a variety of
syntax errors, such as the use of `except Foo, e` and `print 'foo'`. Fix#13463
by making code compatible with both Python 2 and 3.
This patch fixes a bug in module_utils/ios.py where the the wrong shared
module arguments are being generated. This bug prevented the shared module
from operating correctly. This patch should be generally applied.
tasks were overriding commandline with their defaults, not with the
explicit setting, removed the setting of defaults from task init and
pushed down to play context at last possible moment.
fixes#13362
Ansible previously added hosts to the host list multiple times for commands
like `ansible -i 'localhost,' -c local -m ping 'localhost,localhost'
--list-hosts`.
8d5f36a fixed the obvious error, but still added the un-deduplicated list to a
cache, so all future invocations of get_hosts() would retrieve a
non-deduplicated list.
This caused problems down the line: For some reason, Ansible only ever
schedules "flush_handlers" tasks (instead of scheduling any actual tasks from
the playbook) for hosts that are contained in the host lists multiple times.
This probably happens because the host states are stored in a dictionary
indexed by the hostnames, so duplicate hostname would cause the state to be
overwritten by subsequent invocations of … something.
* sudo was not working, now it supports full become
* now default checkout dir works, not only when specifying
* paths for checkout dir get expanded
* fixed limit options for playbook
* added verbose and debug info
This should fix issues with fish shell users as && and || are
not valid syntax, fish uses actual 'and' and 'or' programs.
Also updated to allow for fish backticks pushed quotes to subshell,
fish seems to handle spaces w/o them.
Lastly, removed encompassing subshell () for fish compatibility.
fixes#13199
* Move self._tqm.load_callbacks() earlier to ensure that v2_on_playbook_start can fire
* Pass the playbook instance to v2_on_playbook_start
* Add a _file_name instance attribute to the playbook
The comment was taken literally from lib/plugins/strategy/linear.py and
makes no sense in free.py where we have no noop tasks.
Also update the debug messages.
This patch fixes an issue with the common args dict in the eapi shared
module. This patch is required for the eapi shared module to be properly
imported and is therefore should be applied to all instances.
This ssh shared module is used for building modules that require an
interactive shell environment such as those required for connecting
to network devices
This commit changes the way modules create an instance of AnsibleModule to
now use a common function, eapi_module. This function will now automatically
append the common argument spec to the module argument_spec. Module
arguments can override common module arguments
since all the --ask pass options end up triggering the same code
and are functionally equivalent, ignore them when it comes to checking
privilege escalation conflicts. This allows using -K when --become-method=su
and so on.
If we request escalation with a password, we start in expecting_prompt
state. If the escalation then succeeds without the password, i.e., the
become_success response arrives, we must explicitly move into the next
state (awaiting_escalation, which immediately goes into ready_to_send),
so that we no longer try to apply the timeout.
Otherwise, we would leak the success notification and eventually
timeout. But if the module response did arrive before the timeout
expired, the "process has already exited" test would do the right
thing by accident (which is why it didn't fail more often).
Fixes#13289
Modeled after the yum tests but also tests local package installations
as fixed with PR#1256.
This depends on PRs #1256, #1261 and #1262 in ansible-modules-extra.
This was caused by accessing the cache using the passed in mod_type
rather than the suffix that we calculate with knowledge of whether this
is a module or non-module plugin.
Please see the [Community Guide](https://docs.ansible.com/ansible/latest/community/index.html) for information on how to ask questions on the [mailing lists](https://docs.ansible.com/ansible/latest/community/communication.html#mailing-list-information) and IRC.
The GitHub issue tracker is not the best place for questions for various reasons, but both IRC and the mailing list are very helpful places for those things, as the community page explains best.
## CONTRIBUTING ?
By contributing you agree that these contributions are your own (or approved by your employer) and you grant a full, complete, irrevocable copyright license to all users and developers of the project, present and future, pursuant to the license of the project. You can also read the same [CLA](https://docs.ansible.com/ansible/latest/community/contributor_license_agreement.html) on the Ansible docsite.
Please review the [Community Guide](https://docs.ansible.com/ansible/latest/community/index.html) for more information on contributing to Ansible.
## BUG TO REPORT ?
First and foremost, also check the [Community Guide](https://docs.ansible.com/ansible/latest/community/index.html).
You can report bugs or make enhancement requests at the [Ansible GitHub issue page](http://github.com/ansible/ansible/issues/new/choose) by filling out the issue template that will be presented.
Also please make sure you are testing on the latest released version of Ansible or the development branch; see the [Installation Guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) for details.
Hi! Thanks for interest in contributing to Ansible.
Here are some guidelines for contributing code. The purpose of this document are to establish what we're looking for in code contributions, and to make sure
new contributions know some of the conventions that we've been using.
We don't think much of this should be too strange to readers familiar with contributing to Python projects, though it helps if we all get on the same page.
Language
========
* While not all components of Ansible must be in Python, core contributions to the Ansible repo must be written in Python. This is to maximize the ability of everyone to contribute.
* If you want to write non-Python ansible modules or inventory scripts, that's fine, but they are not going to get merged in most likely. Sorry!!
PEP8 and basic style checks
===========================
* PEP8 is a great Python style guide, which you should read.
* PEP8 must not be strictly followed in all aspects, but most of it is good advice
* In particular, we don't really care about line lengths. Buy a bigger monitor!
* To run checks for things we care about, run "make pep8"
* Similarly, additional checks can be made with "make pyflakes"
* There is no need to submit code changes for pep8 and pyflake fixes, as these break attribution history. Project leadership will make these periodically.
* Do not submit pull requests that simply adjust whitespace in the code
Testing
=======
* Much of ansible's testing needs are in integration, not unit tests. Add module tests there.
* That being said, there are unit tests too!
* Code written must absolutely pass tests (i.e. "make tests")
* You should anticipate any error paths in your code and test down those error paths.
* Additions to tests for core code is welcome, but not always possible. Be sure things are at least well tested manually in that case.
Whitespace
==========
* Four space indent is strictly required
* Include meaningful whitespace between lines of code
Shebang Lines
=============
* /usr/bin/scripts should start with '/usr/bin/env python'
* module code should still use '/usr/bin/python' as this is replaced automatically by 'ansible_python_interpreter', see the FAQ in the docs for more info.
Comments
========
* Readability is one of the most important goals for this project
* Comment any non-trivial code where someone might not know why you are doing something in a particular way
* Though if something should be commented, that's often a sign someone should write a function
* All new functions must have a basic docstring comment
* Commenting above a line is preferable to commenting at the end of a line
Classes
=======
* With the exception of module code (where inline is better), it is desirable to see classes in their own files.
* Classes should generally not cause side effects as soon as they are instantiated, move meaningful behavior to methods rather than constructors.
Functions and Methods
=====================
* In general, functions should not be 'too long' and should describe a meaningful amount of work
* When code gets too nested, that's usually the sign the loop body could benefit from being a function
* Parts of our existing code are not the best examples of this at times.
* Functions should have names that describe what they do, along with docstrings
* Functions should be named with_underscores
* "Don't repeat yourself" is generally a good philosophy
Variables
=========
* Use descriptive variable names instead of variables like 'x', unless x is a obvious loop index
* Ansible python code uses identifiers like 'ClassesLikeThis and variables_like_this
* Module parameters should also use_underscores and not runtogether
Module Security
===============
* Modules must take steps to avoid passing user input from the shell and always check return codes
* always use module.run_command instead of subprocess or Popen or os.system -- this is mandatory
* if you use need the shell you must pass use_unsafe_shell=True to module.run_command
* if you do not need the shell, avoid using the shell
* any variables that can come from the user input with use_unsafe_shell=True must be wrapped by pipes.quote(x)
* downloads of https:// resource urls must import module_utils.urls and use the fetch_url method
Misc Preferences
================
Use the dict constructor where possible when allocating dictionaries:
# not this:
foo = {
'a' : 12,
'b' : 34
}
# this:
foo = dict(
a = 12,
b = 34
)
Line up variables
# not this
a = 12
foosball = 34
xyz = 'dog'
# this
a = 12
foosball = 34
xyz = 'dog'
Don't use line continuations:
# no
if (this_is_a_very_long_line and foo and \
i_am_going_to_continue_it):
bar()
# better:
if (this_is_a_very_long_line and foo and i_am_going_to_continue_it):
bar()
Spacing:
# no
x=[1,2,3]
# no
x = [1,2,3]
# yes
x = [ 1, 2, 3 ]
Spacing continued:
# no
x=foo(12)
# no
x = foo( 12 )
# yes
x = foo(12)
Licenses
========
Every file should have a license header, including the copyright of the original author. Major additions to the module are allowed
to add an additional copyright line, and this is especially true of rewrites, but original authorship copyright messages should be preserved.
All contributions to the core repo should preserve original licenses and new contributions must include the GPLv3 header.
Module Documentation
====================
All module pull requests must include a DOCUMENTATION docstring (YAML format,
see other modules for examples) as well as an EXAMPLES docstring, which is free form.
When adding new modules, any new parameter must have a "version_added" attribute.
When submitting a new module, the module should have a "version_added" attribute in the
pull request as well, set to the current development version.
Be sure to check grammar and spelling.
It's frequently the case that modules get submitted with YAML that isn't valid,
so you can run "make webdocs" from the checkout to preview your module's documentation.
If it fails to build, take a look at your DOCUMENTATION string
or you might have a Python syntax error in there too.
Python Imports
==============
To make it clear what a module is importing, imports should not be sprinkled throughout the code.
Python Imports should happen at the top of the file, exempting code from module_utils.
When a conditional runtime import is required, do so something like this instead:
HAS_FOO = False
try:
import foo
HAS_FOO = True
except ImportError:
pass
...
if not HAS_FOO:
raise Exception("the foo library is required")
This makes it clear what optional dependencies are but allows this to be deferred until runtime. In the case of module code, the raising of the Exception will be replaced
with a "module.exit_json" call.
Exceptions
==========
In the main body of the code, use typed exceptions where possible:
# not this
raise Exception("panic!")
# this
from ansible import errors
...
raise errors.AnsibleError("panic!")
Similarly, exception checking should be fine grained:
# not this
try:
foo()
except:
bar()
# but this
try:
foo()
except SomeTypedException:
bar()
List Comprehensions
===================
In general list comprehensions are always preferred to map() and filter() calls.
However, they can be abused. Optimize for readability, and avoid nesting them too deeply.
Regexes
=======
There is a time and place for them, but here's an illustrative joke.
"A developer had a problem, and used a regular expression to solve it. Now the developer had two problems".
Often regexes are difficult to maintain, and a trusty call to other string operations can be a great solution, faster,
and more readable.
File Conventions
================
If a piece of code looks for a named YAML file in a directory, it should assume it can take no extension, or an extension of '.yml' or '.yaml'.
This should be true against all code that loads files.
Any code that uses directories should consider the possibility that the directory may be symlink.
New Ansible language parameters
===============================
If adding a new parameter, like 'can_fizzbuzz: True/False' be sure the value of the parameter is templated somewhere in the Runner code, as if anything can be parameterized in Ansible,
there is a user that will try to parameterize it.
String Find
===========
Use 'in':
# not this:
if x.find('foo') != -1:
# this:
if 'foo' in x:
String checks
=============
To test if something is a string, consider that it may be unicode.
# no
if type(x) == str:
# yes
if isinstance(x, basestring):
Cleverness
==========
Ansible's code is intended to be read by as many people as possible, so we don't particularly encourage clever or heavily idiomatic code.
In particular, metaclasses are probably not appropriate, however entertaining they may be to add.
Git Practices
=============
Pull requests cannot be accepted that contain merge commits.
Always do "git pull --rebase" and "git rebase" vs "git pull" or "git merge".
Always create a new branch for each pull request to avoid intertwingling different features or fixes on the same branch.
Python Version Compliance
=========================
All code in Ansible core must support a minimum version of Python 2.6.
Module code must support a minimum of Python 2.4, with occasional exception for modules that require code that themselves require 2.6 and later.
A quick reminder is that list comprehensions in Python 2.4 are not as fully fleshed out, there are no 'dict' comprehensions, and there is no 'with' statement.
But otherwise it's pretty much all the same.
The End
=======
This was not meant to be a scary document, so we hope it wasn't, but we also hope this helps you write code that is easier to maintain by others in the future.
If you have questions about this document, please ask on the ansible-devel mailing list.
Please see [this web page ](http://docs.ansible.com/community.html) for community information, which includes pointers on how to ask questions on the [mailing lists](http://docs.ansible.com/community.html#mailing-list-information) and IRC.
The github issue tracker is not the best place for questions for various reasons, but both IRC and the mailing list are very helpful places for those things, and that page has the pointers to those.
If you'd like to contribute code
================================
Please see [this web page](http://docs.ansible.com/community.html) for information about the contribution process. Important license agreement information is also included on that page.
If you'd like to file a bug
===========================
I'd also read the community page above, but in particular, make sure you copy [this issue template](https://github.com/ansible/ansible/blob/devel/ISSUE_TEMPLATE.md) into your ticket description. We have a friendly neighborhood bot that will remind you if you forget :) This template helps us organize tickets faster and prevents asking some repeated questions, so it's very helpful to us and we appreciate your help with it.
Also please make sure you are testing on the latest released version of Ansible or the development branch.
Can you help us out in labelling this by telling us what kind of ticket this this? You can say:
- Bug Report
- Feature Idea
- Feature Pull Request
- New Module Pull Request
- Bugfix Pull Request
- Documentation Report
- Docs Pull Request
##### Ansible Version:
Let us know what version of Ansible you are using. Please supply the verbatim output from running “ansible --version”. If you're filing a ticket on a version of Ansible which is not the latest, we'd greatly appreciate it if you could retest on the latest version first. We don't expect you to test against the development branch most of the time, but we may ask for that if you have cycles. Thanks!
##### Ansible Configuration:
What have you changed about your Ansible installation? What configuration settings have you changed/added/removed? Compare your /etc/ansible/ansible.cfg against a clean version from Github and let us know what's different.
##### Environment:
What OS are you running Ansible from and what OS are you managing? Examples include RHEL 5/6, Centos 5/6, Ubuntu 12.04/13.10, *BSD, Solaris. If this is a generic feature request or it doesn’t apply, just say “N/A”. Not all tickets may be about operating system related things and we understand that.
##### Summary:
Please summarize your request in this space. You will earn bonus points for being succinct, but please add enough detail so we can understand the request. Thanks!
##### Steps To Reproduce:
If this is a bug ticket, please enter the steps you use to reproduce the problem in the space below. If this is a feature request, please enter the steps you would use to use the feature. If an example playbook is useful, please include a short reproducer inline, indented by four spaces. If a longer one is necessary, linking to one uploaded to gist.github.com would be great. Much appreciated!
##### Expected Results:
Please enter your expected results in this space. When running the steps supplied above in the previous section, what did you expect to happen? If showing example output, please indent your output by four spaces so it will render correctly in GitHub's viewer thingy.
##### Actual Results:
Please enter your actual results in this space. When running the steps supplied above, what actually happened? If you are showing example output, please indent your output by four spaces so it will render correctly in GitHub. Thanks again!
tar -C deb-build/$${DIST} -xvf dist/$(NAME)-$(VERSION).tar.gz ;\
cp -a packaging/debian deb-build/$${DIST}/$(NAME)-$(VERSION)/ ;\
sed -ie "s|%VERSION%|$(DEB_VERSION)|g;s|%RELEASE%|$(DEB_RELEASE)|;s|%DIST%|$${DIST}|g;s|%DATE%|$(DEB_DATE)|g" deb-build/$${DIST}/$(NAME)-$(VERSION)/debian/changelog ;\
sed -ie "s|%VERSION%|$(VERSION)|g;s|%RELEASE%|$(DEB_RELEASE)|;s|%DIST%|$${DIST}|g;s|%DATE%|$(DEB_DATE)|g" deb-build/$${DIST}/$(NAME)-$(VERSION)/debian/changelog ;\
Ansible is a radically simple IT automation system. It handles configuration-management, application deployment, cloud provisioning, ad-hoc task-execution, and multinode orchestration - including trivializing things like zero downtime rolling updates with load balancers.
Read the documentation and more at http://ansible.com/
Many users run straight from the development branch (it's generally fine to do so), but you might also wish to consume a release.
You can find instructions [here](http://docs.ansible.com/intro_getting_started.html) for a variety of platforms. If you decide to go with the development branch, be sure to run `git submodule update --init --recursive` after doing a checkout.
If you want to download a tarball of a release, go to [releases.ansible.com](http://releases.ansible.com/ansible), though most users use `yum` (using the EPEL instructions linked above), `apt` (using the PPA instructions linked above), or `pip install ansible`.
Design Principles
=================
* Have a dead simple setup process and a minimal learning curve
* Manage machines very quickly and in parallel
* Avoid custom-agents and additional open ports, be agentless by leveraging the existing SSH daemon
* Describe infrastructure in a language that is both machine and human friendly
* Focus on security and easy auditability/review/rewriting of content
* Manage new remote machines instantly, without bootstrapping any software
* Allow module development in any dynamic language, not just Python
* Be usable as non-root
* Be the easiest IT automation system to use, ever.
Get Involved
============
* Read [Community Information](http://docs.ansible.com/community.html) for all kinds of ways to contribute to and interact with the project, including mailing list information and how to submit bug reports and code to Ansible.
* All code submissions are done through pull requests. Take care to make sure no merge commits are in the submission, and use `git rebase` vs `git merge` for this reason. If submitting a large code change (other than modules), it's probably a good idea to join ansible-devel and talk about what you would like to do or add first and to avoid duplicate efforts. This not only helps everyone know what's going on, it also helps save time and effort if we decide some changes are needed.
* Development list: [ansible-devel](http://groups.google.com/group/ansible-devel)
* Announcement list: [ansible-announce](http://groups.google.com/group/ansible-announce) - read only
* irc.freenode.net: #ansible
Branch Info
===========
* Releases are named after Led Zeppelin songs. (Releases prior to 2.0 were named after Van Halen songs.)
* The devel branch corresponds to the release actively under development.
* As of 1.8, modules are kept in different repos, you'll want to follow [core](https://github.com/ansible/ansible-modules-core) and [extras](https://github.com/ansible/ansible-modules-extras)
* Various release-X.Y branches exist for previous releases.
* We'd love to have your contributions, read [Community Information](http://docs.ansible.com/community.html) for notes on how to get started.
Authors
=======
Ansible was created by [Michael DeHaan](https://github.com/mpdehaan) (michael.dehaan/gmail/com) and has contributions from over 1000 users (and growing). Thanks everyone!
Ansible is sponsored by [Ansible, Inc](http://ansible.com)
We document our Coding Guidelines in the `Developer Guide <https://docs.ansible.com/ansible/devel/dev_guide/>`_. We particularly suggest you review:
*`Contributing your module to Ansible <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_checklist.html>`_
*`Conventions, tips and pitfalls <https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_best_practices.html>`_
Branch Info
===========
* The ``devel`` branch corresponds to the release actively under development.
* The ``stable-2.X`` branches correspond to stable releases.
* Create a branch based on ``devel`` and set up a `dev environment <https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html#common-environment-setup>`_ if you want to open a PR.
* See the `Ansible release and maintenance <https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html>`_ page for information about active branches.
Roadmap
=======
Based on team and community feedback, an initial roadmap will be published for a major or minor version (ex: 2.7, 2.8).
The `Ansible Roadmap page <https://docs.ansible.com/ansible/devel/roadmap/>`_ details what is planned and how to influence the roadmap.
Authors
=======
Ansible was created by `Michael DeHaan <https://github.com/mpdehaan>`_
and has contributions from over 4700 users (and growing). Thanks everyone!
`Ansible <https://www.ansible.com>`_ is sponsored by `Red Hat, Inc.
- ssh - connection plugin now supports a new variable ``sshpass_prompt`` which gets passed to ``sshpass`` allowing the user to set a custom substring to search for a password prompt (requires sshpass 1.06+)
- apt_repository - Implemented an exponential backoff behaviour when retrying to update the apt cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
- combine filter - ``[dict1, [dict2]] | combine`` now raise an error; previously ``combine`` had an undocumented behaviour where it was flattening the list before combining it (https://github.com/ansible/ansible/pull/57894#discussion_r339517518).
minor_changes:
- combine filter - now accept a ``list_merge`` argument which modifies its behaviour when the hashes to merge contain arrays/lists.
- CLI - the `ANSIBLE_PLAYBOOK_DIR` envvar or `playbook_dir` config can now substitute for the --playbook-dir arg on CLIs that support it (https://github.com/ansible/ansible/issues/59464)
- cron and cronvar - use get_bin_path utility to locate the default crontab executable instead of the hardcoded /usr/bin/crontab. (https://github.com/ansible/ansible/pull/59765)
- apt - Implemented an exponential backoff behaviour when retrying to update the cache with new params ``update_cache_retry_max_delay`` and ``update_cache_retries`` to control the behavior.
- Handle exception encountered while parsing the argument description in module when invoked via ansible-doc command (https://github.com/ansible/ansible/issues/60587).
- vars plugins - Support vars plugins in collections by adding the ability to whitelist plugins.
- host_group_vars plugin - Require whitelisting and whitelist by default.
- Add a global toggle to control when vars plugins are executed (per task by default for backward compatibility or after importing inventory).
- Add a per-plugin stage option to override the global toggle to control the execution of individual vars plugins (per task, after inventory, or both).
- Add --ask-vault-password and --vault-pass-file options to ansible cli commands
- Change order of arguments in ansible cli to use --ask-vault-password and --vault-password-file by default
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
