Update changelog for release.

Allow "=" in k-v values.

.gitignore

@@ -3,3 +3,10 @@ build
 # Emacs backup files...
 *~
 .\#*
+# (s)rpm building stuff
+MANIFEST
+dist
+rpm-build
+# Eclipse/PyDev stuff...
+.project
+.pydevproject

AUTHORS.md

@@ -1,13 +0,0 @@
-Patches and Contributions 
-=========================
-
-   * Michael DeHaan - michael.dehaan AT gmail DOT com
-   * Jeremy Katz - katzj AT fedoraproject DOT org
-   * Seth Vidal - skvidal AT fedoraproject DOT org
-   * Tim Bielawa - tbielawa AT redhat DOT com
-   * Christopher Johnston - chjohnst AT gmail DOT com 
-
-Send in a github pull request to get your name here.
-
-Upstream: github.com/mpdehaan/ansible
-

CHANGELOG.md

@@ -0,0 +1,93 @@
+Ansible Changes By Release
+==========================
+
+0.4 "Unchained" ------- May 23, 2012
+
+Internals/Core
+* internal inventory API now more object oriented, parsers decoupled
+* async handling improvements
+* misc fixes for running ansible on OS X (overlord only)
+* sudo improvements, now works much more smoothly
+* sudo to a particular user with -U/--sudo-user, or using 'sudo_user: foo' in a playbook
+* --private-key CLI option to work with pem files
+
+Inventory
+* can use -i host1,host2,host3:port to specify hosts not in inventory (replaces --override-hosts)
+* ansible INI style format can do groups of groups [groupname:children] and group vars [groupname:vars]
+* groups and users module takes an optional system=yes|no on creation (default no)
+* list of hosts in playbooks can be expressed as a YAML list in addition to ; delimited
+
+Playbooks
+* variables can be replaced like ${foo.nested_hash_key.nested_subkey[array_index]}
+* unicode now ok in templates (assumes utf8)
+* able to pass host specifier or group name in to "hosts:" with --extra-vars
+* ansible-pull script and example playbook (extreme scaling, remediation)
+* inventory_hostname variable available that contains the value of the host as ansible knows it
+* variables in the 'all' section can be used to define other variables based on those values
+* 'group_names' is now a variable made available to templates
+* first_available_file feature, see selective_file_sources.yml in examples/playbooks for info
+* --extra-vars="a=2 b=3" etc, now available to inject parameters into playbooks from CLI
+
+Incompatible Changes
+* jinja2 is only usable in templates, not playbooks, use $foo instead
+* --override-hosts removed, can use -i with comma notation (-i "ahost,bhost")
+* modules can no longer include stderr output (paramiko limitation from sudo)
+
+Module Changes
+* tweaks to SELinux implementation for file module
+* fixes for yum module corner cases on EL5
+* file module now correctly returns the mode in octal
+* fix for symlink handling in the file module
+* service takes an enable=yes|no which works with chkconfig or updates-rc.d as appropriate
+* service module works better on Ubuntu
+* git module now does resets and such to work more smoothly on updates
+* modules all now log to syslog
+* enabled=yes|no on a service can be used to toggle chkconfig & updates-rc.d states
+* git module supports branch=
+* service fixes to better detect status using return codes of the service script
+* custom facts provided by the setup module mean no dependency on Ruby, facter, or ohai
+* service now has a state=reloaded
+* raw module for bootstrapping and talking to routers w/o Python, etc
+
+Misc Bugfixes
+* fixes for variable parsing in only_if lines
+* misc fixes to key=value parsing
+* variables with mixed case now legal
+* fix to internals of hacking/test-module development script
+
+
+0.3 "Baluchitherium" -- April 23, 2012
+
+* Packaging for Debian, Gentoo, and Arch
+* Improvements to the apt and yum modules
+* A virt module
+* SELinux support for the file module
+* Ability to use facts from other systems in templates (aka exported
+resources like support)
+* Built in Ansible facts so you don't need ohai, facter, or Ruby
+* tempdir selections that work with noexec mounted /tmp
+* templates happen locally, not remotely, so no dependency on
+python-jinja2 for remote computers
+* advanced inventory format in YAML allows more control over variables
+per host and per group
+* variables in playbooks can be structured/nested versus just a flat namespace
+* manpage upgrades (docs)
+* various bugfixes
+* can specify a default --user for playbooks rather than specifying it
+in the playbook file
+* able to specify ansible port in ansible host file (see docs)
+* refactored Inventory API to make it easier to write scripts using Ansible
+* looping capability for playbooks (with_items)
+* support for using sudo with a password
+* module arguments can be unicode
+* A local connection type, --connection=local,  for use with cron or
+in kickstarts
+* better module debugging with -D
+* fetch module for pulling in files from remote hosts
+* command task supports creates=foo for idempotent semantics, won't
+run if file foo already exists 
+
+0.0.2 and 0.0.1
+
+* Initial stages of project
+

COPYING

@@ -0,0 +1,675 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+the GNU General Public License is intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.  We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors.  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+  To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights.  Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received.  You must make sure that they, too, receive
+or can get the source code.  And you must show them these terms so they
+know their rights.
+
+  Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+
+  For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software.  For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+  Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so.  This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software.  The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable.  Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products.  If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+  Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary.  To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                       TERMS AND CONDITIONS
+
+  0. Definitions.
+
+  "This License" refers to version 3 of the GNU General Public License.
+
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.
+
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+  A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+  1. Source Code.
+
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.
+
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+  The Corresponding Source for a work in source code form is that
+same work.
+
+  2. Basic Permissions.
+
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Use with the GNU Affero General Public License.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+    <program>  Copyright (C) <year>  <name of author>
+    This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU GPL, see
+<http://www.gnu.org/licenses/>.
+
+  The GNU General Public License does not permit incorporating your program
+into proprietary programs.  If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.  But first, please read
+<http://www.gnu.org/philosophy/why-not-lgpl.html>.
+

MANIFEST.in

@@ -1,3 +1,6 @@
-include README.md AUTHORS.md ansible.spec
+include README.md packaging/rpm/ansible.spec COPYING
+include examples/hosts
+include packaging/distutils/setup.py
 recursive-include docs *
+recursive-include library *
 include Makefile

Makefile

@@ -1,23 +1,59 @@
 #!/usr/bin/make
 
+########################################################
+# Makefile for Ansible
+#
+# useful targets:
+#   make sdist ---------------- produce a tarball
+#   make rpm  ----------------- produce RPMs
+#   make debian --------------- produce a dpkg (FIXME?)
+#   make docs ----------------- rebuild the manpages (results are checked in)
+#   make tests ---------------- run the tests
+#   make pyflakes, make pep8 -- source code checks  
+
+########################################################
+# variable section
+
+NAME = "ansible"
+
+# Manpages are currently built with asciidoc -- would like to move to markdown
+# This doesn't evaluate until it's called. The -D argument is the
+# directory of the target file ($@), kinda like `dirname`.
 ASCII2MAN = a2x -D $(dir $@) -d manpage -f manpage $<
 ASCII2HTMLMAN = a2x -D docs/html/man/ -d manpage -f xhtml
-MANPAGES := docs/man/man1/ansible.1
+MANPAGES := docs/man/man1/ansible.1 docs/man/man1/ansible-playbook.1
+
 SITELIB = $(shell python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")
 
+# VERSION file provides one place to update the software version
+VERSION := $(shell cat VERSION)
+
+# RPM build parameters
+RPMSPECDIR= packaging/rpm
+RPMSPEC = $(RPMSPECDIR)/ansible.spec
+RPMVERSION := $(shell awk '/Version/{print $$2; exit}' < $(RPMSPEC) | cut -d "%" -f1)
+RPMRELEASE := $(shell awk '/Release/{print $$2; exit}' < $(RPMSPEC) | cut -d "%" -f1)
+RPMDIST = $(shell rpm --eval '%dist')
+RPMNVR = "$(NAME)-$(RPMVERSION)-$(RPMRELEASE)$(RPMDIST)"
+
+########################################################
+
 all: clean python
 
 tests: 
-	PYTHONPATH=./lib nosetests
+	PYTHONPATH=./lib nosetests -v
 
-docs: manuals
+# To force a rebuild of the docs run 'touch VERSION && make docs'
+docs: $(MANPAGES)
 
-manuals: $(MANPAGES)
+# Regenerate %.1.asciidoc if %.1.asciidoc.in has been modified more
+# recently than %.1.asciidoc.
+%.1.asciidoc: %.1.asciidoc.in
+	sed "s/%VERSION%/$(VERSION)/" $< > $@
 
-%.1: %.1.asciidoc
-	$(ASCII2MAN)
-
-%.5: %.5.asciidoc
+# Regenerate %.1 if %.1.asciidoc or VERSION has been modified more
+# recently than %.1. (Implicitly runs the %.1.asciidoc recipe)
+%.1: %.1.asciidoc VERSION
 	$(ASCII2MAN)
 
 loc:
@@ -27,38 +63,79 @@ pep8:
 	@echo "#############################################"
 	@echo "# Running PEP8 Compliance Tests"
 	@echo "#############################################"
-	pep8 -r --ignore=E501,E221,W291,W391,E302,E251,E203,W293,E231,E303,E201,E225 lib/ bin/
+	pep8 -r --ignore=E501,E221,W291,W391,E302,E251,E203,W293,E231,E303,E201,E225,E261 lib/ bin/
 
 pyflakes:
-	pyflakes lib/ansible/*.py
+	pyflakes lib/ansible/*.py bin/*
 
 clean:
 	@echo "Cleaning up distutils stuff"
-	-rm -rf build
-	-rm -rf dist
+	rm -rf build
+	rm -rf dist
 	@echo "Cleaning up byte compiled python stuff"
-	find . -regex ".*\.py[co]$$" -delete
+	find . -type f -regex ".*\.py[co]$$" -delete
 	@echo "Cleaning up editor backup files"
 	find . -type f \( -name "*~" -or -name "#*" \) -delete
 	find . -type f \( -name "*.swp" \) -delete
 	@echo "Cleaning up asciidoc to man transformations and results"
-	find ./docs/man -type f \( -name "*.xml" -or -regex ".*\.[0-9]$$" \) -delete
+	find ./docs/man -type f -name "*.xml" -delete
+	find ./docs/man -type f -name "*.asciidoc" -delete
 	@echo "Cleaning up output from test runs"
-	-rm -rf test/test_data
-	@echo "Cleaning up RPM stuff"
-	-rm MANIFEST
+	rm -rf test/test_data
+	@echo "Cleaning up RPM building stuff"
+	rm -rf MANIFEST rpm-build
+	@echo "Cleaning up Debian building stuff"
+	rm -rf debian
+	rm -rf deb-build
 
-python: docs
+python:
 	python setup.py build
 
-install: docs
+install:
+	mkdir -p /usr/share/ansible
+	cp ./library/* /usr/share/ansible/
 	python setup.py install
 
-rpm: 
-	python setup.py sdist
-	rpmbuild -ta dist/ansible-1.0.tar.gz
+sdist: clean
+	python setup.py sdist -t MANIFEST.in
 
-.PHONEY: docs manual clean pep8
-vpath %.asciidoc docs/man/man1
+rpmcommon: sdist
+	@mkdir -p rpm-build
+	@cp dist/*.gz rpm-build/
 
+srpm: rpmcommon
+	@rpmbuild --define "_topdir %(pwd)/rpm-build" \
+	--define "_builddir %{_topdir}" \
+	--define "_rpmdir %{_topdir}" \
+	--define "_srcrpmdir %{_topdir}" \
+	--define "_specdir $(RPMSPECDIR)" \
+	--define "_sourcedir %{_topdir}" \
+	-bs $(RPMSPEC)
+	@echo "#############################################"
+	@echo "Ansible SRPM is built:"
+	@echo "    rpm-build/$(RPMNVR).src.rpm"
+	@echo "#############################################"
+
+rpm: rpmcommon
+	@rpmbuild --define "_topdir %(pwd)/rpm-build" \
+	--define "_builddir %{_topdir}" \
+	--define "_rpmdir %{_topdir}" \
+	--define "_srcrpmdir %{_topdir}" \
+	--define "_specdir $(RPMSPECDIR)" \
+	--define "_sourcedir %{_topdir}" \
+	-ba $(RPMSPEC)
+	@echo "#############################################"
+	@echo "Ansible RPM is built:"
+	@echo "    rpm-build/noarch/$(RPMNVR).noarch.rpm"
+	@echo "#############################################"
+
+debian: sdist
+deb: debian
+	cp -r packaging/debian ./
+	chmod 755 debian/rules
+	fakeroot debian/rules clean
+	fakeroot dh_install
+	fakeroot debian/rules binary
+
+# for arch or gentoo, read instructions in the appropriate 'packaging' subdirectory directory
 

README.md

@@ -24,6 +24,34 @@ Get Involved
    * [ansible-project mailing list](http://groups.google.com/group/ansible-project)
    * irc.freenode.net: #ansible
 
+Branch Info
+===========
+
+   * The master branch corresponds to release 0.3 "Baluchitherium".
+   * Bug fixes WILL be periodically made to the master branch.
+   * The devel branch corresponds to release 0.4 "Unchained".
+   * Note: sudo support works much better in 0.4/devel, consider trying it.
+   * See CHANGELOG.md for release notes to track each release.
+
+Patch Instructions
+==================
+
+Contributions to the core and modules are greatly welcome.
+
+   * Required Process:
+       * Submit github pull requests to the "ansible/devel" branch for features
+       * Fixes for bugs may be submitted to "ansible/master"
+       * Make sure "make tests" passes before submitting any requests.
+   * Bonus points:
+       * Joining the mailing list
+       * Using squash merges
+       * Updating the "rst/*" files in the docs project and "docs/" manpage content
+       * Adding more unit tests 
+   * Avoid:
+       * Sending patches to the mailing list directly.
+       * Sending feature pull requests to the 'master' branch instead of the devel branch
+       * Sending pull requests to mpdehaan's personal ansible fork.
+
 Author
 ======
 

VERSION

@@ -0,0 +1 @@
+0.4

ansible.spec

@@ -1,50 +0,0 @@
-%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
-
-Summary: Minimal SSH command and control
-Name: ansible
-Version: 1.0
-Release: 1
-Source0: ansible-%{version}.tar.gz
-License: GPLv3
-Group: Development/Libraries
-BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot
-Prefix: %{_prefix}
-BuildArch: noarch
-Url: http://github.com/mpdehaan/ansible/
-BuildRequires: asciidoc
-Requires: python-paramiko
-Requires: python-jinja2
-
-%description
-Ansible is a extra-simple tool/API for doing 'parallel remote things' over SSH
-executing commands, running "modules", or executing larger 'playbooks' that
-can serve as a configuration management or deployment system.
-
-%prep
-%setup -n %{name}-%{version}
-
-%build
-python setup.py build
-make docs
-
-%install
-python setup.py install -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
-mkdir -p $RPM_BUILD_ROOT/etc/ansible/
-
-%clean
-rm -rf $RPM_BUILD_ROOT
-
-%files
-%doc README.md AUTHORS.md PKG-INFO
-%defattr(-,root,root)
-%{_mandir}/man1/*.gz
-%{_mandir}/man5/*.gz
-%{python_sitelib}/*
-%{_bindir}/ansible*
-%{_datadir}/ansible/*
-%{_sysconfdir}/ansible/
-
-%changelog
-* Mon Mar  5 2012 Seth Vidal <skvidal at fedoraproject.org>
-- spec file
-

bin/ansible

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 
 # (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
 #
@@ -19,20 +19,15 @@
 
 ########################################################
 
-try:
-    import json
-except ImportError:
-    import simplejson as json
-from optparse import OptionParser
 import sys
-import os
 import getpass
-import shlex
-import ansible.runner
-import ansible.playbook
+
+from ansible.runner import Runner
 import ansible.constants as C
-from optparse import OptionParser
-from ansible.utils import *
+from ansible import utils
+from ansible import errors
+from ansible import callbacks
+from ansible import inventory
 
 ########################################################
 
@@ -42,38 +37,24 @@ class Cli(object):
     # ----------------------------------------------     
 
     def __init__(self):
-        pass
+        self.stats = callbacks.AggregateStats()
+        self.callbacks = callbacks.CliRunnerCallbacks()
 
     # ----------------------------------------------     
 
     def parse(self):
         ''' create an options parser for bin/ansible '''
-
-        parser = OptionParser(usage = 'ansible <host-pattern> [options]')
-        parser.add_option("-a", "--args", dest="module_args",
-            help="module arguments", default=C.DEFAULT_MODULE_ARGS)
-        parser.add_option("-B", "--background", dest="seconds", default=0,
-            help="run asynchronously, failing after X seconds")
-        parser.add_option('-f','--forks', dest='forks', default=C.DEFAULT_FORKS, type='int',
-            help='number of parallel processes to use')
-        parser.add_option("-i", "--inventory-file", dest="inventory",
-           help="inventory host file", default=C.DEFAULT_HOST_LIST)
-        parser.add_option("-k", "--ask-pass", default=False, action="store_true",
-            help="ask for SSH password")
-        parser.add_option("-M", "--module-path", dest="module_path",
-           help="path to module library", default=C.DEFAULT_MODULE_PATH)
-        parser.add_option("-m", "--module-name", dest="module_name",
-            help="module name to execute", default=C.DEFAULT_MODULE_NAME)
-        parser.add_option('-o', '--one-line', dest='one_line', action='store_true',
-            help="condense output")
-        parser.add_option('-t', '--tree', dest='tree', default=None,
-            help="log output to this directory")
-        parser.add_option('-T', '--timeout', default=C.DEFAULT_TIMEOUT, type='int',
-            dest='timeout', help="set the SSH timeout in seconds")
-        parser.add_option('-u', '--user', default=C.DEFAULT_REMOTE_USER,
-            dest='remote_user', help='connect as this user')
-        options, args = parser.parse_args()
         
+        parser = utils.base_parser(constants=C, runas_opts=True, async_opts=True,
+            output_opts=True, connect_opts=True, usage='%prog <host-pattern> [options]')
+        parser.add_option('-a', '--args', dest='module_args',
+            help="module arguments", default=C.DEFAULT_MODULE_ARGS)
+        parser.add_option('-m', '--module-name', dest='module_name',
+            help="module name to execute (default=%s)" % C.DEFAULT_MODULE_NAME, 
+            default=C.DEFAULT_MODULE_NAME)
+        options, args = parser.parse_args()
+        self.callbacks.options = options        
+
         if len(args) == 0 or len(args) > 1:
             parser.print_help()
             sys.exit(1)
@@ -85,57 +66,69 @@ class Cli(object):
         ''' use Runner lib to do SSH things '''
 
         pattern = args[0]
+
+        inventory_manager = inventory.Inventory(options.inventory)
+        hosts = inventory_manager.list_hosts(pattern)
+        if len(hosts) == 0:
+            print >>sys.stderr, "No hosts matched"
+            sys.exit(1)
+        
         sshpass = None
+        sudopass = None
         if options.ask_pass:
             sshpass = getpass.getpass(prompt="SSH password: ")
+        if options.ask_sudo_pass:
+            sudopass = getpass.getpass(prompt="sudo password: ")
+            options.sudo = True
+        if options.sudo_user:
+            options.sudo = True
+        options.sudo_user = options.sudo_user or C.DEFAULT_SUDO_USER
+        if options.tree:
+            utils.prepare_writeable_dir(options.tree)
 
-        return ansible.runner.Runner(
-            module_name=options.module_name,
-            module_path=options.module_path,
-            module_args=shlex.split(options.module_args),
-            remote_user=options.remote_user,
-            remote_pass=sshpass,
-            host_list=options.inventory,
-            timeout=options.timeout,
-            forks=options.forks,
-            background=options.seconds,
-            pattern=pattern,
-            verbose=True,
-        ).run()
+        runner = Runner(
+            module_name=options.module_name, module_path=options.module_path,
+            module_args=options.module_args,
+            remote_user=options.remote_user, remote_pass=sshpass,
+            inventory=inventory_manager, timeout=options.timeout, 
+            private_key_file=options.private_key_file,
+            forks=options.forks, 
+            pattern=pattern, 
+            callbacks=self.callbacks, sudo=options.sudo, 
+            sudo_pass=sudopass,sudo_user=options.sudo_user,
+            transport=options.connection, debug=options.debug
+        )
+
+        if options.seconds:
+            print "background launch...\n\n"
+            results, poller = runner.runAsync(options.seconds)
+            results = self.poll_while_needed(poller, options)
+        else:
+            results = runner.run()
+
+        return (runner, results)
 
     # ----------------------------------------------     
 
-    def output(self, results, options, args):
+    def poll_while_needed(self, poller, options):
         ''' summarize results from Runner '''
 
-        if results is None:
-           exit("No hosts matched")
-        if options.tree:
-            prepare_writeable_dir(options.tree)
+        # BACKGROUND POLL LOGIC when -B and -P are specified
+        if options.seconds and options.poll_interval > 0:
+            poller.wait(options.seconds, options.poll_interval)
 
-        buf = '' 
-        for hostname in contacted_hosts(results):
-            msg = host_report_msg(
-                hostname, 
-                options.module_name, 
-                contacted_host_result(results, hostname),
-                options.one_line
-            )
-            if options.tree:
-                write_tree_file(options.tree, hostname, bigjson(results))
-            buf += msg
-            
-        if has_dark_hosts(results):
-            buf += dark_hosts_msg(results)
+        return poller.results
 
-        print buf
 
 ########################################################
 
 if __name__ == '__main__':
     cli = Cli()
     (options, args) = cli.parse()
-    results = cli.run(options, args)
-    cli.output(results, options, args)
-    
+    try:
+        (runner, results) = cli.run(options, args)
+    except errors.AnsibleError, e:
+        # Generic handler for ansible specific errors
+        print "ERROR: %s" % str(e)
+        sys.exit(1)
 

bin/ansible-playbook

@@ -1,4 +1,4 @@
-#!/usr/bin/python -tt
+#!/usr/bin/env python
 # (C) 2012, Michael DeHaan, <michael.dehaan@gmail.com>
 
 # This file is part of Ansible
@@ -19,86 +19,90 @@
 #######################################################
 
 import sys
+import getpass
+
 import ansible.playbook
 import ansible.constants as C
-from ansible.utils import *
-import getpass
-from optparse import OptionParser
-
-#######################################################
-
-class PlaybookCallbacks(object):
-  
-    def __init__(self):
-        pass
-
-    def set_playbook(self, playbook):
-        self.playbook = playbook
-
-    def on_start(self):
-        print "\n"
-
-    def on_task_start(self, name, is_conditional):
-        print task_start_msg(name, is_conditional)
-
-    def on_unreachable(self, host, msg):
-        print "unreachable: [%s] => %s" % (host, msg)
-
-    def on_failed(self, host, results):
-        print "failed: [%s] => %s\n" % (host, smjson(results))
- 
-    def on_ok(self, host):
-        print "ok: [%s]\n" % (host)
-
-    def on_play_start(self, pattern):
-        print "PLAY [%s] ****************************\n" % pattern
-
+from ansible import errors
+from ansible import callbacks
+from ansible import utils
 
 def main(args):
     ''' run ansible-playbook operations '''
 
     # create parser for CLI options
-    parser = OptionParser()
-    parser.usage = "ans-playbook playbook.yml ..."
-    parser.add_option('-f','--forks', dest='forks', default=C.DEFAULT_FORKS, type='int',
-        help='set the number of forks to start up')
-    parser.add_option("-i", "--inventory-file", dest="inventory",
-        help="inventory host file", default=C.DEFAULT_HOST_LIST)
-    parser.add_option("-k", "--ask-pass", default=False, action="store_true",
-        help="ask for SSH password")
-    parser.add_option("-M", "--module-path", dest="module_path",
-        help="path to module library", default=C.DEFAULT_MODULE_PATH)
+    usage = "%prog playbook.yml"
+    parser = utils.base_parser(constants=C, usage=usage, connect_opts=True, runas_opts=True)
+    parser.add_option('-e', '--extra-vars', dest="extra_vars", default=None,
+        help="set additional key=value variables from the CLI")
 
     options, args = parser.parse_args(args)
 
     if len(args) == 0:
-        print >> sys.stderr, "playbook path is a required argument"
+        parser.print_help(file=sys.stderr)
         return 1
 
     sshpass = None
+    sudopass = None
     if options.ask_pass:
         sshpass = getpass.getpass(prompt="SSH password: ")
+    if options.ask_sudo_pass:
+        sudopass = getpass.getpass(prompt="sudo password: ")
+        options.sudo = True
+    if options.sudo_user:
+        options.sudo = True
+    options.sudo_user = options.sudo_user or C.DEFAULT_SUDO_USER
+    extra_vars = utils.parse_kv(options.extra_vars)
 
     # run all playbooks specified on the command line
     for playbook in args:
+
+        stats = callbacks.AggregateStats()
+        playbook_cb = callbacks.PlaybookCallbacks()
+        runner_cb = callbacks.PlaybookRunnerCallbacks(stats)
+
         pb = ansible.playbook.PlayBook(
             playbook=playbook,
-            host_list=options.inventory,
             module_path=options.module_path,
-            forks=options.forks,
-            verbose=True,
-            remote_pass=sshpass,
-            callbacks=PlaybookCallbacks()
+            host_list=options.inventory, 
+            forks=options.forks, 
+            debug=options.debug, 
+            remote_user=options.remote_user,
+            remote_pass=sshpass, 
+            callbacks=playbook_cb, 
+            runner_callbacks=runner_cb, 
+            stats=stats,
+            timeout=options.timeout, 
+            transport=options.connection,
+            sudo=options.sudo,
+            sudo_user=options.sudo_user,
+            sudo_pass=sudopass,
+            extra_vars=extra_vars,
+            private_key_file=options.private_key_file            
         )
-        pb.run()
+        try:
+
+            pb.run()
+            hosts = sorted(pb.stats.processed.keys())
+            print "\n\nPLAY RECAP **********************\n\n"
+            for h in hosts:
+                t = pb.stats.summarize(h)
+                print "%-30s : ok=%4s changed=%4s unreachable=%4s failed=%4s " % (h, 
+                   t['ok'], t['changed'], t['unreachable'], t['failures']
+                )
+            print "\n"
+
+        except errors.AnsibleError, e:
+            print >>sys.stderr, "ERROR: %s" % e
+            return 1
 
     return 0
 
 
 if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))
-
-
-
-
+    try:
+        sys.exit(main(sys.argv[1:]))
+    except errors.AnsibleError, e:
+        print >>sys.stderr, "ERROR: %s" % e
+        sys.exit(1)
 

bin/ansible-pull

@@ -0,0 +1,75 @@
+#!/usr/bin/env python
+
+# ansible-pull is a script that runs ansible in local mode
+# after checking out a playbooks directory from git.  There is an
+# example playbook to bootstrap this script in the examples/ dir which
+# installs ansible and sets it up to run on cron.
+#
+# usage: 
+#   ansible-pull -d /var/ansible/local -U http://wherever/content.git -C production
+#
+# the git repo must contain a playbook named 'local.yml'
+
+# (c) 2012, Stephen Fromm <sfromm@gmail.com>
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import subprocess
+import sys
+from optparse import OptionParser
+
+DEFAULT_PLAYBOOK = 'local.yml'
+
+def _run(cmd):
+    cmd = subprocess.Popen(cmd, shell=True,
+                           stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    (out, err) = cmd.communicate()
+    print out
+    if cmd.returncode != 0:
+        print err
+    return cmd.returncode
+
+def main(args):
+    """ Set up and run a local playbook """
+    usage = "%prog [options]"
+    parser = OptionParser()
+    parser.add_option('-d', '--directory', dest='dest', default=None,
+                      help='Directory to checkout git repository')
+    parser.add_option('-U', '--url', dest='url',
+                      default=None,
+                      help='URL of git repository')
+    parser.add_option('-C', '--checkout', dest='checkout',
+                      default="HEAD",
+                      help='Branch/Tag/Commit to checkout.  Defaults to HEAD.')
+    options, args = parser.parse_args(args)
+
+    git_opts = "repo=%s dest=%s version=%s" % (options.url, options.dest, options.checkout)
+    cmd = 'ansible all -c local -m git -a "%s"' % git_opts
+    print "cmd=%s" % cmd
+    rc = _run(cmd)
+    if rc != 0:
+        return rc
+
+    os.chdir(options.dest)
+    cmd = 'ansible-playbook -c local %s' % DEFAULT_PLAYBOOK
+    rc = _run(cmd)
+    return rc
+
+if __name__ == '__main__':
+    try:
+        sys.exit(main(sys.argv[1:]))
+    except KeyboardInterrupt, e:
+        print >>sys.stderr, "Exit on user request.\n"
+        sys.exit(1)

docs/comparisons.md

@@ -1,203 +0,0 @@
-Comparisons
-===========
-
-Confused about how Ansible fits in?  Here's a comparison with some common tools.
-
-Accuracy is important, so corrections are VERY welcome if we've got something wrong.
-For space reasons, we can't list everybody's favorite management tool.
-
-Ansible mainly tries to be different in it's goals of minimalism, while also cutting across
-the categories of configuration management, deployment, and ad-hoc task firing.
-
-It strives to be a super-easy to use app, requiring no daemons or databases, and having
-a super-simple configuration management language that takes minutes to learn.
-
-Generally speaking, Ansible is answer to the question "why is the world's systems
-management software so complicated?"  The code does not try to show off. It is an 
-experiment in radical simplicity and encourages anyone to be able to easily contribute to it.
-
-<table>
-   <tr>
-      <td></td>
-      <td><B><U>Ansible</U></B></font></td>
-      <td>Puppet</td>
-      <td>Chef</td>
-      <td>Func</td>
-      <td>Capistrano</td>
-      <td>mCollective</td>
-   </tr>
-   <tr>
-      <td>Purpose</td>
-      <td><B>Config, Deployment, Ad-Hoc</B></td>
-      <td>Config, Deployment</td>
-      <td>Config, Deployment</td>
-      <td>Ad-Hoc</td>
-      <td>Deployment</td> 
-      <td>Ad-Hoc</td>
-   </tr>
-   <tr>
-      <td>Inspirations</td>
-      <td>Cobbler, Puppet, Func, Taboot</B></td>
-      <td>cfengine</td>
-      <td>puppet</td>
-      <td>sysadmin SSH scripts</td>
-      <td>sysadmin SSH scripts</td> 
-      <td>Func, puppet</td>
-   </tr>
-
-   <tr>
-      <td>Config Language</td>
-      <td><B>Simple YAML format</B></td>
-      <td>Custom DSL</td>
-      <td>Ruby code</td>
-      <td>None</td>
-      <td>None</td>
-      <td>None</td>
-   </tr>
-   <tr>
-      <td>Config Language Style</td>
-      <td><B>Very simple</B></td>
-      <td>Very rich, complex</td>
-      <td>Very rich, complex</td>
-      <td>None</td>
-      <td>None</td>
-      <td>None</td>
-   </tr>
-   <tr>
-      <td>Config Ordering</td>
-      <td><B>Simply ordered, with notifiers</B></td>
-      <td>Strict dependency DAG</td>
-      <td>Simply ordered</td>
-      <td>None</td>
-      <td>Simply ordered</td>
-      <td>None</td>
-   </tr>
-   <tr>
-      <td>Communication</td>
-      <td>SSH push</td>
-      <td>SSL pull or push trigger</td>
-      <td>SSL pull or push trigger(?)</td>
-      <td>SSL push</td>
-      <td>SSH</td>
-      <td>AMQP bus</td>
-   </tr>
-   <tr>
-      <td>Daemons Required?</td>
-      <td><B>no</B></td>
-      <td>yes</td>
-      <td>yes</td>
-      <td>yes</td>
-      <td><B>no</B></td>
-      <td>yes</td>
-   </tr>
-   <tr>
-      <td>Database Needed?</td>
-      <td><B>no</B></td>
-      <td>some features</td>
-      <td>yes</td>
-      <td><B>no</B></td>
-      <td><B>no</B></td>
-      <td>some features</td>
-   </tr>
-   <tr>
-      <td>Message Bus Required?</td>
-      <td><B>no</B></td>
-      <td><B>no</B></td>
-      <td>yes</td>
-      <td><B>no</B></td>
-      <td><B>no</B></td>
-      <td>yes</td>
-   </tr>
-   <tr>
-      <td>Inventory Features</td>
-      <td>planned</td>
-      <td><B>yes</B></td>
-      <td><B>yes?</B></td>
-      <td>some</td>
-      <td>no</td>
-      <td>no</td>
-   </tr>
-   <tr>
-      <td>Implemented In</td>
-      <td>Python</td>
-      <td>Ruby</td>
-      <td>Ruby, Erlang</td>
-      <td>Python</td>
-      <td>Ruby</td>
-      <td>Ruby</td>
-   </tr>
-   <tr>
-      <td>Extensible In</td>
-      <td><B>Anything</B></td>
-      <td>Ruby</td>
-      <td>Ruby</td>
-      <td>Python</td>
-      <td>Ruby</td>
-      <td>Ruby</td>
-   </tr>
-   <tr>
-      <td>Codebase Size</td>
-      <td><B>Small</B></td>
-      <td>Large</td>
-      <td>Large</td>
-      <td>Medium</td>
-      <td>Medium</td>
-      <td>Medium</td>
-   </tr>
-   <tr>
-      <td>Module Support</td>
-      <td>Emerging</td>
-      <td><B>Wide/Established</B></td>
-      <td><B>Wide/Established</B></td>
-      <td>Medium/Established</td>
-      <td>Poor</td>
-      <td>Medium/Established</td>
-   </tr>
-   <tr>
-      <td>Users Targeted</td>
-      <td><B>Sysadmins, developers, QA, web admins</B></td>
-      <td>Sysadmins</td>
-      <td>Sysadmins, web admins</td>
-      <td>Sysadmins, developers</td>
-      <td>Web admins</td>
-      <td>Sysadmins</td>
-   </tr>
-   <tr>
-      <td>Learning & Setup Curve</td>
-      <td><B>Minimal<B></td>
-      <td>High</td>
-      <td>High</td> 
-      <td>Medium</td>
-      <td>Medium</td>
-      <td>Medium</td>
-   </tr>
-   <tr>
-      <td>Can Easily Build Applications On It</td>
-      <td><B>Yes</B></td>
-      <td>No</td>
-      <td>No</td>
-      <td><B>Yes</B></td>
-      <td>No</td>
-      <td>Yes?</td>
-   </tr>
-   <tr>
-      <td>Parallelism & Scaling Tech</td>
-      <td>Fork/Merge</td>
-      <td>Add Servers / Staged Commands / No Server</td>
-      <td>Adding More Erlang</td>
-      <td>Fork/Merge</td>
-      <td>No</td>
-      <td>Message Bus</td>
-   </tr>
-   <tr>
-      <td>Delegated Hierachies</td>
-      <td>No</td>
-      <td>No</td>
-      <td>No</td>
-      <td><B>Yes</B></td>
-      <td>No</td>
-      <td><B>Yes</B></td>
-   </tr>
-</td>
-
-

docs/man/.gitignore

@@ -1,2 +1,2 @@
 *.xml
-*.[0-9]
+*.asciidoc

docs/man/man1/ansible-playbook.1

@@ -0,0 +1,120 @@
+'\" t
+.\"     Title: ansible-playbook
+.\"    Author: [see the "AUTHOR" section]
+.\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
+.\"      Date: 04/26/2012
+.\"    Manual: System administration commands
+.\"    Source: Ansible 0.4
+.\"  Language: English
+.\"
+.TH "ANSIBLE\-PLAYBOOK" "1" "04/26/2012" "Ansible 0\&.4" "System administration commands"
+.\" -----------------------------------------------------------------
+.\" * set default formatting
+.\" -----------------------------------------------------------------
+.\" disable hyphenation
+.nh
+.\" disable justification (adjust text to left margin only)
+.ad l
+.\" -----------------------------------------------------------------
+.\" * MAIN CONTENT STARTS HERE *
+.\" -----------------------------------------------------------------
+.SH "NAME"
+ansible-playbook \- run an ansible playbook
+.SH "SYNOPSIS"
+.sp
+ansible\-playbook <filename\&.yml> \&... [options]
+.SH "DESCRIPTION"
+.sp
+\fBAnsible playbooks\fR are a configuration and multinode deployment system\&. Ansible\-playbook is the tool used to run them\&. See the project home page (link below) for more information\&.
+.SH "ARGUMENTS"
+.PP
+\fBfilename\&.yml\fR
+.RS 4
+The names of one or more YAML format files to run as ansible playbooks\&.
+.RE
+.SH "OPTIONS"
+.sp
+\fB\-D\fR, \fB\-\-debug\fR
+.sp
+Print any messages the remote module sends to standard error to the console
+.PP
+\fB\-i\fR \fIPATH\fR, \fB\-\-inventory=\fR\fIPATH\fR
+.RS 4
+The
+\fIPATH\fR
+to the inventory hosts file, which defaults to
+\fI/etc/ansible/hosts\fR\&.
+.RE
+.PP
+\fB\-M\fR \fIDIRECTORY\fR, \fB\-\-module\-path=\fR\fIDIRECTORY\fR
+.RS 4
+The
+\fIDIRECTORY\fR
+to load modules from\&. The default is
+\fI/usr/share/ansible\fR\&.
+.RE
+.PP
+\fB\-e\fR \fIVARS\fR, \fB\-\-extra\-vars=\fR\fIVARS\fR
+.RS 4
+Extra variables to inject into a playbook, in key=value key=value format\&.
+.RE
+.PP
+\fB\-f\fR \fINUM\fR, \fB\-\-forks=\fR\fINUM\fR
+.RS 4
+Level of parallelism\&.
+\fINUM\fR
+is specified as an integer, the default is 5\&.
+.RE
+.PP
+\fB\-k\fR, \fB\-\-ask\-pass\fR
+.RS 4
+Prompt for the SSH password instead of assuming key\-based authentication with ssh\-agent\&.
+.RE
+.PP
+\fB\-K\fR, \fB\-\-ask\-sudo\-pass\fR
+.RS 4
+Prompt for the password to use for playbook plays that request sudo access, if any
+.RE
+.PP
+\fB\-T\fR \fISECONDS\fR, \fB\-\-timeout=\fR\fISECONDS\fR
+.RS 4
+Connection timeout to use when trying to talk to hosts, in
+\fISECONDS\fR\&.
+.RE
+.PP
+\fB\-O\fR \fIOVERRIDE_HOSTS\fR, \fB\-\-override\-hosts=\fR\fIOVERRIDE_HOSTS\fR
+.RS 4
+Ignore the inventory file and run the playbook against only these hosts\&. "hosts:" line in playbook should be set to
+\fIall\fR
+when using this option\&.
+.RE
+.PP
+\fB\-s\fR, \fB\-\-sudo\fR
+.RS 4
+Force all plays to use sudo, even if not marked as such\&.
+.RE
+.PP
+\fB\-u\fR \fIUSERNAME\fR, \fB\-\-remote\-user=\fR\fIUSERNAME\fR
+.RS 4
+Use this remote user name on playbook steps that do not indicate a user name to run as\&.
+.RE
+.SH "ENVIRONMENT"
+.sp
+The following environment variables may specified\&.
+.sp
+ANSIBLE_HOSTS  \(em Override the default ansible hosts file
+.sp
+ANSIBLE_LIBRARY \(em Override the default ansible module library path
+.SH "AUTHOR"
+.sp
+Ansible was originally written by Michael DeHaan\&. See the AUTHORS file for a complete list of contributors\&.
+.SH "COPYRIGHT"
+.sp
+Copyright \(co 2012, Michael DeHaan
+.sp
+Ansible is released under the terms of the GPLv3 License\&.
+.SH "SEE ALSO"
+.sp
+\fBansible\fR(1)
+.sp
+Extensive documentation as well as IRC and mailing list info is available on the ansible home page: https://ansible\&.github\&.com/

docs/man/man1/ansible-playbook.1.asciidoc.in

@@ -0,0 +1,118 @@
+ansible-playbook(1)
+===================
+:doctype:manpage
+:man source:   Ansible
+:man version:  %VERSION%
+:man manual:   System administration commands
+
+NAME
+----
+ansible-playbook - run an ansible playbook
+
+
+SYNOPSIS
+--------
+ansible-playbook <filename.yml> ... [options]
+
+
+DESCRIPTION
+-----------
+
+*Ansible playbooks* are a configuration and multinode deployment
+system.  Ansible-playbook is the tool used to run them.  See the
+project home page (link below) for more information.
+
+
+ARGUMENTS
+---------
+
+*filename.yml*::
+
+The names of one or more YAML format files to run as ansible playbooks.
+
+
+OPTIONS
+-------
+
+*-D*, *--debug*          
+
+Debug mode
+
+*-i* 'PATH', *--inventory=*'PATH'::
+
+The 'PATH' to the inventory hosts file, which defaults to '/etc/ansible/hosts'.
+
+
+*-M* 'DIRECTORY', *--module-path=*'DIRECTORY'::
+
+The 'DIRECTORY' to load modules from. The default is '/usr/share/ansible'.
+
+*-e* 'VARS', *--extra-vars=*'VARS'::
+
+Extra variables to inject into a playbook, in key=value key=value format.
+
+*-f* 'NUM', *--forks=*'NUM'::
+
+Level of parallelism. 'NUM' is specified as an integer, the default is 5.
+
+
+*-k*, *--ask-pass*::
+
+Prompt for the SSH password instead of assuming key-based authentication with ssh-agent.
+
+
+*-K*, *--ask-sudo-pass*::
+
+Prompt for the password to use for playbook plays that request sudo access, if any
+
+
+*-T* 'SECONDS', *--timeout=*'SECONDS'::
+
+Connection timeout to use when trying to talk to hosts, in 'SECONDS'.
+
+
+*-s*, *--sudo*::
+
+Force all plays to use sudo, even if not marked as such.
+
+
+*-u* 'USERNAME', *--remote-user=*'USERNAME'::
+
+Use this remote user name on playbook steps that do not indicate a user name to run as.
+
+
+ENVIRONMENT
+-----------
+
+The following environment variables may specified.
+
+ANSIBLE_HOSTS  -- Override the default ansible hosts file
+
+ANSIBLE_LIBRARY -- Override the default ansible module library path
+
+
+AUTHOR
+------
+
+Ansible was originally written by Michael DeHaan. See the AUTHORS file
+for a complete list of contributors.
+
+
+COPYRIGHT
+---------
+
+Copyright © 2012, Michael DeHaan
+
+Ansible is released under the terms of the GPLv3 License.
+
+
+SEE ALSO
+--------
+
+*ansible*(1)
+
+Extensive documentation as well as IRC and mailing list info
+is available on the ansible home page: <https://ansible.github.com/>
+
+
+

docs/man/man1/ansible.1

@@ -0,0 +1,173 @@
+'\" t
+.\"     Title: ansible
+.\"    Author: [see the "AUTHOR" section]
+.\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
+.\"      Date: 05/06/2012
+.\"    Manual: System administration commands
+.\"    Source: Ansible 0.4
+.\"  Language: English
+.\"
+.TH "ANSIBLE" "1" "05/06/2012" "Ansible 0\&.4" "System administration commands"
+.\" -----------------------------------------------------------------
+.\" * set default formatting
+.\" -----------------------------------------------------------------
+.\" disable hyphenation
+.nh
+.\" disable justification (adjust text to left margin only)
+.ad l
+.\" -----------------------------------------------------------------
+.\" * MAIN CONTENT STARTS HERE *
+.\" -----------------------------------------------------------------
+.SH "NAME"
+ansible \- run a command somewhere else
+.SH "SYNOPSIS"
+.sp
+ansible <host\-pattern> [\-f forks] [\-m module_name] [\-a args]
+.SH "DESCRIPTION"
+.sp
+\fBAnsible\fR is an extra\-simple tool/framework/API for doing \'remote things\' over SSH\&.
+.SH "ARGUMENTS"
+.PP
+\fBhost\-pattern\fR
+.RS 4
+A name of a group in the inventory file, a shell\-like glob selecting hosts in inventory file, or any combination of the two separated by semicolons\&.
+.RE
+.SH "OPTIONS"
+.PP
+\fB\-i\fR \fIPATH\fR, \fB\-\-inventory=\fR\fIPATH\fR
+.RS 4
+The
+\fIPATH\fR
+to the inventory hosts file, which defaults to
+\fI/etc/ansible/hosts\fR\&.
+.RE
+.PP
+\fB\-f\fR \fINUM\fR, \fB\-\-forks=\fR\fINUM\fR
+.RS 4
+Level of parallelism\&.
+\fINUM\fR
+is specified as an integer, the default is 5\&.
+.RE
+.PP
+\fB\-m\fR \fINAME\fR, \fB\-\-module\-name=\fR\fINAME\fR
+.RS 4
+Execute the module called
+\fINAME\fR\&.
+.RE
+.PP
+\fB\-M\fR \fIDIRECTORY\fR, \fB\-\-module\-path=\fR\fIDIRECTORY\fR
+.RS 4
+The
+\fIDIRECTORY\fR
+to load modules from\&. The default is
+\fI/usr/share/ansible\fR\&.
+.RE
+.PP
+\fB\-a\fR \'\fIARGUMENTS\fR\', \fB\-\-args=\fR\'\fIARGUMENTS\fR\'
+.RS 4
+The
+\fIARGUMENTS\fR
+to pass to the module\&.
+.RE
+.PP
+\fB\-D\fR, \fB\-\-debug\fR
+.RS 4
+Debug mode
+.RE
+.PP
+\fB\-k\fR, \fB\-\-ask\-pass\fR
+.RS 4
+Prompt for the SSH password instead of assuming key\-based authentication with ssh\-agent\&.
+.RE
+.PP
+\fB\-K\fR, \fB\-\-ask\-sudo\-pass\fR
+.RS 4
+Prompt for the password to use with \-\-sudo, if any
+.RE
+.PP
+\fB\-o\fR, \fB\-\-one\-line\fR
+.RS 4
+Try to output everything on one line\&.
+.RE
+.PP
+\fB\-s\fR, \fB\-\-sudo\fR
+.RS 4
+Run the command as the user given by \-u and sudo to root\&.
+.RE
+.PP
+\fB\-t\fR \fIDIRECTORY\fR, \fB\-\-tree=\fR\fIDIRECTORY\fR
+.RS 4
+Save contents in this output
+\fIDIRECTORY\fR, with the results saved in a file named after each host\&.
+.RE
+.PP
+\fB\-T\fR \fISECONDS\fR, \fB\-\-timeout=\fR\fISECONDS\fR
+.RS 4
+Connection timeout to use when trying to talk to hosts, in
+\fISECONDS\fR\&.
+.RE
+.PP
+\fB\-B\fR \fINUM\fR, \fB\-\-background=\fR\fINUM\fR
+.RS 4
+Run commands in the background, killing the task after
+\fINUM\fR
+seconds\&.
+.RE
+.PP
+\fB\-P\fR \fINUM\fR, \fB\-\-poll=\fR\fINUM\fR
+.RS 4
+Poll a background job every
+\fINUM\fR
+seconds\&. Requires
+\fB\-B\fR\&.
+.RE
+.PP
+\fB\-u\fR \fIUSERNAME\fR, \fB\-\-remote\-user=\fR\fIUSERNAME\fR
+.RS 4
+Use this remote
+\fIUSERNAME\fR
+instead of root\&.
+.RE
+.PP
+\fB\-U\fR \fISUDO_USERNAME\fR, \fB\-\-sudo\-user=\fR\fISUDO_USERNAME\fR
+.RS 4
+Sudo to
+\fISUDO_USERNAME\fR
+instead of root\&. Implies \-\-sudo\&.
+.RE
+.PP
+\fB\-c\fR \fICONNECTION\fR, \fB\-\-connection=\fR\fICONNECTION\fR
+.RS 4
+Connection type to use\&. Possible options are
+\fIparamiko\fR
+(SSH) and
+\fIlocal\fR\&. Local is mostly useful for crontab or kickstarts\&.
+.RE
+.SH "INVENTORY"
+.sp
+Ansible stores the hosts it can potentially operate on in an inventory file\&. The syntax is one host per line\&. Groups headers are allowed and are included on their own line, enclosed in square brackets\&.
+.SH "FILES"
+.sp
+/etc/ansible/hosts \(em Default inventory file
+.sp
+/usr/share/ansible/ \(em Default module library
+.SH "ENVIRONMENT"
+.sp
+The following environment variables may specified\&.
+.sp
+ANSIBLE_HOSTS  \(em Override the default ansible hosts file
+.sp
+ANSIBLE_LIBRARY \(em Override the default ansible module library path
+.SH "AUTHOR"
+.sp
+Ansible was originally written by Michael DeHaan\&. See the AUTHORS file for a complete list of contributors\&.
+.SH "COPYRIGHT"
+.sp
+Copyright \(co 2012, Michael DeHaan
+.sp
+Ansible is released under the terms of the GPLv3 License\&.
+.SH "SEE ALSO"
+.sp
+\fBansible\-playbook\fR(1)
+.sp
+Extensive documentation as well as IRC and mailing list info is available on the ansible home page: https://ansible\&.github\&.com/

docs/man/man1/ansible.1.asciidoc

@@ -1,108 +0,0 @@
-ansible(1)
-=========
-:doctype:manpage
-:man source:   Ansible
-:man version:  0.0.1
-:man manual:   System administration commands
-
-NAME
-----
-ansible - run a command somewhere else
-
-
-SYNOPSIS
---------
-ansible <host-pattern> [-f forks] [-m module_name] [-a args]
-
-
-DESCRIPTION
------------
-
-*Ansible* is an extra-simple tool/framework/API for doing \'remote things' over
-SSH.
-
-
-ARGUMENTS
----------
-
-*host-pattern*::
-
-A name of a group in the inventory file, a shell-like glob selecting hosts in inventory
-file, or any combination of the two seperated by semicolons.
-
-
-OPTIONS
--------
-
-
-*-i*, *--inventory*::
-
-Path to the inventory hosts file, which defaults to /etc/ansible/hosts.
-
-
-*-f*, *--forks*::
-
-Level of parallelism. Specify as an integer, the default is 5.
-
-
-*-m*, *--module-name*::
-
-Module name to execute.
-
-
-*-p*, *--pattern*::
-
-Hostname pattern. Accepts shell-like globs which can be seperated with ";"
-The default is "*" which matches all hosts in the ansible hosts file.  Group
-names from the ansible inventory file can also be used.
-
-
-See ansible --help for additional options.
-
-
-INVENTORY
----------
-
-Ansible stores the hosts it can potentially operate on in an inventory
-file. The syntax is one host per line.  Groups headers are allowed and
-are included on their own line, enclosed in square brackets.
-
-FILES
------
-
-/etc/ansible/hosts -- Default hosts file
-
-/usr/share/ansible -- Default module library
-
-
-ENVIRONMENT
------------
-
-The following environment variables may specified.
-
-ANSIBLE_HOSTS  -- Override the default ansible hosts file
-
-ANSIBLE_LIBRARY -- Override the default ansible module library path
-
-
-AUTHOR
-------
-
-Ansible was originally written by Michael DeHaan. See the AUTHORS file
-for a complete list of contributors.
-
-
-COPYRIGHT
----------
-
-Copyright © 2012, Michael DeHaan
-
-Ansible is released under the terms of the GPLv3 License.
-
-
-SEE ALSO
---------
-
-Ansible home page: <https://ansible.github.com/>
-
-

docs/man/man1/ansible.1.asciidoc.in

@@ -0,0 +1,160 @@
+ansible(1)
+=========
+:doctype:manpage
+:man source:   Ansible
+:man version:  %VERSION%
+:man manual:   System administration commands
+
+NAME
+----
+ansible - run a command somewhere else
+
+
+SYNOPSIS
+--------
+ansible <host-pattern> [-f forks] [-m module_name] [-a args]
+
+
+DESCRIPTION
+-----------
+
+*Ansible* is an extra-simple tool/framework/API for doing \'remote things' over
+SSH.
+
+
+ARGUMENTS
+---------
+
+*host-pattern*::
+
+A name of a group in the inventory file, a shell-like glob selecting
+hosts in inventory file, or any combination of the two separated by
+semicolons.
+
+
+OPTIONS
+-------
+
+
+*-i* 'PATH', *--inventory=*'PATH'::
+
+The 'PATH' to the inventory hosts file, which defaults to '/etc/ansible/hosts'.
+
+
+*-f* 'NUM', *--forks=*'NUM'::
+
+Level of parallelism. 'NUM' is specified as an integer, the default is 5.
+
+
+*-m* 'NAME', *--module-name=*'NAME'::
+
+Execute the module called 'NAME'.
+
+
+*-M* 'DIRECTORY', *--module-path=*'DIRECTORY'::
+
+The 'DIRECTORY' to load modules from. The default is '/usr/share/ansible'.
+
+
+*-a* \'_ARGUMENTS_', *--args=*\'_ARGUMENTS_'::
+
+The 'ARGUMENTS' to pass to the module.
+
+*-D*, *--debug*::
+
+Debug mode
+
+*-k*, *--ask-pass*::
+
+Prompt for the SSH password instead of assuming key-based authentication with ssh-agent.
+
+*-K*, *--ask-sudo-pass*::
+
+Prompt for the password to use with --sudo, if any
+
+*-o*, *--one-line*::
+
+Try to output everything on one line.
+
+*-s*, *--sudo*::
+
+Run the command as the user given by -u and sudo to root.
+
+*-t* 'DIRECTORY', *--tree=*'DIRECTORY'::
+
+Save contents in this output 'DIRECTORY', with the results saved in a
+file named after each host.
+
+*-T* 'SECONDS', *--timeout=*'SECONDS'::
+
+Connection timeout to use when trying to talk to hosts, in 'SECONDS'.
+
+*-B* 'NUM', *--background=*'NUM'::
+
+Run commands in the background, killing the task after 'NUM' seconds.
+
+*-P* 'NUM', *--poll=*'NUM'::
+
+Poll a background job every 'NUM' seconds. Requires *-B*.
+
+*-u* 'USERNAME', *--remote-user=*'USERNAME'::
+
+Use this remote 'USERNAME' instead of root.
+
+*-U* 'SUDO_USERNAME', *--sudo-user=*'SUDO_USERNAME'::
+
+Sudo to 'SUDO_USERNAME' instead of root.  Implies --sudo.
+
+*-c* 'CONNECTION', *--connection=*'CONNECTION'::
+
+Connection type to use.  Possible options are 'paramiko' (SSH) and 'local'.
+Local is mostly useful for crontab or kickstarts.
+
+
+INVENTORY
+---------
+
+Ansible stores the hosts it can potentially operate on in an inventory
+file. The syntax is one host per line.  Groups headers are allowed and
+are included on their own line, enclosed in square brackets.
+
+FILES
+-----
+
+/etc/ansible/hosts -- Default inventory file
+
+/usr/share/ansible/ -- Default module library
+
+
+ENVIRONMENT
+-----------
+
+The following environment variables may specified.
+
+ANSIBLE_HOSTS  -- Override the default ansible hosts file
+
+ANSIBLE_LIBRARY -- Override the default ansible module library path
+
+
+AUTHOR
+------
+
+Ansible was originally written by Michael DeHaan. See the AUTHORS file
+for a complete list of contributors.
+
+
+COPYRIGHT
+---------
+
+Copyright © 2012, Michael DeHaan
+
+Ansible is released under the terms of the GPLv3 License.
+
+
+SEE ALSO
+--------
+
+*ansible-playbook*(1)
+
+Extensive documentation as well as IRC and mailing list info
+is available on the ansible home page: <https://ansible.github.com/>

docs/man/man5/ansible-modules.5.asciidoc

@@ -1,235 +0,0 @@
-ansible-modules(5)
-=================
-:doctype:manpage
-:man source: Ansible-modules
-:man version: 0.0.1
-:man manual: Ansible
-
-
-NAME
-----
-ansible-modules - stock modules shipped with ansible
-
-
-DESCRIPTION
------------
-
-Ansible ships with a number of modules that can be executed directly on remote hosts or through
-ansible playbooks.
-
-
-IDEMPOTENCE
------------
-
-Most modules other than command are idempotent, meaning they will seek to avoid changes
-unless a change needs to be made.  When using ansible playbooks, these modules can
-trigger change events, as described in *ansible-playbooks*(5).
-
-Unless otherwise noted, all modules support change hooks.
-
-
-command
--------
-
-The command module takes the command name followed by a list of arguments, space delimited.
-This is the only module that does not use key=value style parameters.
-
-Example usage::
-
-/sbin/shutdown -t now
-
-This module does not support change hooks.
-
-Returns the return code from the program as well as timing information.
-
-(Async command running and command execution time limits are in plan.)
-
-copy
-----
-
-The copy module moves a file on the local box to remote locations.
-
-*src=*::
-
-Local absolute path to a file to copy to the remote server
-
-
-*dest=*::
-
-Remote absolute path where the file should end up
-
-
-This module also returns md5sum information about the resultant file.
-
-
-facter
-------
-
-Runs the discovery program 'facter' on the remote system, returning
-JSON data that can be useful for inventory purposes.
-
-Requires that 'facter' and 'ruby-json' be installed on the remote end.
-
-This module is informative only - it takes no parameters & does not support change hooks,
-nor does it make any changes on the system.
-
-
-file
-----
-
-Ensures the ownership and permissions of files are as desired.
-
-Use copy or template first if you need to make sure a file is on the box.
-
-In plan.
-
-
-git
----
-
-Deploys software from git checkouts.
-
-*repo=*::
-
-git or http protocol address of the repo to checkout
-
-*dest=*::
-
-where to check it out, an absolute directory path
-
-*version=*::
-
-what version to check out -- either the git SHA, the literal string 'HEAD', or a tag name
-
-
-ohai
-----
-
-Similar to the facter module, this returns JSON inventory data.  Ohai
-data is a bit more verbose and nested than facter.
-
-Requires that 'ohai' be installed on the remote end.
-
-This module is information only - it takes no parameters & does not
-support change hooks, nor does it make any changes on the system.
-
-
-ping
-----
-
-A trivial test module, this module always returns the integer '1' on
-successful contact.
-
-This module does not support change hooks.
-
-This module is informative only - it takes no parameters & does not
-support change hooks, nor does it make any changes on the system.
-
-
-service
--------
-
-Controls services on remote machines.
-
-*state=*::
-
-Values are 'started', 'stopped', or 'restarted'.   Started/stopped
-are idempotent actions that will not run commands unless neccessary.
-'restarted' will always bounce the service
-
-
-*name=*::
-
-The name of the service
-
-
-setup
------
-
-Writes a JSON file containing key/value data, for use in templating.
-Call this once before using the template modules, usually as the very
-first step in your playbook.
-
-If facter or ohai are installed, variables from these programs will also
-be snapshotted into the JSON file for usage in templating. These variables
-are prefixed with 'facter_' and 'ohai_" so it's easy to tell their source.
-
-*metadata=*::
-
-Optionally overrides the default JSON file location of /etc/ansible/setup.
-If used, also supply the metadata parameter to 'template'.  Change if
-running as a non-root remote user who does not have permissions on /etc/ansible.
-
-*anything=*::
-
-any other parameters can be named basically anything, and set a key=value
-pair in the JSON file for use in templating.
-
-
-template
---------
-
-Templates a file out to a remote server.  Call the setup module prior to usage.
-
-*src=*::
-
-path of a Jinja2 formatted template on the local server
-
-
-*dest*::
-
-location to render the template on the remote server
-
-
-*metadata*::
-
-location of a JSON file to use to supply template data.  Default is /etc/ansible/setup
-which is the same as the default for the setup module.   Change if running as a non-root
-remote user who does not have permissions on /etc/ansible.
-
-
-This module also returns md5sum information about the resultant file.
-
-
-user
-----
-
-This module is in plan.
-
-
-yum
----
-
-This module is in plan.
-
-
-WRITING YOUR OWN MODULES
-------------------------
-
-To write your own modules, simply follow the convention of those already available in
-/usr/share/ansible.  Modules must return JSON but can be written in any language.
-To support change hooks, modules should return hashes, with a changed: True/False
-element at the top level.  Modules can also choose to indicate a failure scenario
-by returning a top level 'failure' element with a True value.
-
-
-ENVIRONMENT
------------
-
-ANSIBLE_LIBRARY -- Override the default ansible module library path
-
-
-AUTHOR
-------
-
-Ansible was originally written by Michael DeHaan. See the AUTHORS file
-for a complete list of contributors.
-
-SEE ALSO
---------
-
-*ansible*(1)
-
-*ansible-playbook*(5)
-
-Ansible home page: <https://github.com/mpdehaan/ansible/>

docs/man/man5/ansible-playbook.5.asciidoc

@@ -1,123 +0,0 @@
-ansible-modules(5)
-=================
-:doctype:manpage
-:man source: Ansible-playbook
-:man version: 0.0.1
-:man manual: Ansible
-
-
-NAME
-----
-ansible-playbook - format and function of an ansible playbook file
-
-
-DESCRIPTION
------------
-
-Ansible ships with 'ansible-playbook', a tool for running playbooks.
-Playbooks can represent frequent tasks, desired system configurations,
-or deployment processes.
-
-
-FORMAT
-------
-
-Playbooks are written in YAML.
-
-
-EXAMPLE
--------
-
-See:
-
-- https://github.com/mpdehaan/ansible/blob/master/examples/playbook.yml
-
-- https://github.com/mpdehaan/ansible/blob/master/examples/base.yml
-
-- https://github.com/mpdehaan/ansible/blob/master/examples/handlers.yml
-
-WHAT THE EXAMPLE MEANS
------------------------
-
-Here's what playbook.yml (above) will do.
-
-The first pattern will select all hosts.  The patterns are the same
-as supported by /usr/bin/ansible.
-
-First, it will run all the modules specified in base.yml.  Includes can
-be used to implement classes of things, and if you wanted, a playbook
-could consist of nothing but include files.  This is an example of an
-include.
-
-After processing base.yml, on each host we'll write for 
-a JSON file into /etc/ansible/setup on each remote system with the 
-values max_clients and http_port.
-
-Next, we'll use a Jinja2 template locally residing at
-/srv/templates/httpd.j2 to write the Apache config file on each host, 
-using the previous values in that setup file.
-
-Next, We'll ensure that apache is running if stopped.
-
-The template task set up a notifier, which means if the configuration
-file actually changed, we have a named handler, in this case, 'restart apache'
-to run.  In this case, all the notifiers come from handlers.yml, though it's
-also ok to express handlers directly in the main yaml file too.  Using
-the include promotes reuse.
-
-What does the handler say?  If and only if the config file changed, note that we need to restart
-apache at the end of the run, otherwise, don't bother because we
-already know it is running.
-
-
-HIGH LEVEL EXPLANATION
-----------------------
-
-Playbooks are executed top down and can contain multiple references to
-patterns.  For instance, a playbook could do something to all
-webservers, then do something to all database servers, then do
-something different to all webservers again.
-
-For each pattern, the tasks in the 'tasks' list are executed in order
-for all hosts in the host file matching the pattern.
-
-For each task, a name/action pair describes what the task is and what
-ansible module to use to accomplish the task, along with any
-arguments.   Additional fields like 'comment:' can be added and will
-be ignored, so feel free to take notes in the file.
-
-Most modules accept key=value format arguments.
-
-Handlers are like tasks, but are conditionally executed.  If a module
-reports a 'change', it can notify one or more handler by name.  If
-notified, it will run only for hosts that changed.
-
-
-ERROR HANDLING
---------------
-
-If a host has a failure, the host will be ignored for the remainder
-of the playbook execution.
-
-
-ENVIRONMENT
------------
-
-ANSIBLE_LIBRARY -- Override the default ansible module library path
-
-
-AUTHOR
-------
-
-Ansible was originally written by Michael DeHaan. See the AUTHORS file
-for a complete list of contributors.
-
-
-SEE ALSO
---------
-
-*ansible*(1)
-
-*ansible-modules*(5)
-
-Ansible home page: <https://github.com/mpdehaan/ansible/>

examples/ansible_hosts

@@ -1,12 +0,0 @@
-[webservers]
-alpha.example.org
-beta.example.org
-192.168.1.100
-192.168.1.110
-
-[dbservers]
-192.168.1.200
-192.168.1.201
-foo.example.org
-bar.example.org
-

examples/base.yml

@@ -1,7 +0,0 @@
----
-- name: no selinux
-  action: command /usr/sbin/setenforce 0
-- name: no iptables
-  action: service name=iptables state=stopped
-- name: this is just to show variables work here, favcolor={{ favcolor }}
-  action: command /bin/true

examples/handlers.yml

@@ -1,5 +0,0 @@
----
-- name: restart apache
-  action: service name=httpd state=restarted
-- name: restart memcached
-  action: service name=memcached state=restarted

examples/hosts

@@ -0,0 +1,37 @@
+# This is the default ansible 'hosts' file.
+#
+# It should live in /etc/ansible/hosts
+#
+#   - Comments begin with the '#' character
+#   - Blank lines are ignored
+#   - Groups of hosts are delimited by [header] elements
+#   - You can enter hostnames or ip addresses
+#   - A hostname/ip can be a member of multiple groups
+#
+# Ex 1: Ungrouped hosts, specify before any group headers.
+green.bikeshed.org
+blue.bikeshed.org
+red.bikeshed.org
+bikeshed.org
+bastion.secure.bikeshed.org
+192.168.100.1
+192.168.100.10
+
+# Ex 2: A collection of hosts belonging to the 'webservers' group
+[webservers]
+www01.bikeshed.org
+www02.bikeshed.org
+wheel.colors.com
+192.168.1.100
+192.168.1.110
+# Your personal website also runs a webserver:
+myserver.com
+
+# Ex 3: A collection of database servers in the 'dbservers' group
+[dbservers]
+db01.intranet.mydomain.net
+10.25.1.56
+db02.intranet.mydomain.net
+10.25.1.57
+# Perhaps you serve a db off your personal server too:
+myserver.com

examples/playbook.yml

@@ -1,18 +0,0 @@
----
-- hosts: all
-  user: root
-  vars:
-    http_port: 80
-    max_clients: 200
-  tasks:
-  - include: base.yml favcolor=blue
-  - name: write the foo config file using vars set above
-    action: template src=foo.j2 dest=/etc/some_random_foo.conf
-    notify:
-    - restart apache
-  - name: ensure apache is running
-    action: service name=httpd state=started
-  - name: pointless test action
-    action: command /bin/echo {{ http_port }}
-  handlers:
-    - include: handlers.yml

examples/playbook2.yml

@@ -1,24 +0,0 @@
----
-- hosts: '*'
-  vars:
-    a: 2
-    b: 3
-    c: 4
-  tasks:
-  - name: copy comand
-    action: copy src=/srv/a dest=/srv/b
-    notify: 
-    - restart apache
-  - name: template step
-    action: template src=/srv/template.j2 dest=/srv/file.out
-    notify:
-    - restart apache
-  - name: execute bin false
-    comment: call something that will fail just to demo failure counts and such
-    action: command /bin/false
-  - name: execute bin true
-    comment: this will never be executed because previous will fail
-    action: command /bin/true
-  handlers:
-    - name: restart apache
-      action: service name=httpd state=restarted

examples/playbooks/ansible_pull.yml

@@ -0,0 +1,51 @@
+# ansibple-pull setup 
+#
+# on remote hosts, set up ansible to run periodically using the latest code
+# from a particular checkout, in pull based fashion, inverting Ansible's
+# usual push-based operating mode.  
+#
+# This particular pull based mode is ideal for:
+#
+# (A) massive scale out
+# (B) continual system remediation
+#
+# DO NOT RUN THIS AGAINST YOUR HOSTS WITHOUT CHANGING THE repo_url
+# TO SOMETHING YOU HAVE PERSONALLY VERIFIED
+#
+# 
+---
+
+- hosts: pull_mode_hosts
+  user: root
+
+  vars:
+
+    # schdule is fed directly to cron
+    schedule: '*/15 * * * *'
+
+    # User to run ansible-pull as from cron
+    cron_user: root
+
+    # Directory to where repository will be cloned
+    workdir: /var/lib/ansible/local
+
+    # Repository to check out -- YOU MUST CHANGE THIS
+    # repo must contain a local.yml file at top level
+    #repo_url: git://github.com/sfromm/ansible-playbooks.git
+    repo_url: SUPPLY_YOUR_OWN_GIT_URL_HERE
+
+  tasks:
+
+    - name: Install ansible
+      action: yum pkg=ansible state=installed
+
+    - name: Create local directory to work from
+      action: file path=$workdir state=directory owner=root group=root mode=0751
+
+    - name: Copy ansible inventory file to client
+      action: copy src=/etc/ansible/hosts dest=/etc/ansible/hosts
+              owner=root group=root mode=0644
+
+    - name: Create crontab entry to clone/pull git repository
+      action: template src=templates/ansible-pull.j2 dest=/etc/cron.d/ansible-pull owner=root group=root mode=0644
+

examples/playbooks/conditionals_part1.yml

@@ -0,0 +1,51 @@
+---
+# this is a demo of conditional imports.  This is a powerful concept
+# and can be used to use the same recipe for different types of hosts,
+# based on variables that bubble up from the hosts from tools such
+# as ohai or facter.
+#
+# Here's an example use case:
+#
+# what to do if the service for apache is named 'httpd' on CentOS
+# but is named 'apache' on Debian?
+
+
+# there is only one play in this playbook, it runs on all hosts
+# as root
+
+- hosts: all
+  user: root
+
+# we have a common list of variables stored in /vars/external_vars.yml
+# that we will always import
+
+# next, we want to import files that are different per operating system
+# and if no per operating system file is found, load a defaults file.
+# for instance, if the OS was "CentOS", we'd try to load vars/CentOS.yml.
+# if that was found, we would immediately stop.  However if that wasn't
+# present, we'd try to load vars/defaults.yml.  If that in turn was not
+# found, we would fail immediately, because we had gotten to the end of
+# the list without importing anything.
+
+  vars_files:
+
+
+     - "vars/external_vars.yml"
+
+     - [ "vars/$facter_operatingsystem.yml", "vars/defaults.yml" ]
+
+# and this is just a regular task line from a playbook, as we're used to.
+# but with variables in it that come from above.  Note that the variables
+# from above are *also* available in templates
+     
+  tasks:
+
+  - name: ensure apache is latest
+    action: $packager pkg=$apache state=latest
+  - name: ensure apache is running
+    action: service name=$apache state=running
+  - name: fail
+    action: command /bin/false
+
+ 
+

examples/playbooks/conditionals_part2.yml

@@ -0,0 +1,47 @@
+---
+# this is a demo of conditional executions using 'only_if', which can skip
+# certain tasks on machines/platforms/etc where they do not apply.
+
+- hosts: all
+  user: root
+
+  vars:
+     favcolor: "red"
+     ssn: 8675309
+
+# Below we're going to define some expressions.
+#
+# Not only can we assign variables for reuse, but we can also assign conditional
+# expressions.  By keeping these in 'vars', the task section remains
+# extraordinarily clean, and not littered with programming language
+# constructs -- so it's easily skimmed by humans.
+#
+# Remember to quote any variables if they are not numbers!  
+#
+# Interesting fact: aside from the $variables, these expressions are actually 
+# tiny bits of Python.  They are evaluated in the context of each host, so different
+# steps can be skipped on different hosts!  They should evaluate to either True
+# or False
+
+     is_favcolor_blue: "'$favcolor' == 'blue'"
+     is_centos: "'$facter_operatingsystem' == 'CentOS'"
+
+# NOTE:
+#
+# facter and ohai variables can be used in only_if statements too
+# ex:  "'$facter_operatingsystem' == 'CentOS'", which bubble up automatically
+# from the managed machines
+#
+# this example won't do that though, as you might not have facter or ohai,
+# but you get the idea...
+
+  tasks:
+
+     - name: "do this if my favcolor is blue"
+       action: shell /bin/false
+       only_if: '$is_favcolor_blue'
+     
+     - name: "do this if my favcolor is not blue"
+       action: shell /bin/true
+       only_if: 'not ($is_favcolor_blue)'
+

examples/playbooks/file_secontext.yml

@@ -0,0 +1,18 @@
+---
+# This is a demo of how to manage the selinux context using the file module
+- hosts: test
+  user: root
+  tasks:
+    - name: Change setype of /etc/exports to non-default value
+      action: file path=/etc/exports setype=etc_t
+    - name: Change seuser of /etc/exports to non-default value
+      action: file path=/etc/exports seuser=unconfined_u
+    - name: Set selinux context back to default value
+      action: file path=/etc/exports context=default
+    - name: Create empty file
+      action: command /bin/touch /tmp/foo
+    - name: Change setype of /tmp/foo
+      action: file path=/tmp/foo setype=default_t
+    - name: Try to set secontext to default, but this will fail
+            because of the lack of a default in the policy
+      action: file path=/tmp/foo context=default

examples/playbooks/group_commands.yml

@@ -0,0 +1,18 @@
+---
+# This is a demo of how the group command works.
+
+- hosts: all
+  user: root
+  sudo: True
+
+  tasks:
+
+    # Walk through group creation, modification, and deletion
+    - name: create a group
+      action: group name=tset
+
+    # You can only modify the group's gid
+    - action: group name=tset gid=7777
+
+    # And finally remove the group
+    - action: group name=tset state=absent

examples/playbooks/handlers/handlers.yml

@@ -0,0 +1,10 @@
+---
+
+# this is an example to show that handlers can be included from yaml files,
+# to promote reuse between different plays or even playbooks.  They work
+# just like normal handlers.
+
+- name: restart apache
+  action: service name=httpd state=restarted
+- name: restart memcached
+  action: service name=memcached state=restarted

examples/playbooks/intermediate_example.yml

@@ -0,0 +1,89 @@
+---
+# see examples.yml first!
+# This file explains some more advanced features of playbooks.  
+# because of the comments it's less concise than it normally is.  But feel
+# free to comment your playbooks if you like.
+
+- hosts: dbservers
+
+  # we can define variables the normal way...
+
+  vars:
+    release: 2.0
+
+  # but they can also come from other files.  This can be a relative
+  # or absolute path.  This is a good way to store 'secret' variable
+  # files but still keep the playbook in public source control
+
+  vars_files:
+    - vars/external_vars.yml
+
+  # as with before, every play has a list of tasks in it
+
+  tasks:
+
+  # tasks can be written the normal way...
+
+  - name: arbitrary command
+    action: command /bin/true
+
+  # or we can promote reuse and simplicity by including tasks
+  # from other files, for instance, to reuse common tasks
+
+  - include: tasks/base.yml
+
+  # we could also have done something like:
+  # - include: wordpress.yml user=timmy
+  # and had access to the template variable $user in the 
+  # included file, if we wanted to.  Variables from vars
+  # and vars_files are also available inside include files
+
+  handlers:
+ 
+    # handlers can also be included from files, to promote reuse
+    # and simpler recipes, you may wish to only have one
+    # handler file for all your plays and playbooks
+
+    - include: handlers/handlers.yml
+
+    # you can mix things that are directly in the file with things
+    # that are included.  Order is executed as written, but only
+    # handlers that have been notified get executed
+
+    - name: restart foo
+      action: service name=foo state=restarted
+
+# ===============================================================
+
+# Here's a second play in the same playbook.  This will be run
+# after the first playbook completes on all hosts.  You may want
+# a different play for each class of systems, or may want a different
+# play for each stage in a complex multi-node deployment push
+# process.  How you use them are up to you.
+
+# any play in a playbook can be executed by a user other than root
+# if you want.  sudo support is coming too.
+
+- hosts: webservers
+  user: mdehaan
+
+  # vars must be specified again for the next play in the playbook
+  # but can be reused by including from vars_files if you want
+  # you can use vars, vars_files, or both.  vars_files overrides
+  # those set in vars. 
+
+  vars:
+     release: 2.0
+  vars_files:
+     - vars/external_vars.yml
+
+
+  # these all runs as the user 'mdehaan'.  If there were any handlers
+  # they would as well.
+
+  tasks:
+
+     - name: some random command
+       action: command /bin/true
+
+

examples/playbooks/intro_example.yml

@@ -0,0 +1,76 @@
+---
+# this is an annotated example of some features available in playbooks
+# it shows how to make sure packages are updated, how to make sure
+# services are running, and how to template files.  It also demos
+# change handlers that can restart things (or trigger other actions)
+# when resources change.  For more advanced examples, see example2.yml
+
+# on all hosts, run as the user root...
+
+- name: example play
+  hosts: all
+  user: root
+
+# could have also have done:
+#  user: mdehaan
+#  sudo: True
+
+  # make these variables available inside of templates
+  # for when we use the 'template' action/module later on...
+
+  vars:
+    http_port: 80
+    max_clients: 200
+
+  # define the tasks that are part of this play...
+
+  tasks:
+
+  # task #1 is to run an arbitrary command
+  # we'll simulate a long running task, wait for up to 45 seconds, poll every 5
+  # obviously this does nothing useful but you get the idea
+
+  - name: longrunner
+    action: command /bin/sleep 15
+    async: 45
+    poll: 5
+
+  # let's demo file operations.
+  #
+  # We can 'copy' files or 'template' them instead, using jinja2
+  # as the templating engine.  This is done using the variables 
+  # from the vars section above mixed in with variables bubbled up 
+  # automatically from tools like facter and ohai.  'copy' 
+  # works just like 'template' but does not do variable subsitution.
+  #
+  # If and only if the file changes, restart apache at the very
+  # end of the playbook run
+
+  - name: write some_random_foo configuration
+    action: template src=templates/foo.j2 dest=/etc/some_random_foo.conf
+    notify:
+    - restart apache
+
+  # make sure httpd is installed at the latest version
+
+  - name: install httpd
+    action: yum pkg=httpd state=latest
+
+  # make sure httpd is running
+
+  - name: httpd start
+    action: service name=httpd state=running
+
+  # handlers are only run when things change, at the very end of each
+  # play.  Let's define some.  The names are significant and must
+  # match the 'notify' sections above
+
+  handlers:
+
+    # this particular handler is run when some_random_foo.conf
+    # is changed, and only then
+
+    - name: restart apache
+      action: service name=httpd state=restarted
+
+

examples/playbooks/loop_with_items.yml

@@ -0,0 +1,30 @@
+---
+# this is an example of how to run repeated task elements over lists
+# of items, for example, installing multiple packages or configuring
+# multiple users
+
+- hosts: all
+  user: root
+
+  tasks:
+
+  - name: install $item
+    action: yum pkg=$item state=installed
+    with_items:
+       - cobbler
+       - httpd
+
+  - name: configure user $item
+    action: user name=$item state=present groups=wheel
+    with_items:
+       - testuser1
+       - testuser2
+
+  - name: remove user $item
+    action: user name=$item state=absent
+    with_items:
+        - testuser1
+        - testuser2
+
+
+

examples/playbooks/prompts.yml

@@ -0,0 +1,33 @@
+---
+
+# it is possible to ask for variables from the user at the start
+# of a playbook run, for example, as part of a release script.
+
+- hosts: all
+  user: root
+
+# regular variables are a dictionary of keys and values
+
+  vars:
+     this_is_a_regular_var: 'moo'
+     so_is_this: 'quack'
+
+# prompted variables are a list of variable names and a description
+# that will be presented to the user.
+#
+# alternatively, they can ALSO be passed in from the outside:
+#    ansible-playbook foo.yml --extra-vars="foo=100 bar=101"
+# or through external inventory scripts (see online API docs)
+
+  vars_prompt:
+     release_version: "product release version"
+
+  tasks:
+
+# this is just a simple example to show that vars_prompt works, but
+# you might ask for a tag to use with the git module or perhaps
+# a package version to use with the yum module.
+
+  - name: imagine this did something interesting with $release_version
+    action: shell echo foo >> /tmp/$release_version-$alpha
+

examples/playbooks/selective_file_sources.yml

@@ -0,0 +1,28 @@
+---
+# this is an example of how to template a file over using some variables derived
+# from the system.  For instance, if you wanted to have different configuration
+# templates by OS version, this is a neat way to do it.  Any Ansible facts, facter facts,
+# or ohai facts could be used to do this.  
+
+- hosts: all
+
+  tasks:
+
+  - name: template a config file
+    action: template dest=/etc/imaginary_file.conf
+    first_available_file:
+
+       # first see if we have a file for this specific host
+       - /srv/whatever/${ansible_hostname}.conf
+
+       # next try to load something like CentOS6.2.conf
+       - /srv/whatever/${ansible_distribution}${ansible_distribution_version}.conf
+
+       # next see if there's a CentOS.conf
+       - /srv/whatever/${ansible_distribution}.conf
+
+       # finally give up and just use something generic
+       - /srv/whatever/default
+
+
+

examples/playbooks/tasks/base.yml

@@ -0,0 +1,22 @@
+---
+
+# this is the example of an included tasks file.  It contains a flat list of tasks
+# they can notify other tasks, and have full access to variables from 'vars'
+# or 'vars_files' directives.  Further, if ohai or facter were installed on 
+# the remote machines, variables from those tools can be accessed on the 'action'
+# line or in templates.  Just prefix with 'facter_' or 'ohai_' before the particular
+# variable.
+
+# possible uses for a included yaml file might be to represent a 'class' of a system
+# like defining what makes up a webserver, or you might have a common 'base.yml'
+# (like this) that might be applied to all your systems as well.
+
+- name: no selinux
+  action: command /usr/sbin/setenforce 0
+
+- name: no iptables
+  action: service name=iptables state=stopped
+
+- name: made up task just to show variables work here
+  action: command /bin/echo release is $release
+

examples/playbooks/templates/ansible-pull.j2

@@ -0,0 +1,2 @@
+# Cron job to git clone/pull a repo and then run locally
+{{ schedule }} {{ cron_user }} ansible-pull -d {{ workdir }} -U {{ repo_url }} >/var/log/ansible-pull.log 2>&1

examples/playbooks/templates/foo.j2

@@ -0,0 +1,4 @@
+# This is a very simple Jinja2 template representing an imaginary configuration file
+# for an imaginary app.
+
+http_port={{ http_port }}

examples/playbooks/user_commands.yml

@@ -0,0 +1,38 @@
+---
+# this is a demo of how the user commands work and how to reference salted passwords
+# in vars sections.  You could also use vars_files if you like (see other examples)
+
+- hosts: all
+  user: root
+  vars:
+    # created with:
+    # crypt.crypt('This is my Password', '$1$SomeSalt')
+    password: $1$SomeSalt$UqddPX3r4kH3UL5jq5/ZI.
+
+  tasks:
+
+    # Walk through account creation, modification, and deletion
+    - name: test basic user account creation
+      action: user name=tset comment=TsetUser group=users shell=/sbin/nologin createhome=no
+
+    # the following is just a simple example of how you don't have to include
+    # the 'name' element for each task
+
+    - action: user name=tset comment=NyetUser
+    - action: user name=tset password=$password
+
+    # The following will add the user to supplementary groups.
+
+    # Add the user to the groups dialout and uucp.
+    - action: user name=tset groups=dialout,uucp
+
+    # Add the user to the groups dialout and wheel,
+    # This will remove tset from the group uucp.
+    - action: user name=tset groups=dialout,wheel
+
+    # Add the user to the group uucp.  Because append=yes, the user
+    # will not be removed from the groups dialout and wheel.
+    - action: user name=tset groups=uucp append=yes
+
+    # Finally, remove the user.
+    - action: user name=tset state=absent

examples/playbooks/vars/CentOS.yml

@@ -0,0 +1,3 @@
+---
+apache: httpd
+packager: yum

examples/playbooks/vars/defaults.yml

@@ -0,0 +1,3 @@
+---
+packager: aptitude
+apache: apache

examples/playbooks/vars/external_vars.yml

@@ -0,0 +1,3 @@
+---
+alpha: one
+beta: two

examples/scripts/cobbler_external_inventory.py

@@ -0,0 +1,127 @@
+#!/usr/bin/python
+
+"""
+Cobbler external inventory script
+=================================
+
+Ansible has a feature where instead of reading from /etc/ansible/hosts
+as a text file, it can query external programs to obtain the list
+of hosts, groups the hosts are in, and even variables to assign to each host.  
+
+To use this, copy this file over /etc/ansible/hosts and chmod +x the file.
+This, more or less, allows you to keep one central database containing
+info about all of your managed instances.
+
+This script is an example of sourcing that data from Cobbler
+(http://cobbler.github.com).  With cobbler each --mgmt-class in cobbler 
+will correspond to a group in Ansible, and --ks-meta variables will be 
+passed down for use in templates or even in argument lines.
+
+NOTE: The cobbler system names will not be used.  Make sure a
+cobbler --dns-name is set for each cobbler system.   If a system
+appears with two DNS names we do not add it twice because we don't want
+ansible talking to it twice.  The first one found will be used. If no
+--dns-name is set the system will NOT be visible to ansible.  We do
+not add cobbler system names because there is no requirement in cobbler
+that those correspond to addresses.  
+
+See http://ansible.github.com/api.html for more info
+
+Tested with Cobbler 2.0.11. 
+"""
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible,
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+######################################################################
+
+
+import sys
+import xmlrpclib
+import shlex
+
+try:
+    import json
+except:
+    import simplejson as json
+
+# NOTE -- this file assumes Ansible is being accessed FROM the cobbler
+# server, so it does not attempt to login with a username and password.
+# this will be addressed in a future version of this script.
+
+conn = xmlrpclib.Server("http://127.0.0.1/cobbler_api", allow_none=True)
+
+###################################################
+# executed with no parameters, return the list of
+# all groups and hosts
+
+if len(sys.argv) == 1:
+
+    systems = conn.get_item_names('system')
+    groups = { 'ungrouped' : [] }
+
+    for system in systems:
+
+        data = conn.get_blended_data(None, system)
+
+        dns_name = None
+        interfaces = data['interfaces']
+        for (iname, ivalue) in interfaces.iteritems():
+            this_dns_name = ivalue.get('dns_name', None)
+            if this_dns_name is not None:
+                dns_name = this_dns_name       
+ 
+        if dns_name is None:
+            continue
+
+        classes  = data['mgmt_classes']
+        for cls in classes:
+            if cls not in groups:
+                groups[cls] = []
+            # hostname is not really what we want to insert, really insert the
+            # first DNS name but no further DNS names
+            groups[cls].append(dns_name)
+ 
+    print json.dumps(groups)
+    sys.exit(0)
+
+#####################################################
+# executed with a hostname as a parameter, return the
+# variables for that host
+
+if len(sys.argv) == 2:
+
+    # look up the system record for the given DNS name
+    result = conn.find_system_by_dns_name(sys.argv[1])
+    system = result.get('name', None)
+    data = {}
+    if system is None:
+        print json.dumps({})
+        sys.exit(1)
+    data = conn.get_system_for_koan(system)
+
+    # return the ksmeta data for that system
+    metadata = data['ks_meta']
+    tokens = shlex.split(metadata)
+    results = {}
+    for t in tokens:
+        if t.find("=") != -1:
+           (k,v) = t.split("=",1)
+           results[k]=v
+    print json.dumps(results)
+    sys.exit(0)
+

hacking/README

@@ -6,6 +6,3 @@ To use it from the root of a checkout:
     $ . ./hacking/env-setup
 
 Note the space between the '.' and the './'
-
-Man pages will not load until you run 'make docs' from the root of the
-checkout.

hacking/env-setup

@@ -1,18 +1,20 @@
-# -*- mode: shell-script -*-
-PREFIX_PYTHONPATH="`pwd`/lib"
-PREFIX_PATH="`pwd`/bin"
-PREFIX_MANPATH="`pwd`/docs/man"
-PREFIX_ANS_LIBRARY="`pwd`/library"
+#!/bin/bash
+# usage: source ./hacking/env-setup
+#    modifies environment for running Ansible from checkout
+
+PREFIX_PYTHONPATH="$PWD/lib"
+PREFIX_PATH="$PWD/bin"
+PREFIX_MANPATH="$PWD/docs/man"
 
-echo "Prefixing PYTHONPATH with $PREFIX_PYTHONPATH"
 export PYTHONPATH=$PREFIX_PYTHONPATH:$PYTHONPATH
-
-echo "Prefixing PATH with $PREFIX_PATH"
 export PATH=$PREFIX_PATH:$PATH
-echo $PATH
-
-echo "Prefixing MANPATH with $PREFIX_MANPATH"
+export ANSIBLE_LIBRARY="$PWD/library"
 export MANPATH=$PREFIX_MANPATH:$MANPATH
 
-echo "Setting ANSIBLE_LIBRARY to ${PREFIX_ANS_LIBRARY}"
-export ANSIBLE_LIBRARY=${PREFIX_ANS_LIBRARY}
+echo "PATH=$PATH"
+echo "PYTHONPATH=$PYTHONPATH"
+echo "ANSIBLE_LIBRARY=$ANSIBLE_LIBRARY"
+echo "MANPATH=$MANPATH"
+
+echo "Reminder: specify your host file with -i"
+echo "Done."

hacking/test-module

@@ -0,0 +1,91 @@
+#!/usr/bin/python
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# this script is for testing modules without running through the
+# entire guts of ansible, and is very helpful for when developing
+# modules
+#
+# example:
+#    test-module ../library/command /bin/sleep 3
+#    test-module ../library/service name=httpd ensure=restarted
+
+import sys
+import os
+import subprocess
+import traceback
+from ansible import utils
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+
+modfile = None
+
+if len(sys.argv) == 1:
+    print >>sys.stderr, "usage: test-module ./library/command [key=value ...]"
+    sys.exit(1)
+
+modfile = sys.argv[1]
+if len(sys.argv) > 1:
+    args = " ".join(sys.argv[2:])
+else:
+    args = ""
+
+argspath = os.path.expanduser("~/.ansible_test_module_arguments")
+argsfile = open(argspath, 'w')
+argsfile.write(args)
+argsfile.close()
+
+os.system("chmod +x %s" % modfile)
+cmd = subprocess.Popen("%s %s" % (modfile, argspath), 
+    shell=True,
+    stdout=subprocess.PIPE, 
+    stderr=subprocess.PIPE)
+(out, err) = cmd.communicate()
+
+if err and err != '':
+    print "***********************************"
+    print "RECIEVED DATA ON STDERR, THIS WILL CRASH YOUR MODULE"
+    print err
+    sys.exit(1)
+
+try:
+    print "***********************************"
+    print "RAW OUTPUT"
+    print out
+    results = utils.parse_json(out)
+    
+except:
+    print "***********************************"
+    print "INVALID OUTPUT FORMAT"
+    print out
+    traceback.print_exc()
+    sys.exit(1)
+
+print "***********************************"
+print "PARSED OUTPUT"
+
+print utils.bigjson(results)
+
+sys.exit(0)
+
+
+

lib/ansible/__init__.py

@@ -14,5 +14,5 @@
 #
 # You should have received a copy of the GNU General Public License
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
-__version__ = '0.0.1'
+__version__ = '0.4'
 __author__ = 'Michael DeHaan'

lib/ansible/callbacks.py

@@ -0,0 +1,248 @@
+# (C) 2012, Michael DeHaan, <michael.dehaan@gmail.com>
+
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#######################################################
+
+import utils
+import sys
+import getpass
+
+#######################################################
+
+class AggregateStats(object):
+    ''' holds stats about per-host activity during playbook runs '''   
+ 
+    def __init__(self):
+        self.processed   = {}
+        self.failures    = {}
+        self.ok          = {}
+        self.dark        = {}
+        self.changed     = {}
+        self.skipped     = {}
+
+    def _increment(self, what, host):
+        ''' helper function to bump a statistic '''
+
+        self.processed[host] = 1
+        prev = (getattr(self, what)).get(host, 0)
+        getattr(self, what)[host] = prev+1
+
+    def compute(self, runner_results, setup=False, poll=False):
+        ''' walk through all results and increment stats '''
+ 
+        for (host, value) in runner_results.get('contacted', {}).iteritems():
+            if ('failed' in value and bool(value['failed'])) or ('rc' in value and value['rc'] != 0):
+                self._increment('failures', host)
+            elif 'skipped' in value and bool(value['skipped']):
+                self._increment('skipped', host)
+            elif 'changed' in value and bool(value['changed']):
+                if not setup and not poll:
+                    self._increment('changed', host)
+                self._increment('ok', host)
+            else:
+                if not poll or ('finished' in value and bool(value['finished'])):
+                    self._increment('ok', host)
+
+        for (host, value) in runner_results.get('dark', {}).iteritems():
+            self._increment('dark', host)
+            
+
+    def summarize(self, host):
+        ''' return information about a particular host '''
+
+        return dict(
+            ok          = self.ok.get(host, 0),
+            failures    = self.failures.get(host, 0),
+            unreachable = self.dark.get(host,0),
+            changed     = self.changed.get(host, 0),
+            skipped     = self.skipped.get(host, 0)
+        )
+
+########################################################################
+
+class DefaultRunnerCallbacks(object):
+    ''' no-op callbacks for API usage of Runner() if no callbacks are specified '''
+
+    def __init__(self):
+        pass
+
+    def on_failed(self, host, res):
+        pass
+
+    def on_ok(self, host, res):
+        pass
+
+    def on_error(self, host, msg):
+        pass
+
+    def on_skipped(self, host):
+        pass
+
+    def on_unreachable(self, host, res):
+        pass
+
+    def on_no_hosts(self):
+        pass
+
+    def on_async_poll(self, host, res, jid, clock):
+        pass
+
+    def on_async_ok(self, host, res, jid):
+        pass
+
+    def on_async_failed(self, host, res, jid):
+        pass
+
+########################################################################
+
+class CliRunnerCallbacks(DefaultRunnerCallbacks):
+    ''' callbacks for use by /usr/bin/ansible '''
+
+    def __init__(self):
+        # set by /usr/bin/ansible later
+        self.options = None 
+        self._async_notified = {}
+
+    def on_failed(self, host, res):
+        invocation = res.get('invocation','')
+        if not invocation.startswith('async_status'):
+            self._on_any(host,res)
+
+    def on_ok(self, host, res):
+        invocation = res.get('invocation','')
+        if not invocation.startswith('async_status'):
+            self._on_any(host,res)
+ 
+    def on_unreachable(self, host, res):
+        print "%s | FAILED => %s" % (host, res)
+        if self.options.tree:
+            utils.write_tree_file(self.options.tree, host, utils.bigjson(dict(failed=True, msg=res)))
+ 
+    def on_skipped(self, host):
+        pass
+
+    def on_error(self, host, err):
+        print >>sys.stderr, "err: [%s] => %s\n" % (host, err)
+    
+    def on_no_hosts(self):
+        print >>sys.stderr, "no hosts matched\n"
+
+    def on_async_poll(self, host, res, jid, clock):
+        if jid not in self._async_notified:
+            self._async_notified[jid] = clock + 1
+        if self._async_notified[jid] > clock:
+            self._async_notified[jid] = clock
+            print "<job %s> polling, %ss remaining"%(jid, clock)
+
+    def on_async_ok(self, host, res, jid):
+        print "<job %s> finished on %s => %s"%(jid, host, utils.bigjson(res))
+
+    def on_async_failed(self, host, res, jid):
+        print "<job %s> FAILED on %s => %s"%(jid, host, utils.bigjson(res))
+
+    def _on_any(self, host, result):
+        print utils.host_report_msg(host, self.options.module_name, result, self.options.one_line)
+        if self.options.tree:
+            utils.write_tree_file(self.options.tree, host, utils.bigjson(result))
+
+########################################################################
+
+class PlaybookRunnerCallbacks(DefaultRunnerCallbacks):
+    ''' callbacks used for Runner() from /usr/bin/ansible-playbook '''
+
+    def __init__(self, stats):
+        self.stats = stats
+        self._async_notified = {}
+
+    def on_unreachable(self, host, msg):
+        print "fatal: [%s] => %s" % (host, msg)
+
+    def on_failed(self, host, results):
+        invocation = results.get('invocation',None)
+        if not invocation or invocation.startswith('setup ') or invocation.startswith('async_status '):
+            print "failed: [%s] => %s\n" % (host, utils.smjson(results))
+        else: 
+            print "failed: [%s] => %s => %s\n" % (host, invocation, utils.smjson(results))
+
+    def on_ok(self, host, host_result):
+        invocation = host_result.get('invocation','')
+        if invocation.startswith('async_status'):
+            pass
+        elif not invocation or invocation.startswith('setup '):
+            print "ok: [%s]\n" % (host)
+        else:
+            print "ok: [%s] => %s\n" % (host, invocation)
+
+    def on_error(self, host, err):
+        print >>sys.stderr, "err: [%s] => %s\n" % (host, err)
+
+    def on_skipped(self, host):
+        print "skipping: [%s]\n" % host
+
+    def on_no_hosts(self):
+        print "no hosts matched or remaining\n"
+
+    def on_async_poll(self, host, res, jid, clock):
+        if jid not in self._async_notified:
+            self._async_notified[jid] = clock + 1
+        if self._async_notified[jid] > clock:
+            self._async_notified[jid] = clock
+            print "<job %s> polling, %ss remaining"%(jid, clock)
+
+    def on_async_ok(self, host, res, jid):
+        print "<job %s> finished on %s"%(jid, host)
+
+    def on_async_failed(self, host, res, jid):
+        print "<job %s> FAILED on %s"%(jid, host)
+
+########################################################################
+
+class PlaybookCallbacks(object):
+    ''' playbook.py callbacks used by /usr/bin/ansible-playbook '''
+  
+    def __init__(self):
+        pass
+
+    def on_start(self):
+        print "\n"
+
+    def on_notify(self, host, handler):
+        pass
+
+    def on_task_start(self, name, is_conditional):
+        print utils.task_start_msg(name, is_conditional)
+
+    def on_vars_prompt(self, varname, private=True):
+        msg = 'input for %s: ' % varname
+        if private:
+            return getpass.getpass(msg)
+        return raw_input(msg)
+        
+    def on_setup_primary(self):
+        print "SETUP PHASE ****************************\n"
+    
+    def on_setup_secondary(self):
+        print "\nVARIABLE IMPORT PHASE ******************\n"
+
+    def on_import_for_host(self, host, imported_file):
+        print "%s: importing %s" % (host, imported_file)
+
+    def on_not_import_for_host(self, host, missing_file):
+        print "%s: not importing file: %s" % (host, missing_file)
+
+    def on_play_start(self, pattern):
+        print "PLAY [%s] ****************************\n" % pattern

lib/ansible/connection.py

@@ -0,0 +1,226 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+################################################
+
+import warnings
+import traceback
+import os
+import time
+import re
+import shutil
+import subprocess
+import pipes
+import socket
+import random
+
+from ansible import errors
+# prevent paramiko warning noise
+# see http://stackoverflow.com/questions/3920502/
+with warnings.catch_warnings():
+    warnings.simplefilter("ignore")
+    import paramiko
+
+
+################################################
+
+class Connection(object):
+    ''' Handles abstract connections to remote hosts '''
+
+    _LOCALHOSTRE = re.compile(r"^(127.0.0.1|localhost|%s)$" % os.uname()[1])
+
+    def __init__(self, runner, transport,sudo_user):
+        self.runner = runner
+        self.transport = transport
+        self.sudo_user = sudo_user
+    def connect(self, host, port=None):
+        conn = None
+        if self.transport == 'local' and self._LOCALHOSTRE.search(host):
+            conn = LocalConnection(self.runner, host)
+        elif self.transport == 'paramiko':
+            conn = ParamikoConnection(self.runner, host, port)
+        if conn is None:
+            raise Exception("unsupported connection type")
+        return conn.connect()
+
+################################################
+# want to implement another connection type?
+# follow duck-typing of ParamikoConnection
+# you may wish to read config files in __init__
+# if you have any.  Paramiko does not need any.
+
+class ParamikoConnection(object):
+    ''' SSH based connections with Paramiko '''
+
+    def __init__(self, runner, host, port=None):
+        self.ssh = None
+        self.runner = runner
+        self.host = host
+        self.port = port
+        if port is None:
+            self.port = self.runner.remote_port
+
+    def _get_conn(self):
+        user = self.runner.remote_user
+
+        ssh = paramiko.SSHClient()
+        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+
+        try:
+            ssh.connect(
+                self.host,
+                username=user,
+                allow_agent=True,
+                look_for_keys=True,
+                key_filename=self.runner.private_key_file,
+                password=self.runner.remote_pass,
+                timeout=self.runner.timeout,
+                port=self.port
+            )
+        except Exception, e:
+            if str(e).find("PID check failed") != -1:
+                raise errors.AnsibleError("paramiko version issue, please upgrade paramiko on the machine running ansible")
+            else:
+                raise errors.AnsibleConnectionFailed(str(e))
+
+        return ssh
+
+    def connect(self):
+        ''' connect to the remote host '''
+
+        self.ssh = self._get_conn()
+        return self
+
+    def exec_command(self, cmd, tmp_path,sudo_user,sudoable=False):
+
+        ''' run a command on the remote host '''
+        bufsize = 4096
+        chan = self.ssh.get_transport().open_session()
+        chan.get_pty() 
+
+        if not self.runner.sudo or not sudoable:
+            quoted_command = '"$SHELL" -c ' + pipes.quote(cmd) 
+            chan.exec_command(quoted_command)
+        else:
+            # Rather than detect if sudo wants a password this time, -k makes 
+            # sudo always ask for a password if one is required. The "--"
+            # tells sudo that this is the end of sudo options and the command
+            # follows.  Passing a quoted compound command to sudo (or sudo -s)
+            # directly doesn't work, so we shellquote it with pipes.quote() 
+            # and pass the quoted string to the user's shell.  We loop reading
+            # output until we see the randomly-generated sudo prompt set with
+            # the -p option.
+            randbits = ''.join(chr(random.randint(ord('a'), ord('z'))) for x in xrange(32))
+            prompt = '[sudo via ansible, key=%s] password: ' % randbits
+            sudocmd = 'sudo -k -p "%s" -u %s -- "$SHELL" -c %s' % (prompt,
+                    sudo_user, pipes.quote(cmd))
+            sudo_output = ''
+            try:
+                chan.exec_command(sudocmd)
+                if self.runner.sudo_pass:
+                    while not sudo_output.endswith(prompt):
+                        chunk = chan.recv(bufsize)
+                        if not chunk:
+                            raise errors.AnsibleError('ssh connection closed waiting for sudo password prompt')
+                        sudo_output += chunk
+                    chan.sendall(self.runner.sudo_pass + '\n')
+            except socket.timeout:
+                raise errors.AnsibleError('ssh timed out waiting for sudo.\n' + sudo_output)
+
+        stdin = chan.makefile('wb', bufsize)
+        stdout = chan.makefile('rb', bufsize)
+        stderr = ''  # stderr goes to stdout when using a pty, so this will never output anything.
+        return stdin, stdout, stderr
+
+    def put_file(self, in_path, out_path):
+        ''' transfer a file from local to remote '''
+        if not os.path.exists(in_path):
+            raise errors.AnsibleFileNotFound("file or module does not exist: %s" % in_path)
+        sftp = self.ssh.open_sftp()
+        try:
+            sftp.put(in_path, out_path)
+        except IOError:
+            traceback.print_exc()
+            raise errors.AnsibleError("failed to transfer file to %s" % out_path)
+        sftp.close()
+
+    def fetch_file(self, in_path, out_path):
+        sftp = self.ssh.open_sftp()
+        try:
+            sftp.get(in_path, out_path)
+        except IOError:
+            traceback.print_exc()
+            raise errors.AnsibleError("failed to transfer file from %s" % in_path)
+        sftp.close()
+
+    def close(self):
+        ''' terminate the connection '''
+
+        self.ssh.close()
+
+############################################
+# add other connection types here
+
+class LocalConnection(object):
+    ''' Local based connections '''
+
+    def __init__(self, runner, host):
+        self.runner = runner
+        self.host = host
+
+    def connect(self, port=None):
+        ''' connect to the local host; nothing to do here '''
+
+        return self
+
+    def exec_command(self, cmd, tmp_path,sudo_user,sudoable=False):
+        ''' run a command on the local host '''
+        if self.runner.sudo and sudoable:
+            cmd = "sudo -s %s" % cmd
+        if self.runner.sudo_pass:
+            # NOTE: if someone wants to add sudo w/ password to the local connection type, they are welcome
+            # to do so.  The primary usage of the local connection is for crontab and kickstart usage however
+            # so this doesn't seem to be a huge priority
+            raise errors.AnsibleError("sudo with password is presently only supported on the paramiko (SSH) connection type")
+
+        p = subprocess.Popen(cmd, shell=True, stdin=None,
+                             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        stdout, stderr = p.communicate()
+        return ("", stdout, stderr)
+
+    def put_file(self, in_path, out_path):
+        ''' transfer a file from local to local '''
+        if not os.path.exists(in_path):
+            raise errors.AnsibleFileNotFound("file or module does not exist: %s" % in_path)
+        try:
+            shutil.copyfile(in_path, out_path)
+        except shutil.Error:
+            traceback.print_exc()
+            raise errors.AnsibleError("failed to copy: %s and %s are the same" % (in_path, out_path))
+        except IOError:
+            traceback.print_exc()
+            raise errors.AnsibleError("failed to transfer file to %s" % out_path)
+
+    def fetch_file(self, in_path, out_path):
+        ''' fetch a file from local to local -- for copatibility '''
+        self.put_file(in_path, out_path)
+
+    def close(self):
+        ''' terminate the connection; nothing to do here '''
+
+        pass

lib/ansible/constants.py

@@ -19,15 +19,25 @@
 import os
 
 
-# control side (aka 'overlord')
 DEFAULT_HOST_LIST      = os.environ.get('ANSIBLE_HOSTS',
-                                        '/etc/ansible/hosts')
+    '/etc/ansible/hosts')
 DEFAULT_MODULE_PATH    = os.environ.get('ANSIBLE_LIBRARY',
-                                        '/usr/share/ansible')
+    '/usr/share/ansible')
+DEFAULT_REMOTE_TMP     = os.environ.get('ANSIBLE_REMOTE_TMP', 
+    '/$HOME/.ansible/tmp')
+
 DEFAULT_MODULE_NAME    = 'command'
 DEFAULT_PATTERN        = '*'
 DEFAULT_FORKS          = 5
 DEFAULT_MODULE_ARGS    = ''
 DEFAULT_TIMEOUT        = 10
+DEFAULT_POLL_INTERVAL  = 15
 DEFAULT_REMOTE_USER    = 'root'
 DEFAULT_REMOTE_PASS    = None
+DEFAULT_PRIVATE_KEY_FILE    = None
+DEFAULT_SUDO_PASS      = None
+DEFAULT_SUDO_USER      = 'root'
+DEFAULT_REMOTE_PORT    = 22
+DEFAULT_TRANSPORT      = 'paramiko'
+DEFAULT_TRANSPORT_OPTS = ['local', 'paramiko']
+

lib/ansible/errors.py

@@ -0,0 +1,37 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+
+class AnsibleError(Exception):
+    """
+    The base Ansible exception from which all others should subclass.
+    """
+
+    def __init__(self, msg):
+        self.msg = msg
+    
+    def __str__(self):
+        return self.msg
+
+
+class AnsibleFileNotFound(AnsibleError):
+    pass
+
+class AnsibleConnectionFailed(AnsibleError):
+    pass
+
+

lib/ansible/group.py

@@ -0,0 +1,76 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#############################################
+
+# from ansible import errors
+
+class Group(object):
+    """ 
+    Group of ansible hosts
+    """
+
+    def __init__(self, name=None):
+        self.name = name
+        self.hosts = []
+        self.vars = {}
+        self.child_groups = []
+        self.parent_groups = []
+        if self.name is None:
+           raise Exception("group name is required")
+
+    def add_child_group(self, group):
+        if self == group:
+            raise Exception("can't add group to itself")
+        self.child_groups.append(group)
+        group.parent_groups.append(self)
+
+    def add_host(self, host):
+        self.hosts.append(host)
+        host.add_group(self)
+
+    def set_variable(self, key, value):
+        self.vars[key] = value
+
+    def get_hosts(self):
+        hosts = []
+        for kid in self.child_groups:
+            hosts.extend(kid.get_hosts())
+        hosts.extend(self.hosts)
+        return hosts 
+
+    def get_variables(self):
+        vars = {}
+        # FIXME: verify this variable override order is what we want
+        for ancestor in self.get_ancestors():
+           vars.update(ancestor.get_variables())
+        vars.update(self.vars)
+        return vars
+
+    def _get_ancestors(self):
+        results = {}
+        for g in self.parent_groups:
+            results[g.name] = g
+            results.update(g._get_ancestors())
+        return results
+
+    def get_ancestors(self):
+        return self._get_ancestors().values()
+
+
+
+

lib/ansible/host.py

@@ -0,0 +1,62 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#############################################
+
+from ansible import errors
+import ansible.constants as C
+
+class Host(object):
+    """ 
+    Group of ansible hosts
+    """
+
+    def __init__(self, name=None, port=None):
+        self.name = name
+        self.vars = {}
+        self.groups = []
+        if port and port != C.DEFAULT_REMOTE_PORT:
+            self.set_variable('ansible_ssh_port', int(port))
+
+        if self.name is None:
+            raise Exception("host name is required")
+
+    def add_group(self, group):
+        self.groups.append(group)
+
+    def set_variable(self, key, value):
+        self.vars[key]=value;
+
+    def get_groups(self):
+        groups = {}
+        for g in self.groups:
+            groups[g.name] = g
+            ancestors = g.get_ancestors()
+            for a in ancestors:
+                groups[a.name] = a                
+        return groups.values()
+
+    def get_variables(self):
+        results = {}
+        for group in self.groups:
+            results.update(group.get_variables())
+        results.update(self.vars)
+        results['inventory_hostname'] = self.name
+        groups = self.get_groups()
+        results['group_names'] = sorted([ g.name for g in groups if g.name != 'all'])
+        return results
+

lib/ansible/inventory.py

@@ -0,0 +1,164 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#############################################
+
+import fnmatch
+import os
+
+import constants as C
+import subprocess
+from ansible.inventory_parser import InventoryParser
+from ansible.inventory_parser_yaml import InventoryParserYaml
+from ansible.inventory_script import InventoryScript
+from ansible.group import Group
+from ansible.host import Host
+from ansible import errors
+from ansible import utils
+
+class Inventory(object):
+    """ 
+    Host inventory for ansible.
+    """
+
+    def __init__(self, host_list=C.DEFAULT_HOST_LIST):
+
+        # the host file file, or script path, or list of hosts
+        # if a list, inventory data will NOT be loaded
+        self.host_list = host_list
+
+        # the inventory object holds a list of groups
+        self.groups = []
+ 
+        # a list of host(names) to contain current inquiries to
+        self._restriction = None
+
+        # whether the inventory file is a script
+        self._is_script = False
+
+        if type(host_list) in [ str, unicode ]:
+            if host_list.find(",") != -1:
+               host_list = host_list.split(",")
+
+        if type(host_list) == list:
+            all = Group('all')
+            self.groups = [ all ]
+            for x in host_list:
+                if x.find(":") != -1:
+                    tokens = x.split(":",1)
+                    all.add_host(Host(tokens[0], tokens[1]))
+                else:
+                    all.add_host(Host(x))
+        elif os.access(host_list, os.X_OK):
+            self._is_script = True
+            self.parser = InventoryScript(filename=host_list)
+            self.groups = self.parser.groups.values()
+        else:
+            data = file(host_list).read()
+            if not data.startswith("---"):
+                self.parser = InventoryParser(filename=host_list)
+                self.groups = self.parser.groups.values()
+            else:         
+                self.parser = InventoryParserYaml(filename=host_list)
+                self.groups = self.parser.groups.values()
+      
+    def _match(self, str, pattern_str):
+        return fnmatch.fnmatch(str, pattern_str)
+
+    def get_hosts(self, pattern="all"):
+        """ Get all host objects matching the pattern """
+        hosts = {}
+        patterns = pattern.replace(";",":").split(":")
+
+        groups = self.get_groups()
+        for group in groups:
+             for host in group.get_hosts():
+                 for pat in patterns:
+                     if group.name == pat or pat == 'all' or self._match(host.name, pat):
+                         if not self._restriction:
+                             hosts[host.name] = host
+                         if self._restriction and host.name in self._restriction:
+                             hosts[host.name] = host
+        return sorted(hosts.values(), key=lambda x: x.name)
+
+    def get_groups(self):
+        return self.groups
+
+    def get_host(self, hostname):
+        for group in self.groups:
+           for host in group.get_hosts():
+               if hostname == host.name:
+                    return host
+        return None
+
+    def get_group(self, groupname):
+        for group in self.groups:
+            if group.name == groupname:
+                return group
+        return None
+
+    def get_group_variables(self, groupname):
+        group = self.get_group(groupname)
+        if group is None:
+            raise Exception("group not found: %s" % groupname)
+        return group.get_variables()
+
+    def get_variables(self, hostname):
+
+        if self._is_script:
+            # TODO: move this to inventory_script.py 
+            host = self.get_host(hostname)
+            cmd = subprocess.Popen(
+                [self.host_list,"--host",hostname], 
+                stdout=subprocess.PIPE, 
+                stderr=subprocess.PIPE
+            )
+            (out, err) = cmd.communicate()
+            results = utils.parse_json(out)
+            results['inventory_hostname'] = hostname
+            groups = [ g.name for g in host.get_groups() if g.name != 'all' ]
+            results['group_names'] = sorted(groups)
+            return results
+
+        host = self.get_host(hostname)
+        if host is None:
+            raise Exception("host not found: %s" % hostname)
+        return host.get_variables()
+
+    def add_group(self, group):
+        self.groups.append(group)
+
+    def list_hosts(self, pattern="all"):
+        return [ h.name for h in self.get_hosts(pattern) ]
+
+    def list_groups(self):
+        return [ g.name for g in self.groups ] 
+
+    def get_restriction(self):
+        return self._restriction
+
+    def restrict_to(self, restriction, append_missing=False):
+        """ Restrict list operations to the hosts given in restriction """
+   
+	if type(restriction) != list:
+	    restriction = [ restriction ]
+	self._restriction = restriction
+
+    def lift_restriction(self):
+        """ Do not restrict list operations """
+
+        self._restriction = None

lib/ansible/inventory_parser.py

@@ -0,0 +1,151 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#############################################
+
+import fnmatch
+import os
+import subprocess
+
+import constants as C
+from ansible.host import Host
+from ansible.group import Group
+from ansible import errors
+from ansible import utils
+
+class InventoryParser(object):
+    """ 
+    Host inventory for ansible.
+    """
+
+    def __init__(self, filename=C.DEFAULT_HOST_LIST):
+
+        fh = open(filename)
+        self.lines = fh.readlines()
+        self.groups = {}
+        self.hosts = {}
+        self._parse()
+ 
+    def _parse(self):
+
+        self._parse_base_groups()
+        self._parse_group_children()
+        self._parse_group_variables()
+        return self.groups
+
+    # [webservers]
+    # alpha
+    # beta:2345
+    # gamma sudo=True user=root
+    # delta asdf=jkl favcolor=red 
+
+    def _parse_base_groups(self):
+
+        ungrouped = Group(name='ungrouped')
+        all = Group(name='all')
+        all.add_child_group(ungrouped)
+
+        self.groups = dict(all=all, ungrouped=ungrouped)
+        active_group_name = 'ungrouped'
+
+        for line in self.lines:
+            if line.startswith("["):
+                active_group_name = line.replace("[","").replace("]","").strip()
+                if line.find(":vars") != -1 or line.find(":children") != -1:
+                    active_group_name = None
+                else:
+                    new_group = self.groups[active_group_name] = Group(name=active_group_name)
+                    all.add_child_group(new_group)
+            elif line.startswith("#") or line == '':
+                pass
+            elif active_group_name:
+                tokens = line.split()
+                if len(tokens) == 0:
+                    continue
+                hostname = tokens[0]
+                port = C.DEFAULT_REMOTE_PORT
+                if hostname.find(":") != -1:
+                   tokens2  = hostname.split(":")
+                   hostname = tokens2[0]
+                   port     = tokens2[1]
+                host = None
+                if hostname in self.hosts:
+                    host = self.hosts[hostname]
+                else:
+                    host = Host(name=hostname, port=port)
+                    self.hosts[hostname] = host
+                if len(tokens) > 1:
+                   for t in tokens[1:]:
+                      (k,v) = t.split("=")
+                      host.set_variable(k,v)
+                self.groups[active_group_name].add_host(host)
+
+    # [southeast:children]
+    # atlanta
+    # raleigh
+
+    def _parse_group_children(self):
+        group = None
+
+        for line in self.lines:
+            line = line.strip()
+            if line is None or line == '':
+                continue
+            if line.startswith("[") and line.find(":children]") != -1:
+                line = line.replace("[","").replace(":children]","")
+                group = self.groups.get(line, None)
+                if group is None:
+                    group = self.groups[line] = Group(name=line)
+            elif line.startswith("#"):
+                pass
+            elif line.startswith("["):
+                group = None
+            elif group:
+                kid_group = self.groups.get(line, None)
+                if kid_group is None:
+                    raise errors.AnsibleError("child group is not defined: (%s)" % line)
+                else:
+                    group.add_child_group(kid_group)
+
+
+    # [webservers:vars]
+    # http_port=1234
+    # maxRequestsPerChild=200
+
+    def _parse_group_variables(self):
+        group = None
+        for line in self.lines:
+            line = line.strip()
+            if line.startswith("[") and line.find(":vars]") != -1:
+                line = line.replace("[","").replace(":vars]","")
+                group = self.groups.get(line, None)
+                if group is None:
+                   raise errors.AnsibleError("can't add vars to undefined group: %s" % line)
+            elif line.startswith("#"):
+                pass
+            elif line.startswith("["):
+                group = None
+            elif line == '':
+                pass
+            elif group:
+                if line.find("=") == -1:
+                    raise errors.AnsibleError("variables assigned to group must be in key=value form")
+                else:
+                    (k,v) = line.split("=")
+                    group.set_variable(k,v)
+
+

lib/ansible/inventory_parser_yaml.py

@@ -0,0 +1,108 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#############################################
+
+import constants as C
+from ansible.host import Host
+from ansible.group import Group
+from ansible import errors
+from ansible import utils
+
+class InventoryParserYaml(object):
+    """ 
+    Host inventory for ansible.
+    """
+
+    def __init__(self, filename=C.DEFAULT_HOST_LIST):
+
+        fh = open(filename)
+        data = fh.read()
+        fh.close()
+        self._hosts = {}
+        self._parse(data)
+
+    def _make_host(self, hostname):
+        if hostname in self._hosts:
+            return self._hosts[hostname]
+        else:
+            host = Host(hostname)
+            self._hosts[hostname] = host
+            return host 
+
+    # see file 'test/yaml_hosts' for syntax
+
+    def _parse(self, data):
+
+        all = Group('all')
+        ungrouped = Group('ungrouped')
+        all.add_child_group(ungrouped)
+
+        self.groups = dict(all=all, ungrouped=ungrouped)
+
+        yaml = utils.parse_yaml(data)
+        for item in yaml:
+
+            if type(item) in [ str, unicode ]:
+                host = self._make_host(item)
+                ungrouped.add_host(host)
+
+            elif type(item) == dict and 'host' in item:
+                host = self._make_host(item['host'])
+                vars = item.get('vars', {})
+                if type(vars)==list:
+                    varlist, vars = vars, {}
+                    for subitem in varlist:
+                        vars.update(subitem)
+                for (k,v) in vars.items():
+                   host.set_variable(k,v)
+
+            elif type(item) == dict and 'group' in item:
+                group = Group(item['group'])
+
+                for subresult in item.get('hosts',[]):
+
+                    if type(subresult) in [ str, unicode ]:
+                        host = self._make_host(subresult)
+                        group.add_host(host)
+                    elif type(subresult) == dict:
+                        host = self._make_host(subresult['host'])
+                        vars = subresult.get('vars',{})
+                        if type(vars) == list:
+                            for subitem in vars:
+                               for (k,v) in subitem.items():
+                                   host.set_variable(k,v) 
+                        elif type(vars) == dict:
+                            for (k,v) in subresult.get('vars',{}).items():
+                                host.set_variable(k,v)
+                        else:
+                            raise errors.AnsibleError("unexpected type for variable")
+                        group.add_host(host)
+
+                vars = item.get('vars',{})
+                if type(vars) == dict:
+                    for (k,v) in item.get('vars',{}).items():
+                        group.set_variable(k,v) 
+                elif type(vars) == list:
+                    for subitem in vars:
+                        if type(subitem) != dict:
+                            raise errors.AnsibleError("expected a dictionary")
+                        for (k,v) in subitem.items():
+                            group.set_variable(k,v)
+
+                self.groups[group.name] = group
+                all.add_child_group(group)

lib/ansible/inventory_script.py

@@ -0,0 +1,59 @@
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+#############################################
+
+import os
+import subprocess
+import constants as C
+import os
+from ansible.host import Host
+from ansible.group import Group
+from ansible import errors
+from ansible import utils
+
+class InventoryScript(object):
+    """ 
+    Host inventory parser for ansible using external inventory scripts.
+    """
+
+    def __init__(self, filename=C.DEFAULT_HOST_LIST):
+
+        cmd = [ filename, "--list" ]
+        sp = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        (stdout, stderr) = sp.communicate()
+        self.data = stdout
+        self.groups = self._parse()
+
+    def _parse(self):
+        groups = {}
+        self.raw = utils.parse_json(self.data)
+        all=Group('all')
+        self.groups = dict(all=all)
+        group = None
+        for (group_name, hosts) in self.raw.items():
+            group = groups[group_name] = Group(group_name)
+            host = None
+            for hostname in hosts:
+                host = Host(hostname)
+                group.add_host(host)
+                # FIXME: hack shouldn't be needed
+                all.add_host(host)
+            all.add_child_group(group)
+        return groups  
+        
+  

lib/ansible/playbook.py

@@ -17,13 +17,12 @@
 
 #############################################
 
+import ansible.inventory
 import ansible.runner
 import ansible.constants as C
-from ansible.utils import *
-import yaml
-import shlex
+from ansible import utils
+from ansible import errors
 import os
-import jinja2
 
 # used to transfer variables to Runner
 SETUP_CACHE={ }
@@ -38,84 +37,156 @@ class PlayBook(object):
     management, or automation based set of commands to
     run in series.
 
-    multiple patterns do not execute simultaneously,
+    multiple plays/tasks do not execute simultaneously,
     but tasks in each pattern do execute in parallel
-    according to the number of forks requested.
+    (according to the number of forks requested) among
+    the hosts they address
     '''
 
+    # *****************************************************
+
     def __init__(self,
-        playbook     =None,
-        host_list    =C.DEFAULT_HOST_LIST,
-        module_path  =C.DEFAULT_MODULE_PATH,
-        forks        =C.DEFAULT_FORKS,
-        timeout      =C.DEFAULT_TIMEOUT,
-        remote_user  =C.DEFAULT_REMOTE_USER,
-        remote_pass  =C.DEFAULT_REMOTE_PASS,
-        verbose=False,
-        callbacks=None):
+        playbook         = None,
+        host_list        = C.DEFAULT_HOST_LIST,
+        module_path      = C.DEFAULT_MODULE_PATH,
+        forks            = C.DEFAULT_FORKS,
+        timeout          = C.DEFAULT_TIMEOUT,
+        remote_user      = C.DEFAULT_REMOTE_USER,
+        remote_pass      = C.DEFAULT_REMOTE_PASS,
+        sudo_pass        = C.DEFAULT_SUDO_PASS,
+        remote_port      = C.DEFAULT_REMOTE_PORT,
+        transport        = C.DEFAULT_TRANSPORT,
+        private_key_file = C.DEFAULT_PRIVATE_KEY_FILE,
+        debug            = False,
+        callbacks        = None,
+        runner_callbacks = None,
+        stats            = None,
+        sudo             = False,
+        sudo_user        = C.DEFAULT_SUDO_USER,
+        extra_vars       = None):
 
-        # TODO, once ansible-playbook is it's own script this will
-        # have much LESS parameters to the constructor and will
-        # read most everything per pattern from the playbook
-        # and this will be greatly simplified
+        """
+        playbook:         path to a playbook file
+        host_list:        path to a file like /etc/ansible/hosts
+        module_path:      path to ansible modules, like /usr/share/ansible/
+        forks:            desired level of paralellism
+        timeout:          connection timeout
+        remote_user:      run as this user if not specified in a particular play
+        remote_pass:      use this remote password (for all plays) vs using SSH keys
+        sudo_pass:        if sudo==True, and a password is required, this is the sudo password
+        remote_port:      default remote port to use if not specified with the host or play
+        transport:        how to connect to hosts that don't specify a transport (local, paramiko, etc)
+        callbacks         output callbacks for the playbook
+        runner_callbacks: more callbacks, this time for the runner API
+        stats:            holds aggregrate data about events occuring to each host
+        sudo:             if not specified per play, requests all plays use sudo mode
+        """
 
-        self.host_list   = host_list
-        self.module_path = module_path
-        self.forks       = forks
-        self.timeout     = timeout
-        self.remote_user = remote_user
-        self.remote_pass = remote_pass
-        self.verbose     = verbose
-        self.callbacks   = callbacks
-        self.callbacks.set_playbook(self)
+        if playbook is None or callbacks is None or runner_callbacks is None or stats is None:
+            raise Exception('missing required arguments')
 
-        # store the list of changes/invocations/failure counts
-        # as a dictionary of integers keyed off the hostname
+        if extra_vars is None:
+            extra_vars = {}
+       
+        self.module_path      = module_path
+        self.forks            = forks
+        self.timeout          = timeout
+        self.remote_user      = remote_user
+        self.remote_pass      = remote_pass
+        self.remote_port      = remote_port
+        self.transport        = transport
+        self.debug            = debug
+        self.callbacks        = callbacks
+        self.runner_callbacks = runner_callbacks
+        self.stats            = stats
+        self.sudo             = sudo
+        self.sudo_pass        = sudo_pass
+        self.sudo_user        = sudo_user
+        self.extra_vars       = extra_vars
+        self.global_vars      = {}
+        self.private_key_file = private_key_file
 
-        self.processed    = {}
-        self.dark         = {}
-        self.changed      = {}
-        self.invocations  = {}
-        self.failures     = {}
+        self.inventory = ansible.inventory.Inventory(host_list)
+        
 
-        # playbook file can be passed in as a path or
-        # as file contents (to support API usage)
+        if not self.inventory._is_script:
+            self.global_vars.update(self.inventory.get_group_variables('all'))
 
-        self.basedir = os.path.dirname(playbook)
-        self.playbook = self._parse_playbook(playbook)
+        self.basedir          = os.path.dirname(playbook)
+        self.playbook         = self._parse_playbook(playbook)
+
+    # *****************************************************
+
+    def _get_vars(self, play, dirname):
+        ''' load the vars section from a play '''
+
+        if play.get('vars') is None:
+            play['vars'] = {}
+        if type(play['vars']) not in [dict, list]:
+            raise errors.AnsibleError("'vars' section must contain only key/value pairs")
+        vars = self.global_vars
+        
+        # translate a list of vars into a dict
+        if type(play['vars']) == list:
+            for item in play['vars']:
+                k, v = item.items()[0]
+                vars[k] = v
+        else:
+            vars.update(play['vars'])
+
+        vars_prompt = play.get('vars_prompt', {})
+        if type(vars_prompt) != dict:
+            raise errors.AnsibleError("'vars_prompt' section must contain only key/value pairs")
+        for vname in vars_prompt:
+            # TODO: make this prompt one line and consider double entry
+            print vars_prompt[vname]
+            vars[vname] = self.callbacks.on_vars_prompt(vname)
+
+        results = self.extra_vars.copy()
+        results.update(vars)
+        return results
+
+    # *****************************************************
 
     def _include_tasks(self, play, task, dirname, new_tasks):
-        # an include line looks like:
+        ''' load tasks included from external files. '''
+
         # include: some.yml a=2 b=3 c=4
-        include_tokens = task['include'].split()
-        path = path_dwim(dirname, include_tokens[0])
-        inject_vars = play.get('vars', {})
+        play_vars = self._get_vars(play, dirname)
+        include_tokens = utils.template(task['include'], play_vars, SETUP_CACHE).split()
+        path = utils.path_dwim(dirname, include_tokens[0])
+        include_vars = {}
         for i,x in enumerate(include_tokens):
             if x.find("=") != -1:
                 (k,v) = x.split("=")
-                inject_vars[k] = v
-            included = file(path).read()
-            template = jinja2.Template(included)
-            included = template.render(inject_vars)
-            included = yaml.load(included)
-            for x in included:
-                new_tasks.append(x)
+                include_vars[k] = v
+        inject_vars = play_vars.copy()
+        inject_vars.update(include_vars)
+        included = utils.parse_yaml_from_file(path)
+        for x in included:
+            if len(include_vars):
+                x["vars"] = include_vars
+            new_tasks.append(x)
+
+    # *****************************************************
 
     def _include_handlers(self, play, handler, dirname, new_handlers):
-        path = path_dwim(dirname, handler['include'])
-        included = file(path).read()
-        inject_vars = play.get('vars', {})
-        template = jinja2.Template(included)
-        included = template.render(inject_vars)
-        included = yaml.load(included)
+        ''' load handlers from external files '''
+        inject_vars = self._get_vars(play, dirname)
+        path = utils.template(handler['include'], inject_vars, SETUP_CACHE) 
+        path = utils.path_dwim(dirname, path)
+        included = utils.template_from_file(path, inject_vars, SETUP_CACHE)
+        included = utils.parse_yaml(included)
         for x in included:
             new_handlers.append(x)
 
+    # *****************************************************
+
     def _parse_playbook(self, playbook):
         ''' load YAML file, including handling for imported files '''
         
         dirname  = os.path.dirname(playbook)
-        playbook = yaml.load(file(playbook).read())
+        playbook = utils.parse_yaml_from_file(playbook)
 
         for play in playbook:
             tasks = play.get('tasks',[])
@@ -124,22 +195,47 @@ class PlayBook(object):
             # process tasks in this file as well as imported tasks
             new_tasks = []
             for task in tasks:
-               if 'include' in task:
-                   self._include_tasks(play, task, dirname, new_tasks)
-               else:
-                   new_tasks.append(task)
-            play['tasks'] = new_tasks
+                if 'include' in task:
+                    self._include_tasks(play, task, dirname, new_tasks)
+                else:
+                    new_tasks.append(task)
+
+            # now new_tasks contains a list of tasks, but tasks may contain
+            # lists of with_items to loop over.  Do that.
+            # TODO: refactor into subfunction
+            new_tasks2 = []
+            for task in new_tasks:
+                if 'with_items' in task:
+                    for item in task['with_items']:
+                        produced_task = {}
+                        name    = task.get('name', task.get('action', 'unnamed task'))
+                        action  = task.get('action', None)
+                        only_if = task.get('only_if', None)
+                        if action is None:
+                            raise errors.AnsibleError('action is required')
+                        produced_task = task.copy()
+                        produced_task['action'] = utils.template(action, dict(item=item), SETUP_CACHE)
+                        produced_task['name'] = utils.template(name, dict(item=item), SETUP_CACHE)
+                        if only_if:
+                            produced_task['only_if'] = utils.template(only_if, dict(item=item), SETUP_CACHE)
+                        new_tasks2.append(produced_task)
+                else:
+                    new_tasks2.append(task)
+
+            play['tasks'] = new_tasks2
 
             # process handlers as well as imported handlers
             new_handlers = [] 
             for handler in handlers:
-               if 'include' in handler:
-                   self._include_handlers(play, handler, dirname, new_handlers)
-               else:
-                   new_handlers.append(handler)
+                if 'include' in handler:
+                    self._include_handlers(play, handler, dirname, new_handlers)
+                else:
+                    new_handlers.append(handler)
             play['handlers'] = new_handlers
 
         return playbook
+
+    # *****************************************************
         
     def run(self):
         ''' run all patterns in the playbook '''
@@ -147,68 +243,97 @@ class PlayBook(object):
         # loop through all patterns and run them
         self.callbacks.on_start()
         for pattern in self.playbook:
-            self._run_pattern(pattern)
+            self._run_play(pattern)
 
         # summarize the results
         results = {}
-        for host in self.processed.keys():
-            results[host]  = {
-                'resources'   : self.invocations.get(host, 0),
-                'changed'     : self.changed.get(host, 0),
-                'dark'        : self.dark.get(host, 0),
-                'failed'      : self.failures.get(host, 0)
-            }
+        for host in self.stats.processed.keys():
+            results[host] = self.stats.summarize(host)
         return results
 
-    def _prune_failed_hosts(self, host_list):
-        new_hosts = []
-        for x in host_list:
-            if not x in self.failures and not x in self.dark:
-                new_hosts.append(x)
-        return new_hosts
+    # *****************************************************
 
-    def _run_module(self, pattern, module, args, hosts, remote_user):
+    def _async_poll(self, poller, async_seconds, async_poll_interval):
+        ''' launch an async job, if poll_interval is set, wait for completion '''
+
+        results = poller.wait(async_seconds, async_poll_interval)
+
+        # mark any hosts that are still listed as started as failed
+        # since these likely got killed by async_wrapper
+        for host in poller.hosts_to_poll:
+            reason = { 'failed' : 1, 'rc' : None, 'msg' : 'timed out' }
+            self.runner_callbacks.on_failed(host, reason)
+            results['contacted'][host] = reason
+
+        return results
+
+    # *****************************************************
+
+    def _run_module(self, pattern, module, args, vars, remote_user, 
+        async_seconds, async_poll_interval, only_if, sudo, sudo_user, transport, port):
         ''' run a particular module step in a playbook '''
-        return ansible.runner.Runner(
-            pattern=pattern,
-            module_name=module,
-            module_args=args,
-            host_list=hosts,
-            forks=self.forks,
-            remote_pass=self.remote_pass,
-            module_path=self.module_path,
-            timeout=self.timeout,
-            remote_user=remote_user,
-            setup_cache=SETUP_CACHE,
-            basedir=self.basedir
-        ).run()
 
-    def _run_task(self, pattern=None, task=None, host_list=None,
-        remote_user=None, handlers=None, conditional=False):
-        '''
-        run a single task in the playbook and
-        recursively run any subtasks.
-        '''
+        hosts = [ h for h in self.inventory.list_hosts() if (h not in self.stats.failures) and (h not in self.stats.dark)]
+        self.inventory.restrict_to(hosts)
 
-        if host_list is None:
-            # pruned host lists occur when running triggered
-            # actions where not all hosts have changed
-            # though top-level tasks will pass in "None" here
-            host_list = self.host_list
-            (host_list, groups) = ansible.runner.Runner.parse_hosts(host_list)
+        if port is None:
+            port=self.remote_port
 
-        # do not continue to run tasks on hosts that have had failures
-        host_list = self._prune_failed_hosts(host_list)
+        runner = ansible.runner.Runner(
+            pattern=pattern, inventory=self.inventory, module_name=module,
+            module_args=args, forks=self.forks,
+            remote_pass=self.remote_pass, module_path=self.module_path,
+            timeout=self.timeout, remote_user=remote_user, 
+            remote_port=port, module_vars=vars,
+            private_key_file=self.private_key_file,
+            setup_cache=SETUP_CACHE, basedir=self.basedir,
+            conditional=only_if, callbacks=self.runner_callbacks, 
+            debug=self.debug, sudo=sudo, sudo_user=sudo_user,
+            transport=transport, sudo_pass=self.sudo_pass, is_playbook=True
+        )
+
+        if async_seconds == 0:
+            results = runner.run()
+        else:
+            results, poller = runner.runAsync(async_seconds)
+            self.stats.compute(results)
+            if async_poll_interval > 0:
+                # if not polling, playbook requested fire and forget
+                # trust the user wanted that and return immediately
+                results = self._async_poll(poller, async_seconds, async_poll_interval)
+
+        self.inventory.lift_restriction()
+        return results
+
+    # *****************************************************
+
+    def _run_task(self, pattern=None, task=None, 
+        remote_user=None, handlers=None, conditional=False, 
+        sudo=False, sudo_user=None, transport=None, port=None):
+        ''' run a single task in the playbook and recursively run any subtasks.  '''
 
         # load the module name and parameters from the task entry
-        name    = task['name']
-        action  = task['action']
-        # comment = task.get('comment', '')
+        name    = task.get('name', None) 
+        action  = task.get('action', None)
+        if action is None:
+            raise errors.AnsibleError("action is required for each item in tasks: offending task is %s" % name if name else "unknown")
+        if name is None:
+            name = action
+        only_if = task.get('only_if', 'True')
+        async_seconds = int(task.get('async', 0))  # not async by default
+        async_poll_interval = int(task.get('poll', 10))  # default poll = 10 seconds
 
-        tokens = shlex.split(action)
+        tokens = action.split(None, 1)
         module_name = tokens[0]
-        module_args = tokens[1:]
+        module_args = ''
+        if len(tokens) > 1:
+            module_args = tokens[1]
 
+        # include task specific vars
+        module_vars = task.get('vars', {})
+        if 'first_available_file' in task:
+            module_vars['first_available_file'] = task.get('first_available_file')
+        
         # tasks can be direct (run on all nodes matching
         # the pattern) or conditional, where they ran
         # as the result of a change handler on a subset
@@ -219,61 +344,38 @@ class PlayBook(object):
         # load up an appropriate ansible runner to
         # run the task in parallel
         results = self._run_module(pattern, module_name, 
-            module_args, host_list, remote_user)
+            module_args, module_vars, remote_user, async_seconds, 
+            async_poll_interval, only_if, sudo, sudo_user, transport, port)
+
+        # add facts to the global setup cache
+        for host, result in results['contacted'].iteritems():
+            if "ansible_facts" in result:
+                for k,v in result['ansible_facts'].iteritems():
+                    SETUP_CACHE[host][k]=v
+
+        self.stats.compute(results)
 
         # if no hosts are matched, carry on, unlike /bin/ansible
         # which would warn you about this
         if results is None:
             results = {}
  
-        # walk through the results and build up
-        # summary information about successes and
-        # failures.  TODO: split into subfunction
-
-        dark      = results.get("dark", {})
-        contacted = results.get("contacted", {})
-
-        for host, msg in dark.items():
-            self.processed[host] = 1
-            self.callbacks.on_unreachable(host, msg)
-            if not host in self.dark:
-                self.dark[host] = 1
-            else:
-                self.dark[host] = self.dark[host] + 1
-
-        for host, results in contacted.items():
-            self.processed[host] = 1
-   
-            if is_failed(results):
-                self.callbacks.on_failed(host, results)
-                if not host in self.failures:
-                    self.failures[host] = 1
-                else:
-                    self.failures[host] = self.failures[host] + 1
-            else:
-                self.callbacks.on_ok(host)
-                if not host in self.invocations:
-                    self.invocations[host] = 1
-                else:
-                    self.invocations[host] = self.invocations[host] + 1
-                if results.get('changed', False):
-                    if not host in self.changed:
-                        self.changed[host] = 1
-                    else:
-                        self.changed[host] = self.changed[host] + 1
-
         # flag which notify handlers need to be run
         # this will be on a SUBSET of the actual host list.  For instance
         # a file might need to be written on only half of the nodes so
         # we would only trigger restarting Apache on half of the nodes
 
         subtasks = task.get('notify', [])
+        if isinstance(subtasks, basestring):
+            subtasks = [subtasks]
         if len(subtasks) > 0:
-            for host, results in contacted.items():
+            for host, results in results.get('contacted',{}).iteritems():
                 if results.get('changed', False):
                     for subtask in subtasks:
                         self._flag_handler(handlers, subtask, host)
 
+    # *****************************************************
+
     def _flag_handler(self, handlers, match_name, host):
         ''' 
         if a task has any notify elements, flag handlers for run
@@ -284,64 +386,164 @@ class PlayBook(object):
         # for all registered handlers in the ansible playbook
         # for this particular pattern group
 
+        found = False
         for x in handlers:
-            name = x['name']
+            name = x.get('name', None)
+            if name is None:
+                raise errors.AnsibleError('handler is missing a name')
             if match_name == name:
-                # flag the handler with the list of hosts
-                # it needs to be run on, it will be run later
+                found = True
+                self.callbacks.on_notify(host, name)
+                # flag the handler with the list of hosts it needs to be run on, it will be run later
                 if not 'run' in x:
                     x['run'] = []
                 x['run'].append(host)
+        if not found:
+            raise errors.AnsibleError("change handler (%s) is not defined" % match_name)
 
-    def _run_pattern(self, pg):
-        '''
-        run a list of tasks for a given pattern, in order
-        '''
+    # *****************************************************
 
-        # get configuration information about the pattern
-        pattern  = pg['hosts']
-        vars     = pg.get('vars', {})
-        tasks    = pg['tasks']
-        handlers = pg['handlers']
-        user     = pg.get('user', C.DEFAULT_REMOTE_USER)
+    def _do_conditional_imports(self, vars_files, pattern=None):
+        ''' handle the vars_files section, which can contain variables '''
 
-        self.host_list, groups = ansible.runner.Runner.parse_hosts(self.host_list)
+        # FIXME: save parsed variable results in memory to avoid excessive re-reading/parsing
+        # FIXME: currently parses imports for hosts not in the pattern, that is not wrong, but it's 
+        #        not super optimized yet either, because we wouldn't have hit them, ergo
+        #        it will raise false errors if there is no defaults variable file without any $vars
+        #        in it, which could happen on uncontacted hosts.
 
-        self.callbacks.on_play_start(pattern)
+        if type(vars_files) != list:
+            raise errors.AnsibleError("vars_files must be a list")
 
-        # first run the setup task on every node, which gets the variables
-        # written to the JSON file and will also bubble facts back up via
-        # magic in Runner()
-        push_var_str=''
-        for (k,v) in vars.items():
-            push_var_str += "%s=%s " % (k,v)
+        host_list = [ h for h in self.inventory.list_hosts(pattern)
+                        if not (h in self.stats.failures or h in self.stats.dark) ]
+
+        for host in host_list:
+            cache_vars = SETUP_CACHE.get(host,{})
+            SETUP_CACHE[host] = cache_vars
+            for filename in vars_files:
+                if type(filename) == list:
+                    # loop over all filenames, loading the first one, and failing if # none found
+                    found = False
+                    sequence = []
+                    for real_filename in filename:
+                        filename2 = utils.path_dwim(self.basedir, utils.template(real_filename, cache_vars, SETUP_CACHE))
+                        sequence.append(filename2)
+                        if os.path.exists(filename2):
+                            found = True
+                            data = utils.parse_yaml_from_file(filename2)
+                            SETUP_CACHE[host].update(data)
+                            self.callbacks.on_import_for_host(host, filename2)
+                            break
+                        else:
+                            self.callbacks.on_not_import_for_host(host, filename2)
+                    if not found:
+                        raise errors.AnsibleError(
+                            "%s: FATAL, no files matched for vars_files import sequence: %s" % (host, sequence)
+                        )
+
+                else:
+                    filename2 = utils.path_dwim(self.basedir, utils.template(filename, cache_vars, SETUP_CACHE))
+                    if not os.path.exists(filename2):
+                        raise errors.AnsibleError("no file matched for vars_file import: %s" % filename2)
+                    data = utils.parse_yaml_from_file(filename2)
+                    SETUP_CACHE[host].update(data)
+                    self.callbacks.on_import_for_host(host, filename2)
+
+    # *****************************************************
+
+    def _do_setup_step(self, pattern, vars, user, port, sudo, sudo_user, transport, vars_files=None):
+        ''' push variables down to the systems and get variables+facts back up '''
+
+        # this enables conditional includes like $facter_os.yml and is only done
+        # after the original pass when we have that data.
+        #
+
+        if vars_files is not None:
+            self.callbacks.on_setup_secondary()
+            self._do_conditional_imports(vars_files, pattern)
+        else:
+            self.callbacks.on_setup_primary()
+
+        host_list = [ h for h in self.inventory.list_hosts(pattern) 
+                        if not (h in self.stats.failures or h in self.stats.dark) ]
+
+        self.inventory.restrict_to(host_list)
 
         # push any variables down to the system
         setup_results = ansible.runner.Runner(
-            pattern=pattern,
-            module_name='setup',
-            module_args=push_var_str,
-            host_list=self.host_list,
-            forks=self.forks,
-            module_path=self.module_path,
-            timeout=self.timeout,
-            remote_user=user,
-            setup_cache=SETUP_CACHE
+            pattern=pattern, module_name='setup',
+            module_args=vars, inventory=self.inventory,
+            forks=self.forks, module_path=self.module_path,
+            timeout=self.timeout, remote_user=user,
+            remote_pass=self.remote_pass, remote_port=port,
+            private_key_file=self.private_key_file,
+            setup_cache=SETUP_CACHE,
+            callbacks=self.runner_callbacks, sudo=sudo, sudo_user=sudo_user, 
+            debug=self.debug, transport=transport, 
+            sudo_pass=self.sudo_pass, is_playbook=True
         ).run()
+        self.stats.compute(setup_results, setup=True)
+
+        self.inventory.lift_restriction()
 
         # now for each result, load into the setup cache so we can
         # let runner template out future commands
         setup_ok = setup_results.get('contacted', {})
-        for (host, result) in setup_ok.items():
-            SETUP_CACHE[host] = result
+        if vars_files is None:
+            # first pass only or we'll erase good work
+            for (host, result) in setup_ok.iteritems():
+                if 'ansible_facts' in result:
+                    SETUP_CACHE[host] = result['ansible_facts']
+
+    # *****************************************************
+
+    def _run_play(self, pg):
+        ''' run a list of tasks for a given pattern, in order '''
+
+        # get configuration information about the pattern
+        pattern = pg.get('hosts')
+        if pattern is None:
+            raise errors.AnsibleError('hosts declaration is required')
+        if isinstance(pattern, list):
+            pattern = ';'.join(pattern)
+        pattern = utils.template(pattern, self.extra_vars, {})
+        name = pg.get('name', pattern)
+
+        vars       = self._get_vars(pg, self.basedir)
+        vars_files = pg.get('vars_files', {})
+        tasks      = pg.get('tasks', [])
+        handlers   = pg.get('handlers', [])
+        user       = pg.get('user', self.remote_user)
+        port       = pg.get('port', self.remote_port)
+        sudo       = pg.get('sudo', self.sudo)
+        sudo_user  = pg.get('sudo_user', self.sudo_user)
+        transport  = pg.get('connection', self.transport)
+
+        # the default sudo user is root, so if you change it, sudo is implied 
+        if sudo_user != 'root':
+           sudo = True
+
+        self.callbacks.on_play_start(name)
+
+        # push any variables down to the system # and get facts/ohai/other data back up
+        self._do_setup_step(pattern, vars, user, port, sudo, sudo_user, transport, None)
+         
+        # now with that data, handle contentional variable file imports!
+        if len(vars_files) > 0:
+            self._do_setup_step(pattern, vars, user, port, sudo, sudo_user, transport, vars_files)
 
         # run all the top level tasks, these get run on every node
         for task in tasks:
             self._run_task(
-                pattern=pattern, 
+                pattern=pattern,
                 task=task, 
                 handlers=handlers,
-                remote_user=user
+                remote_user=user,
+                sudo=sudo,
+                sudo_user=sudo_user,
+                transport=transport,
+                port=port
             )
 
         # handlers only run on certain nodes, they are flagged by _flag_handlers
@@ -351,15 +553,21 @@ class PlayBook(object):
         # but Apache will only be restarted once (at the end).
 
         for task in handlers:
-            if type(task.get("run", None)) == list:
+            triggered_by = task.get('run', None)
+            if type(triggered_by) == list:
+                self.inventory.restrict_to(triggered_by)
                 self._run_task(
                    pattern=pattern, 
                    task=task,
-                   handlers=handlers,
-                   host_list=task.get('run',[]),
+                   handlers=[],
                    conditional=True,
-                   remote_user=user
+                   remote_user=user,
+                   sudo=sudo,
+                   sudo_user=sudo_user,
+                   transport=transport,
+                   port=port
                 )
+                self.inventory.lift_restriction()
 
         # end of execution for this particular pattern.  Multiple patterns
         # can be in a single playbook file

lib/ansible/utils.py

@@ -19,11 +19,21 @@
 
 import sys
 import os
+import shlex
+import re
+import codecs
+import jinja2
+import yaml
+import optparse
+from operator import methodcaller
 try:
     import json
 except ImportError:
     import simplejson as json
 
+from ansible import errors
+import ansible.constants as C
+
 ###############################################################
 # UTILITY FUNCTIONS FOR COMMAND LINE TOOLS
 ###############################################################
@@ -39,24 +49,32 @@ def exit(msg, rc=1):
 
 def bigjson(result):
     ''' format JSON output (uncompressed) '''
-    return json.dumps(result, sort_keys=True, indent=4)
+    # hide some internals magic from command line userland
+    result2 = result.copy()
+    if 'invocation' in result2:
+        del result2['invocation']
+    return json.dumps(result2, sort_keys=True, indent=4)
 
 def smjson(result):
     ''' format JSON output (compressed) '''
-    return json.dumps(result, sort_keys=True)
+    # hide some internals magic from command line userland
+    result2 = result.copy()
+    if 'invocation' in result2:
+        del result2['invocation']
+    return json.dumps(result2, sort_keys=True)
 
 def task_start_msg(name, conditional):
     if conditional:
-        return "NOTIFIED: [%s] **********\n" % name
+        return "\nNOTIFIED: [%s] **********\n" % name
     else:
-        return "TASK: [%s] *********\n" % name
+        return "\nTASK: [%s] *********\n" % name
 
 def regular_generic_msg(hostname, result, oneline, caption):
     ''' output on the result of a module run that is not command '''
     if not oneline:
-        return "%s | %s >>\n%s" % (hostname, caption, bigjson(result))
+        return "%s | %s >> %s\n" % (hostname, caption, bigjson(result))
     else:
-        return "%s | %s >> %s" % (hostname, caption, smjson(result))
+        return "%s | %s >> %s\n" % (hostname, caption, smjson(result))
 
 def regular_success_msg(hostname, result, oneline):
     ''' output the result of a successful module run '''
@@ -80,12 +98,13 @@ def command_generic_msg(hostname, result, oneline, caption):
             buf += stderr
         if msg:
             buf += msg
+        buf += "\n"
         return buf
     else:
         if stderr:
-            return "%s | %s | rc=%s | (stdout) %s (stderr) %s" % (hostname, caption, rc, stdout, stderr)
+            return "%s | %s | rc=%s | (stdout) %s (stderr) %s\n" % (hostname, caption, rc, stdout, stderr)
         else:
-            return "%s | %s | rc=%s | (stdout) %s" % (hostname, caption, rc, stdout)
+            return "%s | %s | rc=%s | (stdout) %s\n" % (hostname, caption, rc, stdout)
 
 def command_success_msg(hostname, result, oneline):
     ''' output from a successful command run '''
@@ -119,7 +138,7 @@ def host_report_msg(hostname, module_name, result, oneline):
     ''' summarize the JSON results for a particular host '''
     buf = ''
     failed = is_failed(result)
-    if module_name == 'command':
+    if module_name in [ 'command', 'shell' ] and 'ansible_job_id' not in result:
         if not failed:
             buf = command_success_msg(hostname, result, oneline)
         else:
@@ -131,28 +150,6 @@ def host_report_msg(hostname, module_name, result, oneline):
             buf = regular_failure_msg(hostname, result, oneline)
     return buf
 
-def dark_hosts_msg(results):
-    ''' summarize the results of all uncontactable hosts '''
-    buf = ''
-    if len(results['dark'].keys()) > 0:
-        buf += "*** Hosts which could not be contacted or did not respond: ***"
-        for hostname in results['dark'].keys():
-            buf += "%s:\n%s\n" % (hostname, results['dark'][hostname])
-    buf += "\n"
-    return buf
-
-def has_dark_hosts(results):
-    ''' are there any uncontactable hosts? '''
-    return len(results['dark'].keys()) > 0
-
-def contacted_hosts(results):
-    ''' what are the contactable hosts? '''
-    return sorted(results['contacted'])
-
-def contacted_host_result(results, hostname):
-    ''' what are the results for a given host? '''
-    return results['contacted'][hostname]
-
 def prepare_writeable_dir(tree):
     ''' make sure a directory exists and is writeable '''
     if tree != '/':
@@ -166,6 +163,7 @@ def prepare_writeable_dir(tree):
         exit("Cannot write to path %s" % tree)
 
 def path_dwim(basedir, given):
+    ''' make relative paths work like folks expect '''
     if given.startswith("/"):
         return given
     elif given.startswith("~/"):
@@ -173,4 +171,192 @@ def path_dwim(basedir, given):
     else:
         return os.path.join(basedir, given)
 
+def json_loads(data):
+    return json.loads(data)
+
+def parse_json(data):
+    ''' this version for module return data only '''
+    try:
+        return json.loads(data)
+    except:
+        # not JSON, but try "Baby JSON" which allows many of our modules to not
+        # require JSON and makes writing modules in bash much simpler
+        results = {}
+        tokens = shlex.split(data)
+        for t in tokens:
+            if t.find("=") == -1:
+                raise errors.AnsibleError("failed to parse: %s" % data)
+            (key,value) = t.split("=", 1)
+            if key == 'changed' or 'failed':
+                if value.lower() in [ 'true', '1' ]:
+                    value = True
+                elif value.lower() in [ 'false', '0' ]:
+                    value = False
+            if key == 'rc':
+                value = int(value)     
+            results[key] = value
+        if len(results.keys()) == 0:
+            return { "failed" : True, "parsed" : False, "msg" : data }
+        return results
+
+_LISTRE = re.compile(r"(\w+)\[(\d+)\]")
+
+def varLookup(name, vars):
+    ''' find the contents of a possibly complex variable in vars. '''
+    path = name.split('.')
+    space = vars
+    for part in path:
+        if part in space:
+            space = space[part]
+        elif "[" in part:
+            m = _LISTRE.search(part)
+            if not m:
+                return
+            try:
+                space = space[m.group(1)][int(m.group(2))]
+            except (KeyError, IndexError):
+                return
+        else:
+            return
+    return space
+
+_KEYCRE = re.compile(r"\$(?P<complex>\{){0,1}((?(complex)[\w\.\[\]]+|\w+))(?(complex)\})")
+#                        if { -> complex     if complex, allow . and need trailing }
+
+def varReplace(raw, vars):
+    '''Perform variable replacement of $vars
+
+    @param raw: String to perform substitution on.  
+    @param vars: Dictionary of variables to replace. Key is variable name
+        (without $ prefix). Value is replacement string.
+    @return: Input raw string with substituted values.
+    '''
+    # this code originally from yum
+
+    done = []                      # Completed chunks to return
+
+    while raw:
+        m = _KEYCRE.search(raw)
+        if not m:
+            done.append(raw)
+            break
+
+        # Determine replacement value (if unknown variable then preserve
+        # original)
+        varname = m.group(2)
+
+        replacement = unicode(varLookup(varname, vars) or m.group())
+
+        start, end = m.span()
+        done.append(raw[:start])    # Keep stuff leading up to token
+        done.append(replacement)    # Append replacement value
+        raw = raw[end:]             # Continue with remainder of string
+
+    return ''.join(done)
+
+def template(text, vars, setup_cache, no_engine=True):
+    ''' run a text buffer through the templating engine '''
+    vars = vars.copy()
+    vars['hostvars'] = setup_cache
+    text = varReplace(unicode(text), vars)
+    if no_engine:
+        # used when processing include: directives so that Jinja is evaluated
+        # in a later context when more variables are available
+        return text
+    else:
+        template = jinja2.Template(text)
+        res = template.render(vars)
+        if text.endswith('\n') and not res.endswith('\n'):
+            res = res + '\n'
+        return res
+
+def double_template(text, vars, setup_cache):
+    return template(template(text, vars, setup_cache), vars, setup_cache)
+
+def template_from_file(path, vars, setup_cache, no_engine=True):
+    ''' run a file through the templating engine '''
+    data = codecs.open(path, encoding="utf8").read()
+    return template(data, vars, setup_cache, no_engine=no_engine)
+
+def parse_yaml(data):
+    return yaml.load(data)
   
+def parse_yaml_from_file(path):
+    try:
+        data = file(path).read()
+    except IOError:
+        raise errors.AnsibleError("file not found: %s" % path)
+    return parse_yaml(data)
+
+def parse_kv(args):
+    ''' convert a string of key/value items to a dict '''
+    options = {}
+    if args is not None:
+        vargs = shlex.split(args, posix=True)
+        for x in vargs:
+            if x.find("=") != -1:
+                k, v = x.split("=", 1)
+                options[k]=v
+    return options
+
+class SortedOptParser(optparse.OptionParser):
+    '''Optparser which sorts the options by opt before outputting --help'''
+    def format_help(self, formatter=None):
+        self.option_list.sort(key=methodcaller('get_opt_string'))
+        return optparse.OptionParser.format_help(self, formatter=None)
+
+def base_parser(constants=C, usage="", output_opts=False, runas_opts=False, async_opts=False, connect_opts=False):
+    ''' create an options parser for any ansible script '''
+
+    parser = SortedOptParser(usage)
+    parser.add_option('-D','--debug', default=False, action="store_true",
+        help='debug mode')
+    parser.add_option('-f','--forks', dest='forks', default=constants.DEFAULT_FORKS, type='int',
+        help="specify number of parallel processes to use (default=%s)" % constants.DEFAULT_FORKS)
+    parser.add_option('-i', '--inventory-file', dest='inventory',
+        help="specify inventory host file (default=%s)" % constants.DEFAULT_HOST_LIST, 
+        default=constants.DEFAULT_HOST_LIST)
+    parser.add_option('-k', '--ask-pass', default=False, dest='ask_pass', action='store_true',
+        help='ask for SSH password')
+    parser.add_option('--private-key', default=None, dest='private_key_file',
+        help='use this file to authenticate the connection')
+    parser.add_option('-K', '--ask-sudo-pass', default=False, dest='ask_sudo_pass', action='store_true',
+        help='ask for sudo password')
+    parser.add_option('-M', '--module-path', dest='module_path',
+        help="specify path to module library (default=%s)" % constants.DEFAULT_MODULE_PATH, 
+        default=constants.DEFAULT_MODULE_PATH)
+    parser.add_option('-T', '--timeout', default=constants.DEFAULT_TIMEOUT, type='int',
+        dest='timeout', 
+        help="override the SSH timeout in seconds (default=%s)" % constants.DEFAULT_TIMEOUT)
+
+    if output_opts:
+        parser.add_option('-o', '--one-line', dest='one_line', action='store_true',
+            help='condense output')
+        parser.add_option('-t', '--tree', dest='tree', default=None,
+            help='log output to this directory')
+
+    if runas_opts:
+        parser.add_option("-s", "--sudo", default=False, action="store_true",
+            dest='sudo', help="run operations with sudo (nopasswd)")
+        parser.add_option('-U', '--sudo-user', dest='sudo_user', help='desired sudo user (default=root)',
+            default=None)   # Can't default to root because we need to detect when this option was given
+        parser.add_option('-u', '--user', default=constants.DEFAULT_REMOTE_USER,
+            dest='remote_user', 
+            help='connect as this user (default=%s)' % constants.DEFAULT_REMOTE_USER)
+    
+    if connect_opts:
+        parser.add_option('-c', '--connection', dest='connection',
+                          choices=C.DEFAULT_TRANSPORT_OPTS,
+                          default=C.DEFAULT_TRANSPORT,
+                          help="connection type to use (default=%s)" % C.DEFAULT_TRANSPORT)
+
+    if async_opts:
+        parser.add_option('-P', '--poll', default=constants.DEFAULT_POLL_INTERVAL, type='int',
+            dest='poll_interval', 
+            help="set the poll interval if using -B (default=%s)" % constants.DEFAULT_POLL_INTERVAL)
+        parser.add_option('-B', '--background', dest='seconds', type='int', default=0,
+            help='run asynchronously, failing after X seconds (default=N/A)')
+
+    return parser
+
+

library/apt

@@ -0,0 +1,174 @@
+#!/usr/bin/python -tt
+# (c) 2012, Flowroute LLC
+# Written by Matthew Williams <matthew@flowroute.com>
+# Based on yum module written by Seth Vidal <skvidal at fedoraproject.org>
+#
+# This module is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This software is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this software.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+import os
+import sys
+import shlex
+import subprocess
+import syslog
+import traceback
+
+APT_PATH = "/usr/bin/apt-get"
+APT = "DEBIAN_PRIORITY=critical %s" % APT_PATH
+
+def exit_json(rc=0, **kwargs):
+    print json.dumps(kwargs)
+    sys.exit(rc)
+
+def fail_json(**kwargs):
+    kwargs['failed'] = True
+    exit_json(rc=1, **kwargs)
+
+try:
+    import apt, apt_pkg
+except ImportError:
+    fail_json(msg="could not import apt, please install the python-apt package on this host")
+
+def run_apt(command):
+    try:
+        cmd = subprocess.Popen(command, shell=True, 
+            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        out, err = cmd.communicate()
+    except (OSError, IOError), e:
+        rc = 1
+        err = str(e)
+        out = ''
+    except:
+        rc = 1
+        err = traceback.format_exc()
+        out = ''
+    else:
+        rc = cmd.returncode
+        return rc, out, err
+
+def package_split(pkgspec):
+    parts = pkgspec.split('=')
+    if len(parts) > 1:
+        return parts[0], parts[1]
+    else:
+        return parts[0], None
+
+def package_status(pkgname, version, cache):
+    try:
+        pkg = cache[pkgname]
+    except KeyError:
+        fail_json(msg="No package matching '%s' is available" % pkgname)
+    if version:
+        return pkg.is_installed and pkg.installed.version == version, False
+    else:
+        return pkg.is_installed, pkg.is_upgradable
+
+def install(pkgspec, cache, upgrade=False, default_release=None):
+    name, version = package_split(pkgspec)
+    installed, upgradable = package_status(name, version, cache)
+    if not installed or (upgrade and upgradable):
+        cmd = "%s --option Dpkg::Options::=--force-confold -q -y install '%s'" % (APT, pkgspec)
+        if default_release:
+            cmd += " -t '%s'" % (default_release,)
+        rc, out, err = run_apt(cmd)
+        if rc:
+            fail_json(msg="'apt-get install %s' failed: %s" % (pkgspec, err))
+        return True
+    else:
+        return False
+
+def remove(pkgspec, cache, purge=False):
+    name, version = package_split(pkgspec)
+    installed, upgradable = package_status(name, version, cache)
+    if not installed:
+        return False
+    else:
+        purge = '--purge' if purge else ''
+        cmd = "%s -q -y %s remove '%s'" % (APT, purge, name)
+        rc, out, err = run_apt(cmd)
+        if rc:
+            fail_json(msg="'apt-get remove %s' failed: %s" % (name, err))
+        return True
+    
+
+# ===========================================
+
+if not os.path.exists(APT_PATH):
+   fail_json(msg="Cannot find apt-get")
+
+argfile = sys.argv[1]
+args    = open(argfile, 'r').read()
+items   = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+if not len(items):
+    fail_json(msg='the module requires arguments -a')
+    sys.exit(1)
+
+params = {}
+for x in items:
+    (k, v) = x.split("=", 1)
+    params[k] = v
+
+state            = params.get('state', 'installed')
+package          = params.get('pkg', params.get('package', params.get('name', None)))
+update_cache     = params.get('update-cache', 'no')
+purge            = params.get('purge', 'no')
+default_release  = params.get('default-release', None)
+
+if state not in ['installed', 'latest', 'removed']:
+    fail_json(msg='invalid state')
+
+if update_cache not in ['yes', 'no']:
+    fail_json(msg='invalid value for update_cache (requires yes or no -- default is no')
+
+if purge not in ['yes', 'no']:
+    fail_json(msg='invalid value for purge (requires yes or no -- default is no)')
+
+if package is None and update_cache != 'yes':
+    fail_json(msg='pkg=name and/or update-cache=yes is required')
+
+cache = apt.Cache()
+if default_release:
+    apt_pkg.config['APT::Default-Release'] = default_release
+    # reopen cache w/ modified config
+    cache.open()
+
+if update_cache == 'yes':
+    cache.update()
+    cache.open()
+    if package == None:
+        exit_json(changed=False)
+
+if package.count('=') > 1:
+    fail_json(msg='invalid package spec')
+
+if state == 'latest':
+    if '=' in package:
+        fail_json(msg='version number inconsistent with state=latest')
+    changed = install(package, cache, upgrade=True,
+                      default_release=default_release)
+elif state == 'installed':
+    changed = install(package, cache, default_release=default_release)
+elif state == 'removed':
+    changed = remove(package, cache, purge == 'yes')
+
+exit_json(changed=changed)
+
+

library/async_status

@@ -28,14 +28,19 @@ import subprocess
 import sys
 import datetime
 import traceback
+import syslog
 
 # ===========================================
-# convert arguments of form a=b c=d
-# to a dictionary
-# FIXME: make more idiomatic
 
-args = " ".join(sys.argv[1:])
+# FIXME: better error handling
+
+argsfile = sys.argv[1]
+args = open(argsfile, 'r').read()
 items = shlex.split(args)
+
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
 params = {}
 for x in items:
     (k, v) = x.split("=")
@@ -81,12 +86,22 @@ if mode == 'cleanup':
 data = file(log_path).read()
 try:
    data = json.loads(data)
-except:
-   print json.dumps({
-       "failed" : True,
-       "msg"    : "Could not parse job output"
-   })
-   sys.exit(1)
+except Exception, e:
+   if data == '':
+       # file not written yet?  That means it is running
+       print json.dumps({
+           "results_file" : log_path,
+           "ansible_job_id" : jid,
+           "started" : 1,
+       })
+   else:
+       print json.dumps({
+           "failed" : True,
+           "ansible_job_id" : jid,
+           "results_file" : log_path,
+           "msg" : "Could not parse job output: %s" % data,
+       })
+   sys.exit(0)
 
 if not data.has_key("started"):
    data['finished'] = 1

library/async_wrapper

@@ -30,20 +30,55 @@ import datetime
 import traceback
 import signal
 import time
+import syslog
+
+def daemonize_self():
+    # daemonizing code: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012
+    # logger.info("cobblerd started")
+    try:
+        pid = os.fork()
+        if pid > 0:
+            # exit first parent
+            sys.exit(0)
+    except OSError, e:
+        print >>sys.stderr, "fork #1 failed: %d (%s)" % (e.errno, e.strerror)
+        sys.exit(1)
+
+    # decouple from parent environment
+    os.chdir("/")
+    os.setsid()
+    os.umask(022)
+
+    # do second fork
+    try:
+        pid = os.fork()
+        if pid > 0:
+            # print "Daemon PID %d" % pid
+            sys.exit(0)
+    except OSError, e:
+        print >>sys.stderr, "fork #2 failed: %d (%s)" % (e.errno, e.strerror)
+        sys.exit(1)
+
+    dev_null = file('/dev/null','rw')
+    os.dup2(dev_null.fileno(), sys.stdin.fileno())
+    os.dup2(dev_null.fileno(), sys.stdout.fileno())
+    os.dup2(dev_null.fileno(), sys.stderr.fileno()) 
 
 if len(sys.argv) < 3:
     print json.dumps({
         "failed" : True,
-        "msg"    : "usage: async_wrapper <jid> <module_script> <time_limit> <args>.  Humans, do not call directly!"
+        "msg"    : "usage: async_wrapper <jid> <time_limit> <modulescript> <argsfile>.  Humans, do not call directly!"
     })
     sys.exit(1)
 
 jid = sys.argv[1]
 time_limit = sys.argv[2]
 wrapped_module = sys.argv[3]
-args = sys.argv[4:]
+argsfile = sys.argv[4]
+cmd = "%s %s" % (wrapped_module, argsfile)
 
-cmd = "%s %s" % (wrapped_module, " ".join(args))
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % " ".join(sys.argv[1:]))
 
 # setup logging directory
 logdir = os.path.expanduser("~/.ansible_async")
@@ -60,65 +95,105 @@ if not os.path.exists(logdir):
 
 def _run_command(wrapped_cmd, jid, log_path):
 
-    logfile = open(log_path, "w+")
+    logfile = open(log_path, "w")
     logfile.write(json.dumps({ "started" : 1, "ansible_job_id" : jid }))
+    logfile.close()
+    logfile = open(log_path, "w")
     result = {}
-
+  
+    outdata = ''
     try:
         cmd = shlex.split(wrapped_cmd)
         script = subprocess.Popen(cmd, shell=False, 
-            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-        out, err = script.communicate()
-        result = json.loads(out)
-    
+            stdin=None, stdout=logfile, stderr=logfile)
+        script.communicate()
+        outdata = file(log_path).read()
+        result = json.loads(outdata)
+
     except (OSError, IOError), e:
         result = {
             "failed": 1,
             "cmd" : wrapped_cmd,
             "msg": str(e),
         }
+        result['ansible_job_id'] = jid
+        logfile.write(json.dumps(result))
     except:
         result = {
             "failed" : 1,
             "cmd" : wrapped_cmd,
+            "data" : outdata, # temporary debug only
             "msg" : traceback.format_exc()
         }   
-
-    result['ansible_job_id'] = jid
-    logfile = open(log_path, "w+")
-    logfile.write(json.dumps(result))
+        result['ansible_job_id'] = jid
+        logfile.write(json.dumps(result))
     logfile.close()
 
 # immediately exit this process, leaving an orphaned process
 # running which immediately forks a supervisory timing process
 
-pid = os.fork()
-if pid == 0:
-   # "RETURNING SUCCESS IN UNO"
-   print json.dumps({ "started" : 1, "ansible_job_id" : jid })
-   sys.exit(0)
-else:
-   # "DAEMONIZED DOS"
-   sub_pid = os.fork()
-   if sub_pid == 0:
-       # "RUNNING IN KID A"
-       _run_command(cmd, jid, log_path)
-       sys.exit(0)
-   else:
-       # "WATCHING IN KID B"
-       remaining = int(time_limit)
-       if os.path.exists("/proc/%s" % sub_pid):
-           # "STILL RUNNING"
-           time.sleep(1)
-           remaining = remaining - 1
-       else:
-           # "DONE IN KID B"
-           sys.exit(0)
-       if remaining == 0:
-           # "SLAYING IN KID B"
-           os.kill(sub_pid, signals.SIGKILL)
-           sys.exit(1)
+#import logging
+#import logging.handlers
 
-   sys.exit(0)
+#logger = logging.getLogger("ansible_async")
+#logger.setLevel(logging.WARNING)
+#logger.addHandler( logging.handlers.SysLogHandler("/dev/log") )
+def debug(msg):
+    #logger.warning(msg)
+    pass
 
+try:
+    pid = os.fork()
+    if pid:
+        # Notify the overlord that the async process started
 
+        # we need to not return immmediately such that the launched command has an attempt
+        # to initialize PRIOR to ansible trying to clean up the launch directory (and argsfile)
+        # this probably could be done with some IPC later.  Modules should always read
+        # the argsfile at the very first start of their execution anyway
+        time.sleep(1)
+        debug("Return async_wrapper task started.")
+        print json.dumps({ "started" : 1, "ansible_job_id" : jid, "results_file" : log_path })
+        sys.stdout.flush()
+        sys.exit(0)
+    else:
+        # The actual wrapper process
+
+        # Daemonize, so we keep on running
+        daemonize_self()
+
+        # we are now daemonized, create a supervisory process
+        debug("Starting module and watcher")
+
+        sub_pid = os.fork()
+        if sub_pid:
+            # the parent stops the process after the time limit
+            remaining = int(time_limit)
+
+            # set the child process group id to kill all children
+            os.setpgid(sub_pid, sub_pid)
+
+            debug("Start watching %s (%s)"%(sub_pid, remaining))
+            time.sleep(5)
+            while os.waitpid(sub_pid, os.WNOHANG) == (0, 0):
+                debug("%s still running (%s)"%(sub_pid, remaining))
+                time.sleep(5)
+                remaining = remaining - 5
+                if remaining == 0:
+                    debug("Now killing %s"%(sub_pid))
+                    os.killpg(sub_pid, signal.SIGKILL)
+                    debug("Sent kill to group %s"%sub_pid)
+                    time.sleep(1)
+                    sys.exit(0)
+            debug("Done in kid B.")
+            os._exit(0)
+        else:
+            # the child process runs the actual module
+            debug("Start module (%s)"%os.getpid())
+            _run_command(cmd, jid, log_path)
+            debug("Module complete (%s)"%os.getpid())
+            sys.exit(0)
+
+except Exception, err:
+    debug("error: %s"%(err))
+    raise err

library/command

@@ -27,25 +27,55 @@ import subprocess
 import sys
 import datetime
 import traceback
+import shlex
+import os
+import syslog
 
-if len(sys.argv) == 1:
-    print json.dumps({
-        "failed" : True,
-        "msg"    : "the command module requires arguments (-a)"
-    })
-    sys.exit(1)
+argfile = sys.argv[1]
+args = open(argfile, 'r').read()
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+shell = False
+
+if args.find("#USE_SHELL") != -1:
+   args = args.replace("#USE_SHELL", "")
+   shell = True
+
+check_args = shlex.split(args)
+for x in check_args:
+   if x.startswith("creates="):
+       # do not run the command if the line contains creates=filename
+       # and the filename already exists.  This allows idempotence 
+       # of command executions.
+       (k,v) = x.split("=",1)
+       if os.path.exists(v):
+           print json.dumps({
+               "cmd"     : args,
+               "stdout"  : "skipped, since %s exists" % v,
+               "skipped" : True,
+               "changed" : False,
+               "stderr"  : "",
+               "rc"      : 0,
+           })
+           sys.exit(0)
+       args = args.replace(x,'')
+       
+
+if not shell:
+    args = shlex.split(args)
 
-args = sys.argv[1:]
 startd = datetime.datetime.now()
 
 try:
-    cmd = subprocess.Popen(args, shell=False, 
+    cmd = subprocess.Popen(args, shell=shell, 
         stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     out, err = cmd.communicate()
 except (OSError, IOError), e:
     print json.dumps({
-        "failed": 1,
-        "msg": str(e),
+        "cmd"    : args,
+        "failed" : 1,
+        "msg"    : str(e),
         })
     sys.exit(1)
 except:
@@ -64,12 +94,14 @@ if err is None:
    err = ''
 
 result = {
-   "stdout" : out.strip(),
-   "stderr" : err.strip(),
-   "rc"     : cmd.returncode,
-   "start"  : str(startd),
-   "end"    : str(endd),
-   "delta"  : str(delta),
+   "cmd"     : args,
+   "stdout"  : out.strip(),
+   "stderr"  : err.strip(),
+   "rc"      : cmd.returncode,
+   "start"   : str(startd),
+   "end"     : str(endd),
+   "delta"   : str(delta),
+   "changed" : True
 }
 
 print json.dumps(result)

library/copy

@@ -21,19 +21,25 @@
 import sys
 import os
 import shlex
-
-try:
-    import json
-except ImportError:
-    import simplejson as json
+import syslog
 
 # ===========================================
 # convert arguments of form a=b c=d
 # to a dictionary
 # FIXME: make more idiomatic
 
-args = " ".join(sys.argv[1:])
+if len(sys.argv) == 1:
+    sys.exit(1)
+argfile = sys.argv[1]
+if not os.path.exists(argfile):
+    sys.exit(1)
+
+args = open(argfile, 'r').read()
 items = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+
 params = {}
 for x in items:
     (k, v) = x.split("=")
@@ -41,14 +47,14 @@ for x in items:
 
 src  = params['src']
 dest = params['dest']
-
+if src:
+    src = os.path.expanduser(src)
+if dest:
+    dest = os.path.expanduser(dest)
  
 # raise an error if there is no src file
 if not os.path.exists(src):
-    print json.dumps({
-        "failed" : 1,
-        "msg"    : "Source %s failed to transfer" % src
-    })
+    print "failed=1 msg='Source %s failed to transfer'" % src
     sys.exit(1)
 
 md5sum = None
@@ -56,17 +62,13 @@ changed = False
 if os.path.exists(dest):
     md5sum = os.popen("md5sum %s" % dest).read().split()[0]
 
-os.system("cp %s %s" % (src, dest))
-
-md5sum2 = os.popen("md5sum %s" % dest).read().split()[0]
+md5sum2 = os.popen("md5sum %s" % src).read().split()[0]
 
 if md5sum != md5sum2:
+    os.system("cp %s %s" % (src, dest))
     changed = True
 
 # mission accomplished
-print json.dumps({
-   "md5sum"   : md5sum2,
-   "changed"  : changed 
-})
+print "md5sum=%s changed=%s" % (md5sum2, changed)
 
 

library/facter

@@ -22,4 +22,5 @@
 #    facter
 #    ruby-json
 
-/usr/bin/facter --json
+/usr/bin/logger -t ansible-facter Invoked as-is
+/usr/bin/facter --json 2>/dev/null

library/failtest

@@ -0,0 +1,34 @@
+#!/usr/bin/python
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+
+print >>sys.stderr, "THIS IS A TEST FAILURE"
+
+print json.dumps({ 
+    "failed" : True, 
+    "msg"    : "this module always fails" 
+})
+
+

library/fetch

@@ -0,0 +1,24 @@
+#!/usr/bin/python
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+### THIS FILE IS FOR REFERENCE OR FUTURE USE ###
+
+# See lib/ansible/runner.py for implementation of the fetch functionality #
+

library/file

@@ -0,0 +1,360 @@
+#!/usr/bin/python
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+import os
+import sys
+import shlex
+import subprocess
+import shutil
+import stat
+import grp
+import pwd
+import syslog
+try:
+    import selinux
+    HAVE_SELINUX=True
+except ImportError:
+    HAVE_SELINUX=False
+
+def dump_kv(vars):
+    return " ".join("%s=%s" % (k,v) for (k,v) in vars.items())
+
+def exit_kv(rc=0, **kwargs):
+    if 'path' in kwargs:
+        add_path_info(kwargs)
+    print dump_kv(kwargs)
+    sys.exit(rc)
+
+def fail_kv(**kwargs):
+    kwargs['failed'] = True
+    exit_kv(rc=1, **kwargs)
+
+def add_path_info(kwargs):
+    path = kwargs['path']
+    if os.path.exists(path):
+        (user, group) = user_and_group(path)
+        kwargs['user']  = user
+        kwargs['group'] = group
+        st = os.stat(path)
+        kwargs['mode']  = oct(stat.S_IMODE(st[stat.ST_MODE]))
+        # secontext not yet supported
+        if os.path.islink(path):
+            kwargs['state'] = 'link'
+        elif os.path.isfile(path):
+            kwargs['state'] = 'file'
+        else:
+            kwargs['state'] = 'directory'
+        if HAVE_SELINUX and selinux_enabled():
+            kwargs['secontext'] = ':'.join(selinux_context(path))
+    else:
+        kwargs['state'] = 'absent'
+    return kwargs 
+
+# Detect whether using selinux that is MLS-aware.
+# While this means you can set the level/range with
+# selinux.lsetfilecon(), it may or may not mean that you
+# will get the selevel as part of the context returned
+# by selinux.lgetfilecon().
+def selinux_mls_enabled():
+    if not HAVE_SELINUX:
+        return False
+    if selinux.is_selinux_mls_enabled() == 1:
+        return True
+    else:
+        return False
+
+def selinux_enabled():
+    if not HAVE_SELINUX:
+        return False
+    if selinux.is_selinux_enabled() == 1:
+        return True
+    else:
+        return False
+
+# Determine whether we need a placeholder for selevel/mls
+def selinux_initial_context():
+    context = [None, None, None]
+    if selinux_mls_enabled():
+        context.append(None)
+    return context
+
+# If selinux fails to find a default, return an array of None
+def selinux_default_context(path, mode=0):
+    context = selinux_initial_context()
+    if not HAVE_SELINUX or not selinux_enabled():
+        return context
+    try:
+        ret = selinux.matchpathcon(path, mode)
+    except OSError:
+        return context
+    if ret[0] == -1:
+        return context
+    context = ret[1].split(':')
+    return context
+
+def selinux_context(path):
+    context = selinux_initial_context()
+    if not HAVE_SELINUX or not selinux_enabled():
+        return context
+    try:
+        ret = selinux.lgetfilecon(path)
+    except:
+        fail_kv(path=path, msg='failed to retrieve selinux context')
+    if ret[0] == -1:
+        return context
+    context = ret[1].split(':')
+    return context
+
+# ===========================================
+
+argfile = sys.argv[1]
+args    = open(argfile, 'r').read()
+items   = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+if not len(items):
+    fail_kv(msg='the module requires arguments -a')
+    sys.exit(1)
+
+params = {}
+for x in items:
+    (k, v) = x.split("=")
+    params[k] = v
+
+state     = params.get('state','file')
+path      = params.get('path', params.get('dest', params.get('name', None)))
+if path:
+   path = os.path.expanduser(path)
+src       = params.get('src', None)
+if src:
+   src = os.path.expanduser(src)
+dest      = params.get('dest', None)
+mode      = params.get('mode', None)
+owner     = params.get('owner', None)
+group     = params.get('group', None)
+
+# presently unused, we always use -R (FIXME?)
+recurse   = params.get('recurse', 'false')
+
+# selinux related options
+seuser    = params.get('seuser', None)
+serole    = params.get('serole', None)
+setype    = params.get('setype', None)
+selevel   = params.get('serange', 's0')
+secontext = [seuser, serole, setype]
+if selinux_mls_enabled():
+    secontext.append(selevel)
+
+default_secontext = selinux_default_context(path)
+for i in range(len(default_secontext)):
+    if i is not None and secontext[i] == '_default':
+        secontext[i] = default_secontext[i]
+
+if state not in [ 'file', 'directory', 'link', 'absent']:
+    fail_kv(msg='invalid state: %s' % state)
+
+if state == 'link' and (src is None or dest is None):
+    fail_kv(msg='src and dest are required for "link" state')
+elif path is None:
+    fail_kv(msg='path is required')
+
+changed = False
+
+# ===========================================
+# support functions
+
+def md5sum(filename):
+    return os.popen("/usr/bin/md5sum %s" % f).read()
+
+def user_and_group(filename):
+    st = os.stat(filename)
+    uid = st.st_uid
+    gid = st.st_gid
+    user = pwd.getpwuid(uid)[0]
+    group = grp.getgrgid(gid)[0]
+    return (user, group)
+
+def set_context_if_different(path, context, changed):
+    if not HAVE_SELINUX or not selinux_enabled():
+        return changed
+    cur_context = selinux_context(path)
+    new_context = list(cur_context)
+    # Iterate over the current context instead of the
+    # argument context, which may have selevel.
+    for i in range(len(cur_context)):
+        if context[i] is not None and context[i] != cur_context[i]:
+            new_context[i] = context[i]
+    if cur_context != new_context:
+        try:
+            rc = selinux.lsetfilecon(path, ':'.join(new_context))
+        except OSError:
+            fail_kv(path=path, msg='invalid selinux context')
+        if rc != 0:
+            fail_kv(path=path, msg='set selinux context failed')
+        changed = True
+    return changed
+    
+def set_owner_if_different(path, owner, changed):
+   if owner is None:
+       return changed
+   user, group = user_and_group(path)
+   if owner != user:
+       rc = os.system("/bin/chown -R %s %s" % (owner, path))
+       if rc != 0:
+           fail_kv(path=path, msg='chown failed')
+       return True
+
+   return changed
+    
+def set_group_if_different(path, group, changed):
+   if group is None:
+       return changed
+   old_user, old_group = user_and_group(path)
+   if old_group != group:
+       rc = os.system("/bin/chgrp -R %s %s" % (group, path))
+       if rc != 0:
+           fail_kv(path=path, msg='chgrp failed')
+       return True
+   return changed
+
+def set_mode_if_different(path, mode, changed):
+   if mode is None:
+       return changed
+   try:
+       # FIXME: support English modes
+       mode = int(mode, 8)
+   except Exception, e:
+       fail_kv(path=path, msg='mode needs to be something octalish', details=str(e))  
+ 
+   st = os.stat(path)
+   prev_mode = stat.S_IMODE(st[stat.ST_MODE])
+
+   if prev_mode != mode:
+       # FIXME: comparison against string above will cause this to be executed
+       # every time
+       try:
+           os.chmod(path, mode)
+       except Exception, e:
+           fail_kv(path=path, msg='chmod failed', details=str(e))
+
+       st = os.stat(path)
+       new_mode = stat.S_IMODE(st[stat.ST_MODE])
+
+       if new_mode != prev_mode:
+           return True
+   return changed
+
+
+def rmtree_error(func, path, exc_info):
+   fail_kv(path=path, msg='failed to remove directory')
+
+# ===========================================
+# go...
+
+prev_state = 'absent'
+if os.path.lexists(path):
+    if os.path.islink(path):
+        prev_state = 'link'
+    elif os.path.isfile(path):
+        prev_state = 'file'
+    else:
+        prev_state = 'directory'
+
+if prev_state != 'absent' and state == 'absent':
+    try:
+        if prev_state == 'directory':
+            if os.path.islink(path):
+                os.unlink(path)
+            else:
+                shutil.rmtree(path, ignore_errors=False, onerror=rmtree_error)
+        else:
+            os.unlink(path)
+    except Exception, e:
+        fail_kv(path=path, msg=str(e))
+    exit_kv(path=path, changed=True)
+    sys.exit(0)
+
+if prev_state != 'absent' and prev_state != state:
+    fail_kv(path=path, msg='refusing to convert between %s and %s' % (prev_state, state))
+
+if prev_state == 'absent' and state == 'absent':
+    exit_kv(path=path, changed=False)
+
+if state == 'file':
+
+    if prev_state == 'absent':
+        fail_kv(path=path, msg='file does not exist, use copy or template module to create')
+
+    # set modes owners and context as needed
+    changed = set_context_if_different(path, secontext, changed)
+    changed = set_owner_if_different(path, owner, changed)
+    changed = set_group_if_different(path, group, changed)
+    changed = set_mode_if_different(path, mode, changed)
+
+    exit_kv(path=path, changed=changed)
+
+elif state == 'directory':
+
+    if prev_state == 'absent':
+        os.makedirs(path)
+        changed = True
+ 
+    # set modes owners and context as needed
+    changed = set_context_if_different(path, secontext, changed)
+    changed = set_owner_if_different(path, owner, changed)
+    changed = set_group_if_different(path, group, changed)
+    changed = set_mode_if_different(path, mode, changed)
+
+    exit_kv(path=path, changed=changed)
+
+elif state == 'link':
+    
+    if os.path.isabs(src):
+        abs_src = src
+    else:
+        abs_src = os.path.join(os.path.dirname(dest), src)
+    if not os.path.exists(abs_src):
+        fail_kv(dest=dest, src=src, msg='src file does not exist')
+   
+    if prev_state == 'absent':
+        os.symlink(src, dest)
+        changed = True
+    elif prev_state == 'link':
+        old_src = os.readlink(dest)
+        if not os.path.isabs(old_src):
+            old_src = os.path.join(os.path.dirname(dest), old_src)
+        if old_src != src:
+            os.unlink(dest)
+            os.symlink(src, dest)
+    else:
+        fail_kv(dest=dest, src=src, msg='unexpected position reached')
+
+    # set modes owners and context as needed
+    changed = set_context_if_different(dest, secontext, changed)
+    changed = set_owner_if_different(dest, owner, changed)
+    changed = set_group_if_different(dest, group, changed)
+    changed = set_mode_if_different(dest, mode, changed)
+
+    exit_kv(dest=dest, src=src, changed=changed)
+
+
+fail_kv(path=path, msg='unexpected position reached')
+sys.exit(0)
+

library/git

@@ -18,27 +18,52 @@
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 
 # I wanted to keep this simple at first, so for now this checks out
-# from the MASTER branch of a repo at a particular SHA or
+# from the given branch of a repo at a particular SHA or
 # tag.  Latest is not supported, you should not be doing
-# that.  Branch checkouts are not supported.  Contribs
-# welcome! -- MPD
+# that. Contribs welcome! -- MPD
 
 try:
     import json
 except ImportError:
     import simplejson as json
 import os
+import re
 import sys
 import shlex
 import subprocess
+import syslog
+
+# ===========================================
+# Basic support methods
+
+def exit_json(rc=0, **kwargs):
+   print json.dumps(kwargs)
+   sys.exit(rc)
+
+def fail_json(**kwargs):
+   kwargs['failed'] = True
+   exit_json(rc=1, **kwargs)
 
 # ===========================================
 # convert arguments of form a=b c=d
 # to a dictionary
 # FIXME: make more idiomatic
 
-args = " ".join(sys.argv[1:])
+if len(sys.argv) == 1:
+   fail_json(msg="the command module requires arguments (-a)")
+
+argfile = sys.argv[1]
+if not os.path.exists(argfile):
+   fail_json(msg="Argument file not found")
+
+args = open(argfile, 'r').read()
 items = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+if not len(items):
+   fail_json(msg="the command module requires arguments (-a)")
+
 params = {}
 for x in items:
     (k, v) = x.split("=")
@@ -46,7 +71,8 @@ for x in items:
 
 dest = params['dest']
 repo = params['repo']
-version = params['version']
+branch  = params.get('branch', 'master')
+version = params.get('version', 'HEAD')
 
 # ===========================================
 
@@ -58,7 +84,7 @@ def get_version(dest):
    sha = sha[0].split()[1]
    return sha
 
-def clone(repo, dest):
+def clone(repo, dest, branch):
    ''' makes a new git repo if it does not already exist '''
    try:
        os.makedirs(dest)
@@ -66,12 +92,68 @@ def clone(repo, dest):
        pass
    cmd = "git clone %s %s" % (repo, dest)
    cmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+   (out, err) = cmd.communicate()
+   rc = cmd.returncode
+   
+   if branch is None or rc != 0: 
+     return (out, err)
+
+   os.chdir(dest)
+   cmd = "git checkout -b %s origin/%s" % (branch, branch) 
+   cmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    return cmd.communicate()
 
-def pull(repo, dest):
+def reset(dest):
+   '''
+   Resets the index and working tree to HEAD.
+   Discards any changes to tracked files in working
+   tree since that commit.
+   '''
+   os.chdir(dest)
+   cmd = "git reset --hard HEAD"
+   cmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+   (out, err) = cmd.communicate()
+   rc = cmd.returncode
+   return (rc, out, err)
+
+def switchLocalBranch( branch ):
+   cmd = "git checkout %s" % branch
+   cmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+   return cmd.communicate()
+
+def pull(repo, dest, branch):
    ''' updates repo from remote sources '''
    os.chdir(dest)
-   cmd = "git pull -u origin"
+   cmd = "git branch -a"
+   cmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+   (gbranch_out, gbranch_err) = cmd.communicate()
+
+   try:
+      m = re.search( '^\* (\S+|\(no branch\))$', gbranch_out, flags=re.M )
+      cur_branch = m.group(1)
+      m = re.search( '\s+remotes/origin/HEAD -> origin/(\S+)', gbranch_out, flags=re.M )
+      default_branch = m.group(1)
+   except:
+      fail_json(msg="could not determine branch data - received: %s" % gbranch_out)
+
+   if branch is None:
+      if cur_branch != default_branch:
+         (out, err) = switchLocalBranch( default_branch )
+
+      cmd = "git pull -u origin"
+
+   elif branch == cur_branch:
+      cmd = "git pull -u origin"
+
+   else:
+      m = re.search( '^\s+%s$' % branch, gbranch_out, flags=re.M ) #see if we've already checked it out
+      if m is None:
+         cmd = "git checkout -b %s origin/%s" % (branch, branch)
+
+      else:
+         (out, err) = switchLocalBranch( branch )
+         cmd = "git pull -u origin"
+
    cmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    return cmd.communicate()
 
@@ -97,33 +179,26 @@ out, err, status = (None, None, None)
 
 before = None
 if not os.path.exists(gitconfig):
-   (out, err) = clone(repo, dest)
+   (out, err) = clone(repo, dest, branch)
 else:
    # else do a pull   
    before = get_version(dest)
-   (out, err) = pull(repo, dest)
+   (rc, out, err) = reset(dest)
+   if rc != 0:
+      fail_json(out=out, err=err)
+   (out, err) = pull(repo, dest, branch)
 
 # handle errors from clone or pull
 
 if out.find('error') != -1:
-   print json.dumps({
-       "failed" : True,
-       "out"    : out,
-       "err"    : err
-   })
-   sys.exit(1)
+   fail_json(out=out, err=err)
 
 # switch to version specified regardless of whether
 # we cloned or pulled
 
 (out, err) = switchver(version, dest)
 if err.find('error') != -1:
-   print json.dumps({
-       "failed" : True,
-       "out"    : out,
-       "err"    : err
-   })
-   sys.exit(1)
+   fail_json(out=out, err=err)
 
 # determine if we changed anything
 
@@ -133,9 +208,4 @@ changed = False
 if before != after:
    changed = True
 
-print json.dumps({
-   "changed" : changed,
-   "before"  : before,
-   "after"   : after
-})
-
+exit_json(changed=changed, before=before, after=after)

library/group

@@ -0,0 +1,173 @@
+#!/usr/bin/env python
+
+# (c) 2012, Stephen Fromm <sfromm@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+import os
+import grp
+import shlex
+import subprocess
+import sys
+import syslog
+
+GROUPADD = "/usr/sbin/groupadd"
+GROUPDEL = "/usr/sbin/groupdel"
+GROUPMOD = "/usr/sbin/groupmod"
+
+def exit_json(rc=0, **kwargs):
+    if 'name' in kwargs:
+        add_group_info(kwargs)
+    print json.dumps(kwargs)
+    sys.exit(rc)
+
+def fail_json(**kwargs):
+    kwargs['failed'] = True
+    exit_json(rc=1, **kwargs)
+
+def add_group_info(kwargs):
+    name = kwargs['name']
+    if group_exists(name):
+        kwargs['state'] = 'present'
+        info = group_info(name)
+        kwargs['gid'] = info[2]
+    else:
+        kwargs['state'] = 'absent'
+    return kwargs
+
+def group_del(group):
+    cmd = [GROUPDEL, group]
+    rc = subprocess.call(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if rc == 0:
+        return True
+    else:
+        return False
+
+def group_add(group, **kwargs):
+    cmd = [GROUPADD]
+    for key in kwargs:
+        if key == 'gid' and kwargs[key] is not None:
+            cmd.append('-g')
+            cmd.append(kwargs[key])
+        elif key == 'system' and kwargs[key] == 'yes':
+            cmd.append('-r')
+    cmd.append(group)
+    rc = subprocess.call(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if rc == 0:
+        return True
+    else:
+        return False
+
+def group_mod(group, **kwargs):
+    cmd = [GROUPMOD]
+    info = group_info(group)
+    for key in kwargs:
+        if key == 'gid':
+            if kwargs[key] is not None and info[2] != int(kwargs[key]):
+                cmd.append('-g')
+                cmd.append(kwargs[key])
+    if len(cmd) == 1:
+        return False
+    cmd.append(group)
+    rc = subprocess.call(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if rc == 0:
+        return True
+    else:
+        return False
+
+def group_exists(group):
+    try:
+        if grp.getgrnam(group):
+            return True
+    except KeyError:
+        return False
+
+def group_info(group):
+    if not group_exists(group):
+        return False
+    try:
+        info = list(grp.getgrnam(group))
+    except KeyError:
+        return False
+    return info
+    
+# ===========================================
+
+if not os.path.exists(GROUPADD):
+    if os.path.exists("/sbin/groupadd"):
+        GROUPADD = "/sbin/groupadd"
+    else:
+        fail_json(msg="Cannot find groupadd")
+if not os.path.exists(GROUPDEL):
+    if os.path.exists("/sbin/groupdel"):
+        GROUPDEL = "/sbin/groupdel"
+    else:
+        fail_json(msg="Cannot find groupdel")
+if not os.path.exists(GROUPMOD):
+    if os.path.exists("/sbin/groupmod"):
+        GROUPDEL = "/sbin/groupmod"
+    else:
+        fail_json(msg="Cannot find groupmod")
+
+if len(sys.argv) == 2 and os.path.exists(sys.argv[1]):
+    argfile = sys.argv[1]
+    args    = open(argfile, 'r').read()
+else:
+    args = ' '.join(sys.argv[1:])
+items   = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+if not len(items):
+    fail_json(msg='the module requires arguments -a')
+    sys.exit(1)
+
+params = {}
+for x in items:
+    (k, v) = x.split("=")
+    params[k] = v
+
+state       = params.get('state','present')
+name        = params.get('name', None)
+gid         = params.get('gid', None)
+system      = params.get('system', 'no')
+
+if state not in [ 'present', 'absent' ]:
+    fail_json(msg='invalid state')
+if system not in ['yes', 'no']:
+    fail_json(msg='invalid system')
+if name is None:
+    fail_json(msg='name is required')
+
+changed = False
+rc = 0
+if state == 'absent':
+    if group_exists(name):
+        changed = group_del(name)
+    exit_json(name=name, changed=changed)
+elif state == 'present':
+    if not group_exists(name):
+        changed = group_add(name, gid=gid, system=system)
+    else:
+        changed = group_mod(name, gid=gid)
+
+    exit_json(name=name, changed=changed)
+
+fail_json(name=name, msg='Unexpected position reached')

library/ohai

@@ -18,4 +18,5 @@
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 #
 
+/usr/bin/logger -t ansible-ohai Invoked as-is
 /usr/bin/ohai

library/ping

@@ -22,4 +22,10 @@ try:
 except ImportError:
     import simplejson as json
 
+import os
+import syslog
+
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked as-is')
+
 print json.dumps({ "ping" : "pong" })

library/raw

@@ -0,0 +1,23 @@
+#!/usr/bin/python
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+# hey the Ansible raw module isn't really a remote transferred
+# module.  All the magic happens in Runner.py, see the web docs
+# for more details.
+

library/service

@@ -21,84 +21,216 @@ try:
     import json
 except ImportError:
     import simplejson as json
-import os
 import sys
 import shlex
 import subprocess
+import os.path
+import syslog
+
+# TODO: switch to fail_json and other helper functions
+# like other modules are using
 
 # ===========================================
-# convert arguments of form a=b c=d
-# to a dictionary
-# FIXME: make more idiomatic
 
-args = " ".join(sys.argv[1:])
+SERVICE = None
+CHKCONFIG = None
+
+def fail_json(d):
+    print json.dumps(d)
+    sys.exit(1)
+
+def _find_binaries():
+    # list of possible paths for service/chkconfig binaries
+    # with the most probable first
+    global CHKCONFIG
+    global SERVICE
+    paths = ['/sbin', '/usr/sbin', '/bin', '/usr/bin']
+    binaries = [ 'service', 'chkconfig', 'update-rc.d' ]
+    location = dict()
+
+    for binary in binaries:
+        location[binary] = None
+
+    for binary in binaries:
+        for path in paths:
+            if os.path.exists(path + '/' + binary):
+                location[binary] = path + '/' + binary
+                break
+
+    if location.get('chkconfig', None):
+        CHKCONFIG = location['chkconfig']
+    elif location.get('update-rc.d', None):
+        CHKCONFIG = location['update-rc.d']
+    else:
+        fail_json(dict(failed=True, msg='unable to find chkconfig or update-rc.d binary'))
+    if location.get('service', None):
+        SERVICE = location['service']
+    else:
+        fail_json(dict(failed=True, msg='unable to find service binary'))
+
+
+def _run(cmd):
+    # returns (rc, stdout, stderr) from shell command
+    process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
+    stdout, stderr = process.communicate()
+    return (process.returncode, stdout, stderr) 
+
+
+def _do_enable(name, enable):
+    # we change argument depending on real binary used
+    # update-rc.d wants enable/disable while
+    # chkconfig wants on/off
+    valid_argument = dict({'on' : 'on', 'off' : 'off'})
+
+    if CHKCONFIG.endswith("update-rc.d"):
+        valid_argument['on'] = "enable"
+        valid_argument['off'] = "disable"
+
+    if enable.lower() in ['on', 'true', 'yes', 'enable']:
+        rc, stdout, stderr = _run("%s %s %s" % (CHKCONFIG, name, valid_argument['on']))
+    elif enable.lower() in ['off', 'false', 'no', 'disable']:
+        rc, stdout, stderr = _run("%s %s %s" % (CHKCONFIG, name, valid_argument['off']))
+    
+    return rc, stdout, stderr
+    
+argfile = sys.argv[1]
+args = open(argfile, 'r').read()
 items = shlex.split(args)
-params = {}
-for x in items:
-    (k, v) = x.split("=")
-    params[k] = v
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+if not len(items):
+    fail_json(dict(failed=True, msg='this module requires arguments (-a)'))
+
+params = {}
+for arg in items:
+    if "=" not in arg:
+        fail_json(dict(failed=True, msg='expected key=value format arguments'))
+
+    (name, value) = arg.split("=")
+    params[name] = value
+
+name = params.get('name', None)
+
+if name is None:
+    fail_json(dict(failed=True, msg='missing name'))
+
+state = params.get('state', None)
+list_items = params.get('list', None)
+enable = params.get('enabled', params.get('enable', None))
+
+# running and started are the same
+if state and state.lower() not in [ 'running', 'started', 'stopped', 'restarted','reloaded' ]:
+    fail_json(dict(failed=True, msg='invalid value for state'))
+if list_items and list_items.lower() not in [ 'status' ]:
+    fail_json(dict(failed=True, msg='invalid value for list'))
+if enable and enable.lower() not in [ 'on', 'off', 'true', 'false', 'yes', 'no', 'enable', 'disable' ]:
+    fail_json(dict(failed=True, msg='invalid value for enable'))
+    
+
+# ===========================================
+# find binaries locations on minion
+_find_binaries()
 
-name = params['name']
-state = params.get('state','running')
 
 # ===========================================
 # get service status
 
-status = os.popen("/sbin/service %s status" % name).read()
+rc, status_stdout, status_stderr = _run("%s %s status" % (SERVICE, name))
+status = status_stdout + status_stderr
 
-# ===========================================
-# determine if we are going to change anything
 
 running = False
-if status.find("not running") != -1:
+if status_stdout.find("stopped") != -1 or rc == 3:
     running = False
-elif status.find("running") != -1:
+elif status_stdout.find("running") != -1 or rc == 0:
     running = True
-elif name == 'iptables' and status.find("ACCEPT") != -1:
+elif name == 'iptables' and status_stdout.find("ACCEPT") != -1:
     # iptables status command output is lame
     # TODO: lookup if we can use a return code for this instead?
     running = True
 
-changed = False
-if not running and state == "started":
-    changed = True
-elif running and state == "stopped":
-    changed = True
-elif state == "restarted":
-    changed = True
+if state or enable:
+    rc = 0
+    out = ''
+    err = ''
+    changed = False
+        
+    if enable:
+        rc_enable, out_enable, err_enable = _do_enable(name, enable)
+        rc += rc_enable
+        out += out_enable
+        err += err_enable
 
-# ===========================================
-# run change commands if we need to
+    if state:
+        # a state change command has been requested
 
-def _run(cmd):
-    return subprocess.call(cmd, 
-        stdout=subprocess.PIPE,
-        stderr=subprocess.PIPE, shell=True)
+        # ===========================================
+        # determine if we are going to change anything
+
+        if not running and state in ("started", "running"):
+            changed = True
+        elif running and state in ("stopped","reloaded"):
+            changed = True
+        elif state == "restarted":
+            changed = True
+
+        # ===========================================
+        # run change commands if we need to
+
+        if changed:
+            if state in ('started', 'running'):
+                rc_state, stdout, stderr = _run("%s %s start" % (SERVICE, name))
+            elif state == 'stopped':
+                rc_state, stdout, stderr = _run("%s %s stop" % (SERVICE, name))
+            elif state == 'reloaded':
+                rc_state, stdout, stderr = _run("%s %s reload" % (SERVICE, name))
+            elif state == 'restarted':
+                rc1, stdout1, stderr1 = _run("%s %s stop" % (SERVICE, name))
+                rc2, stdout2, stderr2 = _run("%s %s start" % (SERVICE, name))
+                rc_state = rc + rc1 + rc2
+                stdout = stdout1 + stdout2
+                stderr = stderr1 + stderr2
+
+            out += stdout
+            err += stderr
+            rc = rc + rc_state
+
+    if rc != 0:
+
+        print json.dumps({
+            "failed" : 1,
+            "rc"     : rc,
+        })
+        print >> sys.stderr, out + err
+        sys.exit(1)
 
 
-rc = 0
-if changed:
-    if state == 'started':
-        rc = _run("/sbin/service %s start" %  name)
-    elif state == 'stopped':
-        rc = _run("/sbin/service %s stop" % name)
-    elif state == 'restarted':
-        rc1 = _run("/sbin/service %s stop" % name)
-        rc2 = _run("/sbin/service %s start" % name)
-        rc  = rc1 and rc2
+    # ===============================================
+    # success
+
+    result = {"changed": changed}
+
+    rc, stdout, stderr = _run("%s %s status" % (SERVICE, name))
+    if list_items and list_items in [ 'status' ]:
+        result['status'] = stdout
+    print json.dumps(result)
+
+    
+elif list_items is not None:
+
+    # solo list=status mode, don't change anything, just return
+    # suitable for /usr/bin/ansible usage or API, playbooks
+    # not so much
 
-if rc != 0:
-    # yeah, should probably include output of failure...
     print json.dumps({
-        "failed" : 1,
-        "rc"     : rc
+        "status" : status
     })
-    sys.exit(1)
 
-# ===============================================
-# success
+else:
+ 
+    print json.dumps(dict(failed=True, msg="expected state or list parameters"))
 
-print json.dumps({
-   "changed" : changed
-})
+
+sys.exit(0)
 

library/setup

@@ -19,21 +19,314 @@
 
 DEFAULT_ANSIBLE_SETUP     = "/etc/ansible/setup"
 
+import array
+import fcntl
+import glob
 import sys
 import os
+import platform
+import re
 import shlex
+import socket
+import struct
 import subprocess
+import traceback
+import syslog
 
 try:
     import json
 except ImportError:
     import simplejson as json
 
+_I386RE = re.compile(r'i[3456]86')
+SIOCGIFCONF   = 0x8912
+SIOCGIFHWADDR = 0x8927
+MEMORY_FACTS = ['MemTotal', 'SwapTotal', 'MemFree', 'SwapFree']
+# DMI bits
+DMI_DICT = { 'form_factor':  '/sys/devices/virtual/dmi/id/chassis_type',
+             'product_name': '/sys/devices/virtual/dmi/id/product_name',
+             'product_serial': '/sys/devices/virtual/dmi/id/product_serial',
+             'product_uuid': '/sys/devices/virtual/dmi/id/product_uuid',
+             'product_version': '/sys/devices/virtual/dmi/id/product_version',
+             'system_vendor': '/sys/devices/virtual/dmi/id/sys_vendor' }
+# From smolt and DMI spec
+FORM_FACTOR = [ "Unknown", "Other", "Unknown", "Desktop",
+                "Low Profile Desktop", "Pizza Box", "Mini Tower", "Tower",
+                "Portable", "Laptop", "Notebook", "Hand Held", "Docking Station",
+                "All In One", "Sub Notebook", "Space-saving", "Lunch Box",
+                "Main Server Chassis", "Expansion Chassis", "Sub Chassis",
+                "Bus Expansion Chassis", "Peripheral Chassis", "RAID Chassis",
+                "Rack Mount Chassis", "Sealed-case PC", "Multi-system",
+                "CompactPCI", "AdvancedTCA" ]
+# For the most part, we assume that platform.dist() will tell the truth.
+# This is the fallback to handle unknowns or exceptions
+OSDIST_DICT = { '/etc/redhat-release': 'RedHat',
+                '/etc/vmware-release': 'VMwareESX' }
+
+def get_file_content(path):
+    if os.path.exists(path) and os.access(path, os.R_OK):
+        data = open(path).read().strip()
+        if len(data) == 0:
+            data = None
+    else:
+        data = None
+    return data
+
+# platform.dist() is deprecated in 2.6
+# in 2.6 and newer, you should use platform.linux_distribution()
+def get_distribution_facts(facts):
+    dist = platform.dist()
+    facts['distribution'] = dist[0].capitalize() or 'NA'
+    facts['distribution_version'] = dist[1] or 'NA'
+    facts['distribution_release'] = dist[2] or 'NA'
+    # Try to handle the exceptions now ...
+    for (path, name) in OSDIST_DICT.items():
+        if os.path.exists(path):
+            if facts['distribution'] == 'Fedora':
+                pass
+            elif name == 'RedHat':
+                data = get_file_content(path)
+                if 'Red Hat' in data:
+                    facts['distribution'] = name
+                else:
+                    facts['distribution'] = data.split()[0]
+            else:
+                facts['distribution'] = name
+
+# Platform
+# patform.system() can be Linux, Darwin, Java, or Windows
+def get_platform_facts(facts):
+    facts['system'] = platform.system()
+    facts['kernel'] = platform.release()
+    facts['machine'] = platform.machine()
+    facts['python_version'] = platform.python_version()
+    if facts['machine'] == 'x86_64':
+        facts['architecture'] = facts['machine']
+    elif _I386RE.search(facts['machine']):
+        facts['architecture'] = 'i386'
+    else:
+        facts['archtecture'] = facts['machine']
+    if facts['system'] == 'Linux':
+        get_distribution_facts(facts)
+
+def get_memory_facts(facts):
+    if not os.access("/proc/meminfo", os.R_OK):
+        return facts
+    for line in open("/proc/meminfo").readlines():
+        data = line.split(":", 1)
+        key = data[0]
+        if key in MEMORY_FACTS:
+            val = data[1].strip().split(' ')[0]
+            facts["%s_mb" % key.lower()] = long(val) / 1024
+
+def get_cpu_facts(facts):
+    i = 0
+    physid = 0
+    sockets = {}
+    if not os.access("/proc/cpuinfo", os.R_OK):
+        return facts
+    for line in open("/proc/cpuinfo").readlines():
+        data = line.split(":", 1)
+        key = data[0].strip()
+        if key == 'model name':
+            if 'processor' not in facts:
+                facts['processor'] = []
+            facts['processor'].append(data[1].strip())
+            i += 1
+        elif key == 'physical id':
+            physid = data[1].strip()
+            if physid not in sockets:
+                sockets[physid] = 1
+        elif key == 'cpu cores':
+            sockets[physid] = int(data[1].strip())
+    if len(sockets) > 0:
+        facts['processor_count'] = len(sockets)
+        facts['processor_cores'] = reduce(lambda x, y: x + y, sockets.values())
+    else:
+        facts['processor_count'] = i
+        facts['processor_cores'] = 'NA'
+
+def get_hardware_facts(facts):
+    get_memory_facts(facts)
+    get_cpu_facts(facts)
+    for (key,path) in DMI_DICT.items():
+        data = get_file_content(path)
+        if data is not None:
+            if key == 'form_factor':
+                facts['form_factor'] = FORM_FACTOR[int(data)]
+            else:
+                facts[key] = data
+        else:
+            facts[key] = 'NA'
+
+def get_linux_virtual_facts(facts):
+    if os.path.exists("/proc/xen"):
+        facts['virtualization_type'] = 'xen'
+        facts['virtualization_role'] = 'guest'
+        if os.path.exists("/proc/xen/capabilities"):
+            facts['virtualization_role'] = 'host'
+    if os.path.exists("/proc/modules"):
+        modules = []
+        for line in open("/proc/modules").readlines():
+            data = line.split(" ", 1)
+            modules.append(data[0])
+        if 'kvm' in modules:
+            facts['virtualization_type'] = 'kvm'
+            facts['virtualization_role'] = 'host'
+        elif 'vboxdrv' in modules:
+            facts['virtualization_type'] = 'virtualbox'
+            facts['virtualization_role'] = 'host'
+        elif 'vboxguest' in modules:
+            facts['virtualization_type'] = 'virtualbox'
+            facts['virtualization_role'] = 'guest'
+    if 'QEMU' in facts['processor'][0]:
+        facts['virtualization_type'] = 'kvm'
+        facts['virtualization_role'] = 'guest'
+    if facts['distribution'] == 'VMwareESX':
+        facts['virtualization_type'] = 'VMware'
+        facts['virtualization_role'] = 'host'
+    # You can spawn a dmidecode process and parse that or infer from devices
+    for dev_model in glob.glob('/proc/ide/hd*/model'):
+        info = open(dev_model).read()
+        if 'VMware' in info:
+            facts['virtualization_type'] = 'VMware'
+            facts['virtualization_role'] = 'guest'
+        elif 'Virtual HD' in info or 'Virtual CD' in info:
+            facts['virtualization_type'] = 'VirtualPC'
+            facts['virtualization_role'] = 'guest'
+
+def get_virtual_facts(facts):
+    facts['virtualization_type'] = 'None'
+    facts['virtualization_role'] = 'None'
+    if facts['system'] == 'Linux':
+        facts = get_linux_virtual_facts(facts)
+
+# get list of interfaces that are up
+def get_interfaces():
+    length = 4096
+    offset = 32
+    step = 32
+    if platform.architecture()[0] == '64bit':
+        offset = 16
+        step = 40
+    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+    names = array.array('B', '\0' * length)
+    bytelen = struct.unpack('iL', fcntl.ioctl(
+        s.fileno(), SIOCGIFCONF, struct.pack(
+            'iL', length, names.buffer_info()[0])
+        ))[0]
+    return [names.tostring()[i:i+offset].split('\0', 1)[0]
+            for i in range(0, bytelen, step)]
+
+def get_iface_hwaddr(iface):
+    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+    info = fcntl.ioctl(s.fileno(), SIOCGIFHWADDR,
+                       struct.pack('256s', iface[:15]))
+    return ''.join(['%02x:' % ord(char) for char in info[18:24]])[:-1]
+
+def get_network_facts(facts):
+    facts['fqdn'] = socket.getfqdn()
+    facts['hostname'] = facts['fqdn'].split('.')[0]
+    facts['interfaces'] = get_interfaces()
+    for iface in facts['interfaces']:
+        facts[iface] = { 'macaddress': get_iface_hwaddr(iface) }
+        # This is lame, but there doesn't appear to be a good way
+        # to get all addresses for both IPv4 and IPv6.
+        cmd = subprocess.Popen("/sbin/ifconfig %s" % iface, shell=True,
+                               stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        out, err = cmd.communicate()
+        for line in out.split('\n'):
+            data = line.split()
+            if 'inet addr' in line:
+                if 'ipv4' not in facts[iface]:
+                    facts[iface]['ipv4'] = {}
+                facts[iface]['ipv4'] = { 'address': data[1].split(':')[1],
+                                         'netmask': data[-1].split(':')[1] }
+                ip = struct.unpack("!L", socket.inet_aton(facts[iface]['ipv4']['address']))[0]
+                mask = struct.unpack("!L", socket.inet_aton(facts[iface]['ipv4']['netmask']))[0]
+                facts[iface]['ipv4']['network'] = socket.inet_ntoa(struct.pack("!L", ip & mask))
+            if 'inet6 addr' in line:
+                (ip, prefix) = data[2].split('/')
+                scope = data[3].split(':')[1].lower()
+                if 'ipv6' not in facts[iface]:
+                    facts[iface]['ipv6'] = []
+                facts[iface]['ipv6'].append( { 'address': ip,
+                                               'prefix': prefix,
+                                               'scope': scope } )
+    return facts
+
+def get_public_ssh_host_keys(facts):
+    dsa = get_file_content('/etc/ssh/ssh_host_dsa_key.pub')
+    rsa = get_file_content('/etc/ssh/ssh_host_rsa_key.pub')
+    if dsa is None:
+        dsa = 'NA'
+    else:
+        facts['ssh_host_key_dsa_public'] = dsa.split()[1]
+    if rsa is None:
+        rsa = 'NA'
+    else:
+        facts['ssh_host_key_rsa_public'] = rsa.split()[1]
+
+def get_selinux_facts(facts):
+    if os.path.exists("/usr/sbin/sestatus"):
+        cmd = subprocess.Popen("/usr/sbin/sestatus", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        out, err = cmd.communicate()
+        if err == '':
+            facts['selinux'] = {}
+            list = out.split("\n")
+            status = re.search("(enabled|disabled)", list[0])
+            if status.group() == "enabled":
+                mode = re.search("(enforcing|disabled|permissive)", list[2])
+                config_mode = re.search("(enforcing|disabled|permissive)", list[3])
+                policyvers = re.search("\d+", list[4])
+                type = re.search("(targeted|strict|mls)", list[5])
+                facts['selinux']['status'] = status.group()
+                facts['selinux']['mode'] = mode.group()
+                facts['selinux']['config_mode'] = config_mode.group()
+                facts['selinux']['policyvers'] = policyvers.group()
+                facts['selinux']['type'] = type.group()
+            elif status.group() == "disabled":
+                facts['selinux']['status'] = status.group()            
+    else:
+        facts['selinux'] = False
+
+def get_service_facts(facts):
+    get_public_ssh_host_keys(facts)
+    get_selinux_facts(facts)
+
+def ansible_facts():
+    facts = {}
+    get_platform_facts(facts)
+    get_hardware_facts(facts)
+    get_virtual_facts(facts)
+    get_network_facts(facts)
+    get_service_facts(facts)
+    return facts
+
 # load config & template variables
 
-input_data = sys.argv[1:]
-new_options = dict([ x.split('=') for x in input_data ])
-ansible_file = new_options.get('metadata', DEFAULT_ANSIBLE_SETUP)
+if len(sys.argv) == 1:
+    sys.exit(1)
+
+
+argfile = sys.argv[1]
+if not os.path.exists(argfile):
+    sys.exit(1)
+
+setup_options = open(argfile).read().strip()
+try:
+   setup_options = json.loads(setup_options)
+except:
+   list_options = shlex.split(setup_options)
+   setup_options = {}
+   for opt in list_options:
+       (k,v) = opt.split("=")
+       setup_options[k]=v   
+
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % setup_options)
+
+ansible_file = os.path.expandvars(setup_options.get('metadata', DEFAULT_ANSIBLE_SETUP))
 ansible_dir = os.path.dirname(ansible_file)
 
 # create the config dir if it doesn't exist
@@ -48,6 +341,10 @@ if not os.path.exists(ansible_file):
 else:
     md5sum = os.popen("md5sum %s" % ansible_file).read().split()[0]
 
+# Get some basic facts in case facter or ohai are not installed
+for (k, v) in ansible_facts().items():
+    setup_options["ansible_%s" % k] = v
+
 # if facter is installed, and we can use --json because
 # ruby-json is ALSO installed, include facter data in the JSON
 
@@ -62,7 +359,7 @@ if os.path.exists("/usr/bin/facter"):
        facter = False
    if facter:
        for (k,v) in facter_ds.items():
-           new_options["facter_%s" % k] = v
+           setup_options["facter_%s" % k] = v
 
 # ditto for ohai, but just top level string keys
 # because it contains a lot of nested stuff we can't use for
@@ -81,13 +378,13 @@ if os.path.exists("/usr/bin/ohai"):
        for (k,v) in ohai_ds.items():
            if type(v) == str or type(v) == unicode:
                k2 = "ohai_%s" % k
-               new_options[k2] = v
+               setup_options[k2] = v
 
 # write the template/settings file using
 # instructions from server
 
 f = open(ansible_file, "w+")
-reformat = json.dumps(new_options, sort_keys=True, indent=4)
+reformat = json.dumps(setup_options, sort_keys=True, indent=4)
 f.write(reformat)
 f.close()
 
@@ -96,9 +393,11 @@ md5sum2 = os.popen("md5sum %s" % ansible_file).read().split()[0]
 if md5sum != md5sum2:
    changed = True
 
-new_options['written'] = ansible_file
-new_options['changed'] = changed
-new_options['md5sum']  = md5sum2 
+setup_result = {}
+setup_result['written'] = ansible_file
+setup_result['changed'] = changed
+setup_result['md5sum']  = md5sum2 
+setup_result['ansible_facts'] = setup_options
 
-print json.dumps(new_options)
+print json.dumps(setup_result)
 

library/shell

@@ -0,0 +1,6 @@
+# VIRTUAL
+
+There is actually no actual shell module source, when you use 'shell' in ansible, 
+it runs the 'command' module with special arguments and it behaves differently.  
+See the command source and the comment "#USE_SHELL".
+

library/slurp

@@ -0,0 +1,76 @@
+#!/usr/bin/python
+
+# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+import os
+import shlex
+import base64
+import syslog
+
+try:
+   import json
+except ImportError:
+   import simplejson as json
+
+# ===========================================
+# convert arguments of form a=b c=d
+# to a dictionary
+
+if len(sys.argv) == 1:
+    sys.exit(1)
+argfile = sys.argv[1]
+if not os.path.exists(argfile):
+    sys.exit(1)
+
+args = open(argfile, 'r').read()
+items = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+params = {}
+for x in items:
+    (k, v) = x.split("=")
+    params[k] = v
+source  = os.path.expanduser(params['src'])
+
+# ==========================================
+ 
+# raise an error if there is no template metadata
+if not os.path.exists(source):
+    print json.dumps(dict(
+        failed = 1,
+        msg    = "file not found: %s" % source
+    ))
+    sys.exit(1)
+
+if not os.access(source, os.R_OK):
+    print json.dumps(dict(
+        failed = 1,
+        msg    = "file is not readable: %s" % source
+    ))
+    sys.exit(1)
+
+# ==========================================
+
+data = file(source).read()
+data = base64.b64encode(data)
+
+print json.dumps(dict(content=data, encoding='base64'))
+sys.exit(0)
+

library/template

@@ -17,94 +17,8 @@
 # You should have received a copy of the GNU General Public License
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 
-import sys
-import os
-import jinja2
-import shlex
-try:
-    import json
-except ImportError:
-    import simplejson as json
-
-# ===========================================
-# convert arguments of form a=b c=d
-# to a dictionary
-# FIXME: make more idiomatic
-
-args = " ".join(sys.argv[1:])
-items = shlex.split(args)
-params = {}
-for x in items:
-    (k, v) = x.split("=")
-    params[k] = v
-
-source  = params['src']
-dest    = params['dest']
-metadata = params.get('metadata', '/etc/ansible/setup') 
-
- 
-# raise an error if there is no template metadata
-if not os.path.exists(metadata):
-    print json.dumps({
-        "failed" : 1,
-        "msg"    : "Missing %s, did you run the setup module yet?" % metadata
-    })
-    sys.exit(1)
-
-# raise an error if we can't parse the template metadata
-try:
-   f = open(metadata)
-   data = json.loads(f.read())
-   f.close()
-except:
-   print json.dumps({
-       "failed" : 1,
-       "msg"    : "Failed to parse/load %s, rerun the setup module?" % metadata
-   })
-   sys.exit(1)
-
-if not os.path.exists(source):
-    print json.dumps({
-        "failed" : 1,
-        "msg"    : "Source template could not be read: %s" % source
-    })
-    sys.exit(1)
-
-source = file(source).read()
-
-if os.path.isdir(dest):
-    print json.dumps({
-         "failed" : 1,
-         "msg"    : "Destination is a directory"
-    })
-    sys.exit(1)
-
-# record md5sum of original source file so we can report if it changed
-changed = False
-md5sum = None
-if os.path.exists(dest):
-    md5sum = os.popen("md5sum %s" % dest).read().split()[0]
-
-# call Jinja2 here and save the new template file
-template = jinja2.Template(source)
-data_out = template.render(data)
-f = open(dest, "w+")
-f.write(data_out)
-f.close()
-
-# TODO: catch templating errors and do not clobber the file on the
-# other end unless things were successful
-
-# record m5sum and return success and whether things have changed
-md5sum2 = os.popen("md5sum %s" % dest).read().split()[0]
-
-if md5sum != md5sum2:
-    changed = True
-
-# mission accomplished
-print json.dumps({
-   "md5sum"   : md5sum2,
-   "changed"  : changed 
-})
-
+# hey the Ansible template module isn't really a remote transferred
+# module.  All the magic happens in Runner.py making use of the
+# copy module, and if not running from a playbook, also the 'slurp'
+# module.
 

library/user

@@ -0,0 +1,337 @@
+#!/usr/bin/python
+
+# (c) 2012, Stephen Fromm <sfromm@gmail.com>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+import os
+import pwd
+import grp
+import shlex
+import subprocess
+import sys
+import syslog
+try:
+    import spwd
+    HAVE_SPWD=True
+except:
+    HAVE_SPWD=False
+
+USERADD = "/usr/sbin/useradd"
+USERMOD = "/usr/sbin/usermod"
+USERDEL = "/usr/sbin/userdel"
+
+def exit_json(rc=0, **kwargs):
+    if 'name' in kwargs:
+        add_user_info(kwargs)
+    print json.dumps(kwargs)
+    sys.exit(rc)
+
+def fail_json(**kwargs):
+    kwargs['failed'] = True
+    exit_json(rc=1, **kwargs)
+
+def add_user_info(kwargs):
+    name = kwargs['name']
+    if user_exists(name):
+        kwargs['state'] = 'present'
+        info = user_info(name)
+        kwargs['uid'] = info[2]
+        kwargs['group'] = info[3]
+        kwargs['comment'] = info[4]
+        kwargs['home'] = info[5]
+        kwargs['shell'] = info[6]
+        kwargs['createhome'] = os.path.exists(info[5])
+        groups = user_group_membership(name)
+        if len(groups) > 0:
+            kwargs['groups'] = groups
+    else:
+        kwargs['state'] = 'absent'
+    return kwargs
+
+def user_del(user, **kwargs):
+    cmd = [USERDEL]
+    for key in kwargs:
+        if key == 'force' and kwargs[key]:
+            cmd.append('-f')
+        elif key == 'remove' and kwargs[key]:
+            cmd.append('-r')
+    cmd.append(user)
+    rc = subprocess.call(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if rc == 0:
+        return True
+    else:
+        return False
+
+def user_add(user, **kwargs):
+    cmd = [USERADD]
+    for key in kwargs:
+        if key == 'uid' and kwargs[key] is not None:
+            cmd.append('-u')
+            cmd.append(kwargs[key])
+        elif key == 'group' and kwargs[key] is not None:
+            if not group_exists(kwargs[key]):
+                fail_json(msg="Group %s does not exist" % (kwargs[key]))
+            cmd.append('-g')
+            cmd.append(kwargs[key])
+        elif key == 'groups' and kwargs[key] is not None:
+            for g in kwargs[key].split(','):
+                if not group_exists(g):
+                    fail_json(msg="Group %s does not exist" % (g))
+            cmd.append('-G')
+            cmd.append(kwargs[key])
+        elif key == 'comment' and kwargs[key] is not None:
+            cmd.append('-c')
+            cmd.append(kwargs[key])
+        elif key == 'home' and kwargs[key] is not None:
+            cmd.append('-d')
+            cmd.append(kwargs[key])
+        elif key == 'shell' and kwargs[key] is not None:
+            cmd.append('-s')
+            cmd.append(kwargs[key])
+        elif key == 'password' and kwargs[key] is not None:
+            cmd.append('-p')
+            cmd.append(kwargs[key])
+        elif key == 'createhome':
+            if kwargs[key] is not None:
+                if kwargs[key] == 'yes':
+                    cmd.append('-m')
+                else:
+                    cmd.append('-M')
+        elif key == 'system' and kwargs[key] == 'yes':
+            cmd.append('-r')
+    cmd.append(user)
+    rc = subprocess.call(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if rc == 0:
+        return True
+    else:
+        return False
+
+"""
+Without spwd, we would have to resort to reading /etc/shadow
+to get the encrypted string.  For now, punt on idempotent password changes.
+"""
+def user_mod(user, **kwargs):
+    cmd = [USERMOD]
+    info = user_info(user)
+    for key in kwargs:
+        if key == 'uid':
+            if kwargs[key] is not None and info[2] != int(kwargs[key]):
+                cmd.append('-u')
+                cmd.append(kwargs[key])
+        elif key == 'group' and kwargs[key] is not None:
+            if not group_exists(kwargs[key]):
+                fail_json(msg="Group %s does not exist" % (kwargs[key]))
+            ginfo = group_info(group)
+            if info[3] != ginfo[2]:
+                cmd.append('-g')
+                cmd.append(kwargs[key])
+        elif key == 'groups' and kwargs[key] is not None:
+            current_groups = user_group_membership(user)
+            groups = kwargs[key].split(',')
+            for g in groups:
+                if not group_exists(g):
+                    fail_json(msg="Group %s does not exist" % (g))
+            group_diff = set(sorted(current_groups)).symmetric_difference(set(sorted(groups)))
+            groups_need_mod = False
+
+            if group_diff:
+                if kwargs['append'] is not None and kwargs['append'] == 'yes':
+                    for g in groups:
+                        if g in group_diff:
+                            cmd.append('-a')
+                            groups_need_mod = True
+                else:
+                   groups_need_mod = True
+
+            if groups_need_mod:
+                cmd.append('-G')
+                cmd.append(','.join(groups))
+
+        elif key == 'comment':
+            if kwargs[key] is not None and info[4] != kwargs[key]:
+                cmd.append('-c')
+                cmd.append(kwargs[key])
+        elif key == 'home':
+            if kwargs[key] is not None and info[5] != kwargs[key]:
+                cmd.append('-d')
+                cmd.append(kwargs[key])
+        elif key == 'shell':
+            if kwargs[key] is not None and info[6] != kwargs[key]:
+                cmd.append('-s')
+                cmd.append(kwargs[key])
+        elif key == 'password':
+            if kwargs[key] is not None and info[1] != kwargs[key]:
+                cmd.append('-p')
+                cmd.append(kwargs[key])
+    # skip if no changes to be made
+    if len(cmd) == 1:
+        return False
+    cmd.append(user)
+    rc = subprocess.call(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if rc == 0:
+        return True
+    else:
+        return False
+
+def group_exists(group):
+    try:
+        if group.isdigit():
+            if grp.getgrgid(group):
+                return True
+        else:
+            if grp.getgrnam(group):
+                return True
+    except KeyError:
+        return False
+
+def group_info(group):
+    if not group_exists(group):
+        return False
+    if group.isdigit():
+        return list(grp.getgrgid(group))
+    else:
+        return list(grp.getgrnam(group))
+
+def user_group_membership(user):
+    groups = []
+    info = get_pwd_info(user)
+    for group in grp.getgrall():
+        if user in group[3] and info[3] != group[2]:
+            groups.append(group[0])
+    return groups
+
+def user_exists(user):
+    try:
+        if pwd.getpwnam(user):
+            return True
+    except KeyError:
+        return False
+
+def get_pwd_info(user):
+    if not user_exists(user):
+        return False
+    return list(pwd.getpwnam(user))
+
+def user_info(user):
+    if not user_exists(user):
+        return False
+    try:
+        info = get_pwd_info(user)
+        if HAVE_SPWD:
+            sinfo = spwd.getspnam(user)
+    except KeyError:
+        return False
+    if HAVE_SPWD:
+        info[1] = sinfo[1]
+    return info
+
+# ===========================================
+
+if not os.path.exists(USERADD):
+    if os.path.exists("/sbin/useradd"):
+        USERADD = "/sbin/useradd"
+    else:
+        fail_json(msg="Cannot find useradd")
+if not os.path.exists(USERMOD):
+    if os.path.exists("/sbin/usermod"):
+        USERMOD = "/sbin/usermod"
+    else:
+        fail_json(msg="Cannot find usermod")
+if not os.path.exists(USERDEL):
+    if os.path.exists("/sbin/userdel"):
+        USERDEL = "/sbin/userdel"
+    else:
+        fail_json(msg="Cannot find userdel")
+
+argfile = sys.argv[1]
+args    = open(argfile, 'r').read()
+items   = shlex.split(args)
+syslog.openlog('ansible-%s' % os.path.basename(__file__))
+syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+if not len(items):
+    fail_json(msg='the module requires arguments -a')
+    sys.exit(1)
+
+params = {}
+for x in items:
+    (k, v) = x.split("=")
+    params[k] = v
+
+state      = params.get('state','present')
+name       = params.get('name', None)
+uid        = params.get('uid', None)
+group      = params.get('group', None)
+groups     = params.get('groups', None)
+comment    = params.get('comment', None)
+home       = params.get('home', None)
+shell      = params.get('shell', None)
+password   = params.get('password', None)
+
+# ===========================================
+# following options are specific to userdel
+force      = params.get('force', False)
+remove     = params.get('remove', False)
+
+# ===========================================
+# following options are specific to useradd
+createhome = params.get('createhome', 'yes')
+system     = params.get('system', 'no')
+
+# ===========================================
+# following options are specific to usermod
+append     = params.get('append', 'no')
+
+if state not in [ 'present', 'absent' ]:
+    fail_json(msg='invalid state')
+if createhome not in [ 'yes', 'no' ]:
+    fail_json(msg='invalid createhome')
+if system not in ['yes', 'no']:
+    fail_json(msg='invalid system')
+if append not in [ 'yes', 'no' ]:
+    fail_json(msg='invalid append')
+if name is None:
+    fail_json(msg='name is required')
+
+changed = False
+rc = 0
+if state == 'absent':
+    if user_exists(name):
+        changed = user_del(name, force=force, remove=remove)
+    exit_json(name=name, changed=changed, force=force, remove=remove)
+elif state == 'present':
+    if not user_exists(name):
+        changed = user_add(name, uid=uid, group=group, groups=groups,
+                           comment=comment, home=home, shell=shell,
+                           password=password, createhome=createhome,
+                           system=system)
+    else:
+        changed = user_mod(name, uid=uid, group=group, groups=groups,
+                           comment=comment, home=home, shell=shell,
+                           password=password, append=append)
+
+    if password is not None:
+        exit_json(name=name, changed=changed, password="XXXXXXXX")
+    else:
+        exit_json(name=name, changed=changed)
+
+fail_json(name=name, msg='Unexpected position reached')
+sys.exit(0)

library/virt

@@ -0,0 +1,453 @@
+#!/usr/bin/python 
+"""
+Virt management features
+
+Copyright 2007, 2012 Red Hat, Inc
+Michael DeHaan <mdehaan@redhat.com>
+Seth Vidal <skvidal@fedoraproject.org>
+
+This software may be freely redistributed under the terms of the GNU
+general public license.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+VIRT_FAILED = 1
+VIRT_SUCCESS = 0
+VIRT_UNAVAILABLE=2
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+
+
+# other modules
+import os
+import sys
+import subprocess
+import syslog
+try:
+    import libvirt
+except ImportError:
+    print json.dumps({
+        "failed" : True,
+        "msg" : "libvirt python module unavailable",
+        "rc": VIRT_UNAVAILABLE,
+        })   
+    sys.exit(1)
+
+
+import shlex
+
+VIRT_STATE_NAME_MAP = {
+   0 : "running",
+   1 : "running",
+   2 : "running",
+   3 : "paused",
+   4 : "shutdown",
+   5 : "shutdown",
+   6 : "crashed"
+}
+
+class LibvirtConnection(object):
+
+    def __init__(self):
+
+        cmd = subprocess.Popen("uname -r", shell=True, stdout=subprocess.PIPE,
+                                close_fds=True)
+        output = cmd.communicate()[0]
+
+        if output.find("xen") != -1:
+            conn = libvirt.open(None)
+        else:
+            conn = libvirt.open("qemu:///system")
+
+        if not conn:
+            raise Exception("hypervisor connection failure")
+
+        self.conn = conn
+
+    def find_vm(self, vmid):
+        """
+        Extra bonus feature: vmid = -1 returns a list of everything
+        """
+        conn = self.conn
+
+        vms = []
+
+        # this block of code borrowed from virt-manager:
+        # get working domain's name
+        ids = conn.listDomainsID()
+        for id in ids:
+            vm = conn.lookupByID(id)
+            vms.append(vm)
+        # get defined domain
+        names = conn.listDefinedDomains()
+        for name in names:
+            vm = conn.lookupByName(name)
+            vms.append(vm)
+
+        if vmid == -1:
+            return vms
+
+        for vm in vms:
+            if vm.name() == vmid:
+                return vm
+
+        raise Exception("virtual machine %s not found" % vmid)
+
+    def shutdown(self, vmid):
+        return self.find_vm(vmid).shutdown()
+
+    def pause(self, vmid):
+        return self.suspend(self.conn,vmid)
+
+    def unpause(self, vmid):
+        return self.resume(self.conn,vmid)
+
+    def suspend(self, vmid):
+        return self.find_vm(vmid).suspend()
+
+    def resume(self, vmid):
+        return self.find_vm(vmid).resume()
+
+    def create(self, vmid):
+        return self.find_vm(vmid).create()
+
+    def destroy(self, vmid):
+        return self.find_vm(vmid).destroy()
+
+    def undefine(self, vmid):
+        return self.find_vm(vmid).undefine()
+
+    def get_status2(self, vm):
+        state = vm.info()[0]
+        # print "DEBUG: state: %s" % state
+        return VIRT_STATE_NAME_MAP.get(state,"unknown")
+
+    def get_status(self, vmid):
+        state = self.find_vm(vmid).info()[0]
+        return VIRT_STATE_NAME_MAP.get(state,"unknown")
+
+    def nodeinfo(self):
+        return self.conn.getInfo()
+
+    def get_type(self):
+        return self.conn.getType()
+
+    def get_maxVcpus(self, vmid):
+        vm = self.conn.lookupByName(vmid)
+        return vm.maxVcpus()
+
+    def get_maxMemory(self, vmid):
+        vm = self.conn.lookupByName(vmid)
+        return vm.maxMemory()
+
+    def getFreeMemory(self):
+        return self.conn.getFreeMemory()
+
+    def get_autostart(self, vmid):
+        vm = self.conn.lookupByName(vmid)
+        return vm.autostart()
+    
+    def set_autostart(self, vmid, val):
+        vm = self.conn.lookupByName(vmid)
+        return vm.setAutostart(val)
+    
+
+
+class Virt(object):
+
+    def __get_conn(self):
+        self.conn = LibvirtConnection()
+        return self.conn
+
+    def get_vm(self, vmid):
+        self.__get_conn()
+        return self.conn.find_vm(vmid)
+        
+    def state(self):
+        vms = self.list_vms()
+        state = []
+        for vm in vms:
+            state_blurb = self.conn.get_status(vm)
+            state.append("%s %s" % (vm,state_blurb))
+        return state
+
+
+    def info(self):
+        vms = self.list_vms()
+        info = dict()
+        for vm in vms:
+            data = self.conn.find_vm(vm).info()
+            # libvirt returns maxMem, memory, and cpuTime as long()'s, which
+            # xmlrpclib tries to convert to regular int's during serialization.
+            # This throws exceptions, so convert them to strings here and
+            # assume the other end of the xmlrpc connection can figure things
+            # out or doesn't care.
+            info[vm] = {
+                "state"     : VIRT_STATE_NAME_MAP.get(data[0],"unknown"),
+                "maxMem"    : str(data[1]),
+                "memory"    : str(data[2]),
+                "nrVirtCpu" : data[3],
+                "cpuTime"   : str(data[4]),
+            }
+            info[vm]["autostart"] =  self.conn.get_autostart(vm)
+
+        return info
+
+    def nodeinfo(self):
+        self.__get_conn()
+        info = dict()
+        data = self.conn.nodeinfo()
+        info = {
+            "cpumodel"     : str(data[0]),
+            "phymemory"    : str(data[1]),
+            "cpus"         : str(data[2]),
+            "cpumhz"       : str(data[3]),
+            "numanodes"    : str(data[4]),
+            "sockets"      : str(data[5]),
+            "cpucores"     : str(data[6]),
+            "cputhreads"   : str(data[7])
+        }
+        return info
+
+    def list_vms(self):
+        self.conn = self.__get_conn()
+        vms = self.conn.find_vm(-1)
+        results = []
+        for x in vms:
+            try:
+                results.append(x.name())
+            except:
+                pass
+        return results
+
+    def virttype(self):
+        return self.__get_conn().get_type()
+        
+
+    def autostart(self, vmid):
+        self.conn = self.__get_conn()
+        return self.conn.set_autostart(vmid, True)
+
+    def freemem(self):
+        self.conn = self.__get_conn()
+        return self.conn.getFreeMemory()
+
+    def shutdown(self, vmid):
+        """
+        Make the machine with the given vmid stop running.
+        Whatever that takes.
+        """
+        self.__get_conn()
+        self.conn.shutdown(vmid)
+        return 0
+
+
+    def pause(self, vmid):
+
+        """
+        Pause the machine with the given vmid.
+        """
+        self.__get_conn()
+        return self.conn.suspend(vmid)
+
+
+    def unpause(self, vmid):
+
+        """
+        Unpause the machine with the given vmid.
+        """
+
+        self.__get_conn()
+        return self.conn.resume(vmid)
+
+
+    def create(self, vmid):
+
+        """
+        Start the machine via the given mac address.
+        """
+        self.__get_conn()
+        return self.conn.create(vmid)
+
+    def start(self, vmid):
+
+        """
+        Start the machine via the given id/name
+        """
+        self.__get_conn()
+        return self.conn.create(vmid)
+
+
+    def destroy(self, vmid):
+
+        """
+        Pull the virtual power from the virtual domain, giving it virtually no
+        time to virtually shut down.
+        """
+        self.__get_conn()
+        return self.conn.destroy(vmid)
+
+
+    def undefine(self, vmid):
+
+        """
+        Stop a domain, and then wipe it from the face of the earth.
+        by deleting the disk image and its configuration file.
+        """
+
+        self.__get_conn()
+        return self.conn.undefine(vmid)
+
+
+    def status(self, vmid):
+
+        """
+        Return a state suitable for server consumption.  Aka, codes.py values, not XM output.
+        """
+
+        self.__get_conn()
+        return self.conn.get_status(vmid)
+
+
+    def get_xml(self, vmid):
+        """
+        Receive a Vm id as input
+        Return an xml describing vm config returned by a libvirt call
+        """
+        conn = libvirt.openReadOnly(None)
+        if conn == None:
+            return (-1,'Failed to open connection to the hypervisor')
+        try:
+            domV = conn.lookupByName(vmid)
+        except:
+            return (-1,'Failed to find the main domain')
+        return domV.XMLDesc(0)
+
+
+    def get_maxVcpus(self, vmid):
+        """
+        Gets the max number of VCPUs on a guest
+        """
+
+        self.__get_conn()
+        return self.conn.get_maxVcpus(vmid)
+
+    def get_max_memory(self, vmid):
+        """
+        Gets the max memory on a guest
+        """
+
+        self.__get_conn()
+        return self.conn.get_MaxMemory(vmid)
+
+def main():
+    rc = VIRT_SUCCESS
+    vm_commands = ['create','status', 'start', 'stop', 'pause', 'unpause', 
+                'shutdown', 'undefine', 'destroy', 'get_xml', 'autostart']
+    host_commands = ['freemem', 'list_vms', 'info', 'nodeinfo', 'virttype']
+    
+    msg = """
+    virtmodule arguments:
+      state=[running|shutdown] guest=guestname
+      command=some_virt_command  [guest=guestname]
+    """
+    
+    if len(sys.argv) == 1:
+        return VIRT_FAILED, msg
+
+    argfile = sys.argv[1]
+    if not os.path.exists(argfile):
+        msg = "Argument file not found"
+        return VIRT_FAILED, msg
+
+    args = open(argfile, 'r').read()
+    items = shlex.split(args)
+    syslog.openlog('ansible-%s' % os.path.basename(__file__))
+    syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+    if not len(items):
+        return VIRT_FAILED, msg
+
+
+    # guest=name state=[running|shutdown|destroyed|undefined]
+    # command=[some command] [guest=name]
+    
+    params = {}
+    if '=' not in args:
+        msg = "No proper arguments provided to virt module: %s" % args
+        return VIRT_FAILED, msg
+        
+    for x in items:
+        (k, v) = x.split("=")
+        params[k] = v
+
+    state      = params.get('state', None)
+    guest      = params.get('guest', None)
+    command    = params.get('command', None)
+    options    = params.get('options', [])
+
+    v = Virt()
+    res = {}
+    
+    if state:
+        if not guest:
+            msg = "state change requires a guest specified"
+            return VIRT_FAILED, msg
+      
+        res['changed'] = False
+        if state == 'running':
+            if v.status(guest) is not 'running':
+                res['changed'] = True 
+                res['msg'] = v.start(guest)
+        elif state == 'shutdown':
+            if v.status(guest) is not 'shutdown':
+                res['changed'] = True
+                res['msg'] = v.shutdown(guest)
+        
+        return VIRT_SUCCESS, res
+        
+    if command:
+        if command in vm_commands:
+            if not guest:
+                msg = "%s requires 1 argument: guest" % command
+                return VIRT_FAILED, msg
+                             
+            res = getattr(v, command)(guest)
+            if type(res) != dict:
+                res = { command: res }
+            return rc, res
+        
+        elif hasattr(v, command):
+            res = getattr(v, command)()
+            if type(res) != dict:
+                res = { command: res }
+            return rc, res
+            
+        else:
+            msg = "Command %s not recognized" % basecmd
+            rc = VIRT_FAILED
+        
+    return rc, msg
+    
+if __name__ == "__main__":
+    try:
+        rc, result = main()
+    except Exception, e:
+        rc = 1
+        result = str(e)
+
+    if rc != 0: # something went wrong emit the msg
+        print json.dumps({
+            "failed" : rc,
+            "msg" : result,
+            "rc": rc,
+        })   
+        sys.exit(rc)
+    else:
+        print json.dumps(result)
+    

library/yum

@@ -0,0 +1,361 @@
+#!/usr/bin/python -tt
+# (c) 2012, Red Hat, Inc
+# Written by Seth Vidal <skvidal at fedoraproject.org>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+import os
+import sys
+import yum
+import subprocess
+import datetime
+import shlex
+import re
+import traceback
+import syslog
+
+
+try:
+    import json
+except ImportError:
+    import simplejson as json
+    
+
+def yum_base(conf_file=None, cachedir=False):
+    my = yum.YumBase()
+    my.preconf.debuglevel=0
+    if conf_file and os.path.exists(conf_file):
+        my.preconf.fn = conf_file
+    if cachedir:
+        if hasattr(my, 'setCacheDir'):
+            my.setCacheDir()
+        else:
+            cachedir = yum.misc.getCacheDir()
+            my.repos.setCacheDir(cachedir)
+            my.conf.cache = 0 
+    
+    return my
+
+def pkg_to_dict(po):
+    d = {
+        'name':po.name,
+        'arch':po.arch,
+        'epoch':po.epoch,
+        'release':po.release,
+        'version':po.version,
+        }
+        
+    if type(po) == yum.rpmsack.RPMInstalledPackage:
+        d['yumstate'] = 'installed'
+        d['repo'] = 'installed'
+    else:
+        d['yumstate'] = 'available'
+        d['repo'] =  po.repoid
+    
+    if hasattr(po, 'ui_from_repo'):
+        d['repo'] = po.ui_from_repo
+        
+    if hasattr(po, 'ui_nevra'):
+        d['_nevra'] = po.ui_nevra
+    else:
+        d['_nevra'] = '%s-%s-%s.%s' % (po.name, po.version, po.release, po.arch)
+        
+    
+
+    return d
+    
+def list_stuff(my, stuff):
+    # FIXME - there are potential tracebacks that could occur here
+    # need some more catching for them so we can see what happened
+    if stuff == 'installed':
+        return [ pkg_to_dict(po) for po in my.rpmdb ]
+    elif stuff == 'updates':
+        return [ pkg_to_dict(po) for 
+                po in my.doPackageLists(pkgnarrow='updates').updates ]
+    elif stuff == 'available':
+        return [ pkg_to_dict(po) for po in my.pkgSack ]
+    elif stuff == 'repos':
+        r = []
+        for repo in my.repos.repos.values():
+            t = {}
+            s = 'disabled'
+            if repo.enabled:
+                s = 'enabled'
+            t[repo.id] = s
+            r.append(t)
+        
+        return r
+    else:
+        e,m,u = my.rpmdb.matchPackageNames([stuff])
+        p = e + m
+        e,m,u = my.pkgSack.matchPackageNames([stuff])
+        p = p + e + m
+        return [ pkg_to_dict(po) for po in p ]
+
+def run_yum(command):
+    try:
+        cmd = subprocess.Popen(command, shell=True, 
+            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        out, err = cmd.communicate()
+    except (OSError, IOError), e:
+        rc = 1
+        err = str(e)
+        out = ''
+    except:
+        rc = 1
+        err = traceback.format_exc()
+        out = ''
+
+        if out is None:
+           out = ''
+        if err is None:
+           err = ''
+    else:
+        rc = cmd.returncode
+        
+    return rc, out, err
+
+def ensure(my, state, pkgspec):
+    yumconf = my.conf.config_file_path
+    res = {}
+    if state == 'installed':
+        # check if pkgspec is installed
+        if re.search('[></=]',pkgspec):
+            try:
+                pkgs = my.returnInstalledPackagesByDep(pkgspec)
+            except yum.Errors.YumBaseError:
+                pkgs = None
+        else:
+            e,m,u = my.rpmdb.matchPackageNames([pkgspec])
+            pkgs = e +m
+        
+        if pkgs:
+            return { 'changed':False }
+    
+        # if not see if it is available
+        if re.search('[></=]',pkgspec):
+            try:
+                pkgs = my.returnPackagesByDep(pkgspec)
+            except yum.Errors.YumBaseError:
+                pkgs = None
+        else:
+            e,m,u = my.pkgSack.matchPackageNames([pkgspec])
+            pkgs = e +m
+        
+        if not pkgs:
+            msg = "No Package matching %s available" % pkgspec
+            return { 'changed':False, 'failed': True, 'msg':msg }
+        
+        my.close()
+        del my
+        cmd = "yum -c %s -d1 -y install '%s'" % (yumconf, pkgspec)
+        rc, out, err = run_yum(cmd)
+        # FIXME - if we did an install - go and check the rpmdb to see if it actually installed
+        # look for the pkg in rpmdb
+        # look for the pkg via obsoletes
+        if rc:
+            changed = False
+            failed = True
+        else:
+            changed = True
+            failed = False
+        
+        res['changed'] = changed
+
+        if failed:
+            res['failed'] = failed
+            res['msg'] = err
+            res['results'] = out
+        
+        return res
+        
+    if state == 'removed':
+        # check if pkgspec is installed
+        # if not return {changed: False, failed=False }
+        # if so try to remove it:
+        # if successfull {changed:True, failed=False, results=stdout, errors=stderr }
+        # if fail {'changed':False, failed='True', results=stdout, errors=stderr }
+        yumconf = my.conf.config_file_path
+        if re.search('[></=]',pkgspec):
+            try:
+                pkgs = my.returnInstalledPackagesByDep(pkgspec)
+            except yum.Errors.YumBaseError:
+                pkgs = None
+        else:
+            e,m,u = my.rpmdb.matchPackageNames([pkgspec])
+            pkgs = e +m
+
+        if not pkgs:
+            return { 'changed':False }
+    
+        my.close()
+        del my
+        cmd = "yum -c %s -d1 -y remove '%s'" % (yumconf, pkgspec)
+        rc, out, err = run_yum(cmd)
+        # FIXME if we ran the remove - check to make sure it actually removed :(
+        # look for the pkg in the rpmdb
+        if rc:
+            changed = False
+            failed = True
+        else:
+            changed = True
+            failed = False
+
+        res['changed'] = changed
+
+        if failed:
+            res['failed'] = failed
+            res['msg'] = err
+            res['results'] = out
+
+        return res
+                
+    if state == 'latest':
+        updates = my.doPackageLists(pkgnarrow='updates', patterns=[pkgspec]).updates
+        # sucks but this is for rhel5 - won't matter for rhel6 or fedora or whatnot
+        e,m,u = yum.parsePackages(updates, [pkgspec], casematch=True)
+        updates = e + m
+        avail =  my.doPackageLists(pkgnarrow='available', patterns=[pkgspec]).available
+        if not updates and not avail:
+            if not my.doPackageLists(pkgnarrow='installed', patterns=[pkgspec]).installed:
+                msg = "No Package matching '%s' found available, installed or updated" % pkgspec
+                return { 'changed':False, 'failed':True, 'msg': msg }
+            return { 'changed':False,}
+
+        # we have something in updates or available
+        if not updates:
+            cmd = "yum -c %s -d1 -y install '%s'" % (yumconf, pkgspec)
+        else:
+            cmd = "yum -c %s -d1 -y update '%s'" % (yumconf, pkgspec)
+        rc, out, err = run_yum(cmd)
+
+        # FIXME if it is - update it and check to see if it applied
+        # check to see if there is no longer an update available for the pkgspec
+        if rc:
+            changed = False
+            failed = True
+        else:
+            changed = True
+            failed = False
+
+
+        res['changed'] = changed
+
+        if failed:
+            res['failed'] = failed
+            res['msg'] = err
+            res['results'] = out
+
+        return res
+        
+    return {'changed': False,
+            'failed': True,
+            'results':'',
+            'errors': 'Ensure state %r unknown' % state }
+
+
+def update(args):
+    #generic update routine 
+    pass
+
+def remove_only(pkgspec):
+    # remove this pkg and only this pkg - fail if it will require more to remove
+    pass
+
+def main():
+    # state=installed pkg=pkgspec
+    # state=removed pkg=pkgspec
+    # state=latest pkg=pkgspec
+    #
+    # informational commands:
+    #   list=installed
+    #   list=updates
+    #   list=available
+    #   list=repos
+    #   list=pkgspec
+    
+    if len(sys.argv) == 1:
+        msg = "the yum module requires arguments (-a)"
+        return 1, msg
+
+    argfile = sys.argv[1]
+    if not os.path.exists(argfile):
+        msg = "Argument file not found"
+        return 1, msg
+
+    args = open(argfile, 'r').read()
+    items = shlex.split(args)
+    syslog.openlog('ansible-%s' % os.path.basename(__file__))
+    syslog.syslog(syslog.LOG_NOTICE, 'Invoked with %s' % args)
+
+    if not len(items):
+        msg = "the yum module requires arguments (-a)"
+        return 1, msg
+
+    # if nothing else changes - it fails
+    results = { 'changed':False, 
+                'failed':True, 
+                'results':'', 
+                'errors':'', 
+                'msg':args }
+    params = {}
+    for x in items:
+        try:
+            (k, v) = x.split("=", 1)
+        except ValueError:
+            msg = "invalid arguments: %s" % args
+            return 1, msg
+            
+        params[k] = v
+
+    if 'conf_file' not in params:
+        params['conf_file'] = None
+
+    
+    if 'list' in params:
+        try:
+            my = yum_base(conf_file=params['conf_file'], cachedir=True)
+            results = dict(results=list_stuff(my, params['list']))
+        except Exception, e:
+            return 1, str(e)
+            
+    else:
+        pkg = params.get('pkg', params.get('package', params.get('name', None))) 
+        if 'pkg' is None:
+            results['msg'] = "No pkg specified"
+        else:
+            try:
+                my = yum_base(conf_file=params['conf_file'], cachedir=True)        
+                state = params.get('state', 'installed')
+                results = ensure(my, state, pkg)
+            except Exception, e:
+                return 1, str(e)
+
+    print json.dumps(results)
+    return 0, None
+
+if __name__ == "__main__":
+    rc, msg = main()
+    if rc != 0: # something went wrong emit the msg
+        print json.dumps({
+            "failed" : rc,
+            "msg" : msg
+        })   
+        sys.exit(rc)
+
+    

packaging/arch/PKGBUILD

@@ -0,0 +1,44 @@
+#Maintainer: Michel Blanc <mblanc@erasme.org>
+pkgname=ansible-git
+pkgver=20120419
+pkgrel=1
+pkgdesc="A radically simple deployment, model-driven configuration management, and command execution framework"
+arch=('any')
+url="https://github.com/ansible/ansible"
+license=('GPL3')
+depends=('python2' 'python2-yaml' 'python-paramiko>=1.7.7' 'python2-jinja' 'python-simplejson')
+makedepends=('git' 'asciidoc' 'fakeroot')
+
+_gitroot="https://github.com/ansible/ansible"
+_gitname="ansible"
+
+build() {
+  cd "$srcdir"
+  msg "Connecting to GIT server...."
+
+  if [ -d $_gitname ] ; then
+    cd $_gitname && git pull origin
+    msg "The local files are updated."
+  else
+    git clone $_gitroot $_gitname
+  fi
+
+  msg "GIT checkout done or server timeout"
+
+  cd "$srcdir/$_gitname"
+  make
+}
+
+package() {
+  cd "$srcdir/$_gitname"
+
+  mkdir -p ${pkgdir}/usr/share/ansible
+  cp ./library/* ${pkgdir}/usr/share/ansible/
+  python setup.py install -O1 --root=${pkgdir}
+
+  install -D docs/man/man1/ansible.1 ${pkgdir}/usr/share/man/man1/ansible.1
+  install -D docs/man/man1/ansible-playbook.1 ${pkgdir}/usr/share/man/man1/ansible-playbook.1
+
+  gzip -9 ${pkgdir}/usr/share/man/man1/ansible.1
+  gzip -9 ${pkgdir}/usr/share/man/man1/ansible-playbook.1
+}

packaging/debian/README.txt

@@ -0,0 +1,11 @@
+I have added a debian folder for use in building a .deb file for ansible. From the ansible directory you can run the following command to construct a debian package of ansible.
+
+~/ansible$ dpkg-buildpackage -us -uc -rfakeroot
+
+The debian package files will be placed in the ../ directory and can be installed with the following command:
+~/$ sudo dpkg -i .deb
+
+Dpkg -i doesn't resolve dependencies, so if the previous command fails because of dependencies, you will need to run the following to install the dependencies (if needed) and then re-run the dpkg -i command to install the package:
+$ sudo apt-get -f install
+
+--Henry Graham

packaging/debian/ansible.dirs

@@ -0,0 +1,3 @@
+etc/ansible
+usr/lib/python2.7/site-packages
+usr/share/ansible

packaging/debian/ansible.install

@@ -0,0 +1,5 @@
+examples/hosts etc/ansible
+library/* usr/share/ansible
+docs/man/man1/ansible.1 usr/share/man/man1
+docs/man/man1/ansible-playbook.1 usr/share/man/man1
+bin/* usr/bin

packaging/debian/changelog

@@ -0,0 +1,11 @@
+ansible (0.3) debian; urgency=low
+
+  * 0.3 update
+
+ -- Michael DeHaan <michael.dehaan@gmail.com> Mon, 23 Apr 2012 11:08:00 -0400
+
+ansible (0.0.2) debian; urgency=low
+
+  * Initial Release
+
+ -- Henry Graham (hzgraham) <Henry.Graham@mail.wvu.edu>  Tue, 17 Apr 2012 17:17:01 -0400

packaging/debian/compat

@@ -0,0 +1 @@
+5

packaging/debian/control

@@ -0,0 +1,13 @@
+Source: ansible
+Section: admin
+Priority: optional
+Maintainer: Henry Graham (hzgraham) <Henry.Graham@mail.wvu.edu>
+Build-Depends: cdbs, debhelper (>= 5.0.0)
+Standards-Version: 3.9.1
+Homepage: http://ansible.github.com/
+
+Package: ansible
+Architecture: all
+Depends: python, python-support (>= 0.90), python-jinja2, python-yaml, python-paramiko
+Description: Ansible Application
+ Ansible is a extra-simple tool/API for doing 'parallel remote things' over SSH executing commands, running "modules", or executing larger 'playbooks' that can serve as a configuration management or deployment system.

packaging/debian/copyright

@@ -0,0 +1,26 @@
+This package was debianized by Henry Graham (hzgraham) <Henry.Graham@mail.wvu.edu> on
+Tue, 17 Apr 2012 12:19:47 -0400.
+
+It was downloaded from https://github.com/ansible/ansible.git
+
+Copyright: Henry Graham (hzgraham) <Henry.Graham@mail.wvu.edu>
+
+License:
+
+   This package is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; version 2 dated June, 1991.
+
+   This package is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this package; if not, write to the Free Software
+   Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301,
+   USA.
+
+On Debian systems, the complete text of the GNU General
+Public License can be found in `/usr/share/common-licenses/GPL'.
+

packaging/debian/docs

@@ -0,0 +1 @@
+README.md

packaging/debian/pycompat

@@ -0,0 +1 @@
+2

packaging/debian/rules

@@ -0,0 +1,6 @@
+#!/usr/bin/make -f
+# -- makefile --
+
+include /usr/share/cdbs/1/rules/debhelper.mk
+DEB_PYTHON_SYSTEM = pysupport
+include /usr/share/cdbs/1/class/python-distutils.mk

packaging/gentoo/README.md

@@ -0,0 +1,3 @@
+Gentoo ebuilds are available here:
+
+https://github.com/uu/ubuilds

packaging/rpm/ansible.spec

@@ -0,0 +1,68 @@
+%if 0%{?rhel} <= 5
+%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
+BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot
+%endif
+
+Name: ansible
+Release: 1%{?dist}
+Summary: Minimal SSH command and control
+Version: 0.4
+
+Group: Development/Libraries
+License: GPLv3
+Source0: https://github.com/downloads/ansible/ansible/%{name}-%{version}.tar.gz
+Url: http://ansible.github.com
+
+BuildArch: noarch
+BuildRequires: python2-devel
+
+Requires: PyYAML
+Requires: python-paramiko
+Requires: python-jinja2
+
+%description
+
+Ansible is a radically simple model-driven configuration management,
+multi-node deployment, and remote task execution system. Ansible works
+over SSH and does not require any software or daemons to be installed
+on remote nodes. Extension modules can be written in any language and
+are transferred to managed machines automatically.
+
+
+%prep
+%setup -q
+
+%build
+%{__python} setup.py build
+
+%install
+%{__python} setup.py install -O1 --root=$RPM_BUILD_ROOT
+mkdir -p $RPM_BUILD_ROOT/etc/ansible/
+cp examples/hosts $RPM_BUILD_ROOT/etc/ansible/
+mkdir -p $RPM_BUILD_ROOT/%{_mandir}/man1/
+cp -v docs/man/man1/*.1 $RPM_BUILD_ROOT/%{_mandir}/man1/
+mkdir -p $RPM_BUILD_ROOT/%{_datadir}/ansible
+cp -v library/* $RPM_BUILD_ROOT/%{_datadir}/ansible/
+
+%clean
+rm -rf $RPM_BUILD_ROOT
+
+%files
+%defattr(-,root,root)
+%{python_sitelib}/ansible*
+%{_bindir}/ansible*
+%{_datadir}/ansible
+%config(noreplace) %{_sysconfdir}/ansible
+%doc README.md PKG-INFO
+%doc %{_mandir}/man1/ansible*
+
+
+%changelog
+* Mon Apr 23 2012 Michael DeHaan <mdehaan@redhat.com> - 0.3-1
+- Release of 0.3
+
+* Tue Apr  3 2012 John Eckersberg <jeckersb@redhat.com> - 0.0.2-1
+- Release of 0.0.2
+
+* Sat Mar 10 2012  <tbielawa@redhat.com> - 0.0.1-1
+- Release of 0.0.1

setup.py

@@ -1,34 +1,28 @@
 #!/usr/bin/env python
 
+# NOTE: setup.py does NOT install the contents of the library dir
+# for you, you should go through "make install" or "make RPMs" 
+# for that, or manually copy modules over.
+
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath('lib'))
+from ansible import __version__, __author__
 from distutils.core import setup
 
 setup(name='ansible',
-      version='1.0',
+      version=__version__,
       description='Minimal SSH command and control',
-      author='Michael DeHaan',
+      author=__author__,
       author_email='michael.dehaan@gmail.com',
-      url='http://github.com/mpdehaan/ansible/',
+      url='http://ansible.github.com/',
       license='GPLv3',
-      package_dir = { 'ansible' : 'lib/ansible' },
+      install_requires=['paramiko', 'jinja2', "PyYAML"],
+      package_dir={ 'ansible': 'lib/ansible' },
       packages=[
          'ansible',
       ],
-      data_files=[ 
-         ('/usr/share/ansible', [ 
-             'library/ping',
-             'library/command',
-             'library/facter',
-             'library/ohai',
-             'library/copy',
-             'library/setup',
-             'library/service',
-             'library/template',
-             'library/git',
-         ]),
-         ('/usr/share/man/man1', [
-                'docs/man/man1/ansible.1'
-         ]),
-      ],
       scripts=[
          'bin/ansible',
          'bin/ansible-playbook'