Commit Graph

1977 Commits (d2c7539ae8907d840a22d32404f625ce9893bc1e)

Author SHA1 Message Date
Brian Coca 6b46cc5c73 added new 'order' directive to sort hosts in play
fixes #10964
2017-03-16 12:13:29 -04:00
Brian Coca dd8d699981 namespace facts (#18445)
* namespace facts

always namespace facts, make the polluting of 'main' conditional on config

* updated to 2.4

* Update intro_configuration.rst
2017-03-15 17:12:16 -07:00
Brian Coca 0c44959800 added new tests any and all (#22665)
* added new tests any and all

* updated code names
2017-03-15 18:09:25 -04:00
Brian Coca 7b0f765a57 made error slightly more neutral in case of 'local' 2017-03-15 15:32:30 -04:00
Matt Davis ce56da69b2 make windows async ... async (#22624)
Fixes #22575 - issue under new exec wrapper where unconstrained handle inheritance (for stdin) caused WinRM to block on breakaway processes. Uses explicit handle inheritance to ensure that only stdin read handle gets inherited. Adds test to ensure that async is actually async.
2017-03-14 16:37:55 -07:00
Matt Davis 07674a8446 prevent winrm Add-Type debug noise from polluting stderr (#22583) 2017-03-13 14:47:06 -07:00
Ganesh Nalawade 8b6d786cff Fix junos prompt issue (#22536)
Need to handle junos shell prompt
that end's with '%'
2017-03-13 08:11:38 -04:00
Peter Sprygada 3f0f7c4f4e updates junos shared lib and action handler (#22541)
* removes cli functions from shared lib
* adds cli functions to junos_netconf module
* statically pins junos_netconf to cli transport
* all other modules use netconf transport
* adds command rpc function to junos shared
2017-03-12 08:37:45 -05:00
Peter Sprygada 1825406e1e Junos fixes (#22423)
* Fixes for junos_config errors

* Check transport settings for core Junos

* Don't pop from the same list you iterate over

* use of persistent connections are now explicitly enabled in junos

* modules must now explicitly enable persistent connections
* adds rpc support to junos_command

fixes #22166
2017-03-11 10:26:42 -06:00
Peter Sprygada 17fc6832ca fixes mapping nxapi arguments for nxos action plugin (#22491)
The nxos action plugin does not properly map task arguments when using
provider.  This patch will fix the problem and properly map the
arguments
2017-03-11 09:46:15 -06:00
Peter Sprygada 027f5955cc correctly maps task arguments for eapi transport (#22490)
The provider arguments where not correctly being mapped in the action
plugin for eapi transport.  This will now correctly map existing
arguments as well as set the correct defaults for values.

fixes #22442
2017-03-11 09:45:55 -06:00
Ricardo Carrillo Cruz 9173a7727c Fix issue on nxos modules when transport is passed via provider (#22488)
The nxos modules read the task level transport variable, thus if
the user pass it via provider the all fail with an UnboundLocalError.

Fixes #22355
2017-03-10 07:32:24 -07:00
Matt Clay 0b250016b0 Fix PEP 8 issue. 2017-03-08 10:05:19 -08:00
Brian Coca 406505bfe5 fixes cache file modules with bad path permissions
now module will fail to load and report an error (turned into warning as this should not be fatal)
fixes #13093
2017-03-08 10:58:53 -05:00
Matt Martz 1fe67f9f43 Extend SSH Retry to put_file and fetch_file (#20187)
* Move retry logic into _ssh_retry decorator, and apply to exec_command, put_file and fetch_file

* Update tests to reflect change

* Move _ssh_retry to _run, and update tests to reflect

* piped should use exec_command instead of removed _exec_command

* Rework tests to support selectors instead of select.select
2017-03-02 16:56:29 -06:00
Peter Sprygada 7b2c013e5a returns xml as a display option to junos_command (#21833)
The display option xml as accidentially removed from the display
argument.  This patch adds xml back as an option.

fixes #21823
2017-03-02 16:05:20 -06:00
Matt Davis 42ccf9ca98 tweak transport_tests to avoid logic duplication (#22158) 2017-03-01 15:54:27 -08:00
Dag Wieers 52959ebdc1 wait_for_connection: Wait for system to become reachable (#20011)
* WIP: wait_for_connection: Wait for system to be reachable

This action plugin allows to check when a system is back online and
usable by Ansible.

As an example, when doing a SysPrep and running Enable-WinRM.ps1, it
takes between 10 to 20 seconds between the WinRM TCP port to open, and
it actually being able to server Ansible requests. This time is variable
and depends on the boot process.

Current implementation is specific for Windows (WinRM) only, this will
be fixed shortly.

This fixes #19998

* Support other transport types

* Various improvements

- Fix reported typo
- Add transport_test support in accelerate plugin
- Ensure port is an integer

* Improve examples

* Small fixes

- Use correct ConfigureRemotingForAnsible.ps1 script name
- Only use win_ping when remote shell is known to be Powershell
- Add integration tests to CI framework
2017-03-01 11:00:49 -08:00
Nathaniel Case d9d2e6deb6 Junos provider readd (#21869)
* Restore `provider` to junos_*

Fixes #21824
Fixes #21824
Fixes #21827

* Fix `confirm_timeout` related errors
* Fix glaring issues with _junos_template
2017-03-01 12:39:44 -05:00
Ricardo Carrillo Cruz 029ccf9aa0 Check right CLI context is set on iosxr action plugin (#22108)
Fixes #21994
2017-03-01 10:44:12 +00:00
Brian Coca bfb3467d52 more set/list stuff 2017-02-28 10:44:20 -05:00
Matt Davis 8265860f07 suppress PS "unapproved verb" warning (#22018) 2017-02-27 14:29:43 -08:00
Brian Coca b3abab1bd5 Fact proccessing fixes (#22003)
* clarify facts assignment for several corner cases

run_once/delegate_facts:
 now delegate_facts > run_once, previously run_once always published facts to all hosts in play

include_vars/delegate_to:
  now include_vars allows to delegate to a specific host

also fix task_vars exception in delegate_facts/loop as var was removed

fixes #15365

* removed unused loop_var
2017-02-27 16:22:30 -05:00
Brian Coca b3251c9585 Exceptit (#21864)
* more centralized exception handling

* only remove when verboxse
2017-02-27 13:29:40 -05:00
Nathaniel Case 6d9fb4d012 Network action plugin misusing display.debug (#21995)
* The correct answer was in fact display.vvvv
2017-02-27 12:34:31 -05:00
Paul Arthur 9c72d478ec hashi_vault: fix token logic
The token should not be set and checked twice, especially when the
second time overrides a previously set token.
2017-02-27 09:36:33 -05:00
Peter Sprygada 0cb2019293 roll up of bug fixs for nxos_evpn_global (#21961)
* updates nxos_evpn_global module
* adds integration test cases
* adds unit test cases
2017-02-26 06:45:26 -05:00
Nathaniel Case 9b9ed59d98 Vyos config fixes (#21862)
* Fix vyos signatures to match new versions

* Fix test cases referring to 'updates' instead of 'commands'

* I think this is an artifact of `connection: network_cli`?
2017-02-25 21:14:40 -05:00
Nathaniel Case ca21d09483 Fix a few errors (#21854) 2017-02-25 21:12:49 -05:00
Peter Sprygada f9b108e319 fixes issue that caused eapi transport setting to get lost (#21919)
The eos action didn't properly set provider transport argument for the
module.  This patch fixes that problem
2017-02-25 15:20:32 -05:00
Dag Wieers 98934939af win_copy: Add force parameter and check-mode support (#20405)
* win_copy: Add force parameter and check-mode support

The rationale behind this is that if you're working with +3GB files,
creating the checksum takes a lot of time, which we can avoid by simply
testing if the file exists.

I also took the liberty to put the various parameters together. It
probably takes a (neglible) performance hit but makes the code a bit
easier to inspect/work with, as its closer to all other windows modules.

On a normal run, the action plugin does a local checksum of the source
and a remote checksum of the destination. And afterwards, the module
will do another remote checksum of the copied source, a remote checksum
of the original destination, and another remote checksum of the copied
destination.

On a very huge file (think 4GB) that means 5x reading the complete file
(if you have a large cache you may get away with it, otherwise you're
doomed !).

This patch will ensure with `force: no` that not checksums are being
performed.

* Moving presence check before remote checksum

* Adapted to wishes

* Even more performance improvements
2017-02-24 18:10:09 -08:00
Matt Martz 305c88700d PEP8 E712 fixes (#21916) 2017-02-24 16:49:43 -06:00
Adrian Likins 8830cde28d Fix 'task name is not templated in retry callback' (add task_name property to TaskResult) (#21214)
Fix 'task name is not templated in retry callback'

Add a task_name property to TaskResult that knows to
check in TaskResult._task_fields.

Add integration test for v2_retry_runner callback

Fixes #18236
2017-02-24 12:33:24 -05:00
Dag Wieers cfb1f72d1c Improve winrm import error message (#20267)
The requests python module is needed, however it is not a dependency of
the python-winrm package. The python-winrm package does require
python-requests_ntlm, which does not seem to pull python-requests.

So for the time being (until Red Hat fixes their package) give a more
informative error message.
2017-02-23 23:42:23 -08:00
Peter Sprygada ab0f992b3a fixes import statement in nxos_facts module (#21834)
trying to import from the wrong shared lib, this patch changes
network_common to connection.  Also fixes a bug found in the nxos
action plugin to detect transport

fixes #21829
2017-02-23 13:00:34 +00:00
Peter Sprygada 0e7ebf6391 fixes issue with setting transport to nxapi (#21811)
The nxos action handler did not map the nxapi value to the provider
argument properly.  There as an additional fix in the nxos shared lib
to properly detect nxapi being set
2017-02-22 22:07:33 -05:00
jctanner 8c521655e1 Fix traceback on indexing empty list (#21784)
* Fix traceback on indexing empty list
* Make it more 'idiomatic'
* Forgot the index
2017-02-22 16:14:39 -05:00
Peter Sprygada f8157d579a fixes error where a transport is not set for nxos (#21790)
Will now make sure transport is set to `cli`
2017-02-22 12:47:21 -05:00
Brian Coca 93ac278f4c added 'extensions' options to include_vars
This allows users to override default 'extensions read' when using the dir option
2017-02-22 10:31:22 -05:00
Toshio Kuratomi 45251f910c Make BaseFileCache into an abstractbaseclass so it's a proper interface
Push the opening and closing of files into the _load and _dump methods
so that we don't invoke the slow codec machinery without reason.
2017-02-22 10:27:29 -05:00
Toshio Kuratomi c033e5111f Graceful error message for abstract base classes in PluginLoader 2017-02-22 10:27:29 -05:00
Brian Coca 374af06cbf added pickle and yaml cache plugins
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
2017-02-22 10:27:29 -05:00
Toshio Kuratomi 2dde7f7768 Fix a case where we mixed text and bytes in the local connection plugin
Fixes #20710
2017-02-21 14:19:08 -08:00
James Cammarata 3d65482927 Stash post-validated fields of the task in the TaskResult
This allows us to have a snapshot of the fields, which we can restore
on the pre-fork side so as to avoid having to re-template fields.
2017-02-21 14:35:13 -06:00
Peter Sprygada 6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 2017-02-21 09:21:41 -05:00
Peter Sprygada 2567e8157f fixes bug where handle_prompts failed if list of prompts provided (#21711)
This will now automatically convert the prompts to a list and iterate
over the regexp looking for a prompt match before supplying the answer.
2017-02-21 08:27:33 -05:00
Peter Sprygada e4a2c804be partial revert of changes introduced in d5f7a0181b (#21688) 2017-02-20 15:37:14 -05:00
Marc Abramowitz e2e4a69425 hashi_vault: Get token from env var or file
This allows getting the Vault token from the `VAULT_TOKEN` env var or
from the file `$HOME/.vault-token`, as both of these are understood by
the Vault CLI and are a common place to put Vault tokens. This allows
avoiding hard-coding a Vault token into playbooks or having to include
lookups.

`HOME/.vault-token` is nice because a user can authenticate with the CLI
using `vault auth` and then the token will be stored in
`$HOME/.vault-token`. If we read this file, then we allow someone to do
`vault auth` "out of band" to set up Vault access.
2017-02-20 10:10:22 -05:00
René Moser 576ff0728d service: deprecate state=running (#21499)
* service: deprecate state=running

* deprecated with version 2.7
2017-02-20 09:34:52 -05:00
Peter Sprygada bd036c15e0 minor fixes to junos action (#21643)
* adds instrumentation messages
* checks cli context is not in config mode
2017-02-19 18:04:00 -05:00
Peter Sprygada ce9180b3a1 roll up of vyos action fixes (#21642)
* check cli context is not in config mode
* add display messages for more details
2017-02-19 17:37:16 -05:00
Peter Sprygada 8472ed640b fixes minor bugs in nxos action (#21641)
* checks cli context is correct
* adds display messages
2017-02-19 17:13:14 -05:00
Peter Sprygada d5f7a0181b roll up of eos fixes (#21629)
* fixes error where eos would close the cli shell
* fixes network_cli connection plugin to check before calling open_shell()
* fixes json commands being sent over eapi
2017-02-19 08:46:14 -05:00
Peter Sprygada 2f10bdf0c7 roll up of fixes in junos action plugin (#21624)
* calls open_shell() or open_session() depending on connection type
* closes shell after module completion
* adds open_session() to netconf
2017-02-18 17:20:03 -05:00
Peter Sprygada 920f9f4815 fixes nxos nxapi implementation (#21615)
* correctly maps play_context to nxapi values
* fixes bug in nxos_nxapi module detecting nxapi feature
* updates nxos shared lib provider values
* fixes missing ssh_keyfile in nxos shared lib
2017-02-18 11:20:26 -05:00
Peter Sprygada c875393916 roll up of fixes for vyos base (#21616)
* adds ssh_keyfile to vyos
* fixes play_context mapping to provider
* adds missing network_os to play_context
* executes close_shell() after module completes
2017-02-18 09:29:17 -05:00
Peter Sprygada e6dfbf63f6 fixes eos action mapping of play_context values (#21614)
* adds mapping for ssh_keyfile
* adds correct mappings for eapi
* fixes missing no_log value for provider arg
2017-02-18 09:29:06 -05:00
Brian Coca ee17b91455 Revert "add systemd-nspawn connection driver"
This reverts commit 3597ca082b.
2017-02-17 16:36:46 -05:00
Brian Coca e84713c76c Revert "Remove the --nspawn-extra-args cli option"
This reverts commit b8125ac1a6.
2017-02-17 16:35:47 -05:00
Brian Coca 2d2874fb3c Revert "Add safety checks to nspawn connection plugin"
This reverts commit 60bb677154.

this connection cannot work as is, systemd-nspawn creates instance on each command
2017-02-17 16:35:47 -05:00
Toshio Kuratomi 275be702a8 Switch from using inspect to globals() 2017-02-17 12:57:26 -08:00
Peter Sprygada 8716a5bc67 maps ios provider ssh_keyfile to play_context (#21597)
This allows the key file to be specified in the playbook for the ios
module provider argument
2017-02-17 18:56:00 +00:00
Thomas Szymanski 60bb677154 Add safety checks to nspawn connection plugin
This patch adds some checks on the path that is accessed as a container,
making sure it looks like one. It implements the connection method and
add adaptations to the modern way of writing connections for Ansible.
It also rewords docs and vars to use the nspawn terminology instead of
chroot.
2017-02-17 12:39:48 -05:00
Thomas Szymanski b8125ac1a6 Remove the --nspawn-extra-args cli option
Providing extra arguments to pass to systemd-nspawn is still possible
through the `nspawn_connection` of the `ansible.cfg` file.
2017-02-17 12:39:48 -05:00
Lars Kellogg-Stedman 3597ca082b add systemd-nspawn connection driver
This commit adds a connection driver built on top of systemd-nspawn.
This is similar to the existing `chroot` driver, except that nspawn
offers a variety of additional services. For example, it takes care of
automatically mounting `/proc` and `/sys` inside the chroot environment,
which will make a variety of tools work correctly that would otherwise
fail.

You can take advantage of other system-nspawn features to perform more
complicated tasks.  For example, on my x86_64 system I have a Raspberry
Pi disk image mounted on `/rpi`.  I can't use `chroot` with this because
the binaries contained in the image are for the wrong architecture.
However, I can use the systemd-nspawn `--bind` option to automatically
insert the appropriate qemu-arm binary into the container using an
inventory file like this:

    pi ansible_host=/rpi ansible_nspawn_extra_args='--bind /usr/bin/qemu-arm --bind /lib64'

See http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
for more information about systemd-nspawn itself.
2017-02-17 12:39:48 -05:00
Peter Sprygada a6cecef6bc clean up of terminal plugins (#21549)
* removes unneeded supports_multiplexing var
* refactors terminal_prompts_re to terminal_stdout_re
* refactors terminal_errors_re to terminal_stderr_re
* updates network_cli unit test cases
2017-02-17 10:00:23 -05:00
Ricardo Carrillo Cruz c5452eef6f Removes default kwarg in iosxr_argument_spec for timeout (#21569) 2017-02-17 15:59:24 +01:00
Peter Sprygada 4cbbed0b37 fixes issue with prompt detection in network_cli (#21574)
The network_cli plugin would return immediately if an error was
detected.  This patch will force the connection plugin to still try to
detect the current prompt even if an error is found.
2017-02-17 14:13:26 +00:00
Peter Sprygada b0abbb5f8b removes the default kwarg in ios_argument_spec for timeout (#21552)
Uses the configured timeout setting instead of the arg_spec

fixes #21520
2017-02-17 13:12:12 +01:00
Matt Davis 8527013fbe Complete rewrite of Windows exec wrapper (#21510)
* supports pipelining for faster execution
* supports become (runas), creates interactive subsession under WinRM batch logon
* supports usage of arbitrary module_utils files
* modular exec wrapper payload supports easier extension
* integrates async wrapper behavior for pipelined/become'd async
* module_utils are loaded as true Powershell modules, no more runtime modifications to module code
2017-02-17 00:09:56 -08:00
James Cammarata 7bf56ceee3 Relocate creation of Templar in process_pending_results
Moving it to after the blocks where per-item results are calculated,
as it's not used there and causes quite a performance hit being there.

Fixes #21340
2017-02-17 00:26:09 -06:00
Peter Sprygada c9f6a2b740 fixes issue with cli shell left open (#21548)
The nxos action plugin will now close the shell connection once the
module has completely run
2017-02-16 22:11:32 -05:00
Peter Sprygada 20c5a1adc1 verifies cli context for iosxr (#21550)
Checks cli context and exits config mode if needed
2017-02-16 22:11:21 -05:00
Peter Sprygada 9d4a3599b8 bug fixes and updates for eos connections (#21534)
* refactors supports_sessions to a property
* exposes supports_sessions as a toplevel function
* adds open_shell() to network_cli
* implements open_shell() in eos action plugin
2017-02-16 20:26:48 -05:00
Peter Sprygada a01288859d check ios cli context and exits config mode if there (#21544) 2017-02-16 20:26:29 -05:00
Toshio Kuratomi eeaec56ed5 Use isinstance instead of type() comparisons. isinstance is more robust 2017-02-16 16:34:43 -08:00
Peter Sprygada 02d2b753db refactors junos modules to support persistent socket connections (#21365)
* updates junos_netconf module
* updates junos_command module
* updates junos_config module
* updates _junos_template module
* adds junos_rpc module
* adds junos_user module
2017-02-16 10:53:03 -05:00
Brian Coca cc0bb54d2c the return of reset_connection
allows user to force persistent connection to close, needed for when
you want to benefit from changes applied to the current play but persistent connections
prevent them from being realized.
2017-02-16 10:49:57 -05:00
Peter Sprygada 34e6cc788f check cli context to be sure out of config mode in ios (#21493)
This change will now check the cli context after a module runs and if
the cli is still in config mode it will exit config mode.  Also fixes a
minor issue with converting list of commands to a dict

fixes #21481
2017-02-16 12:54:33 +00:00
James 87646595e3 Remove the end of the task_data.path which causes #21461 2017-02-15 17:35:16 -08:00
Michael Scherer 6fae1d2bbf Make password lookup treat /dev/null as a special case
I have from time to time a need of random password without
wanting to write them down (one example is mailman list creation,
that requires a password to be given to be sent to the list owner).

But using /dev/null do not return null, but the empty string, which
doesn't generate a password at all and so do not achieve my use case.
2017-02-15 12:15:09 -08:00
Matt Clay 09f4242ce4 Fix junit callback handling of surrogate escapes. 2017-02-15 10:51:55 -08:00
Toshio Kuratomi 99fd2328af Fix hash filter for non-ascii strings and Python3
hashlib hashes operate on byte strings.  When given a text string on
Python3, hashlib backtraces.  When given a text string on Python2,
hashlib will backtrace if the string contains non-ascii characters.
Encode the text string to utf-8 prior to hashing to avoid this problem.

Fixes #21452
2017-02-15 10:50:10 -08:00
Peter Sprygada 48b02336ab code clean of old code from network modules updates (#21469)
* removes unused code
* removes module_utils/local.py
* removes plugins/action/network.py
* removes action_handler from connection plugins
* removes code to use action_handler in task_executor
* updates action plugins to subclass from normal
2017-02-15 12:46:30 -05:00
Peter Sprygada 21d993a4b8 refactors nxos module to use persistent connections (#21470)
This completes the refactor of the nxos modules to use the persistent
connection.  It also updates all of the nxos modules to use the
new connection module and preserves use of nxapi as well.
2017-02-15 11:43:09 -05:00
Peter Sprygada eb1453a366 updates iosxr modules to support socket (#21231)
* updates all iosxr modules to support persistent socket
* adds iosxr action plugin to connect to device
* adds exec_command() to iosxr shared module
* fixes iosxr_config and iosxr_template local action
* update all unit test cases
* adds base test module for iosxr module testing
2017-02-15 10:47:02 -05:00
Peter Sprygada 381a045089 updates network local actions to check for connection=local (#21437)
* updates vyos local action
* updates ios local action
2017-02-15 11:27:54 +00:00
Peter Sprygada 8e4f0ec162 checks connection type is local and fails if not in eos (#21429) 2017-02-14 18:42:15 -05:00
Peter Sprygada e1a2c6e1d3 roll up of fixes for eos modules (#21406)
* fixes issue with load_provider() not checking for an existing key
* adds updates to eos_config results key
* lots of minor syntax fixes in eos shared module
* adds eos_argument_spec to eos_eapi

fixes #21402
2017-02-14 18:47:29 +00:00
Patrick Deelman 65cd21e9a8 passwordstore (pass) lookup module (#21314)
* passwordstore (pass) lookup module

* removed shebang
changed licensing
2017-02-14 11:12:39 -05:00
Peter Sprygada d43eac93bc ios local action creates circular reference (#21404)
When loading the params in the local action, the provider key was
inadvertently being loaded as well.  This created the circular
reference.  The load_provider() method will now check for the provider
key and skip it when encountered.

fixes #21399
2017-02-14 09:56:52 -05:00
Peter Sprygada 14b942f3fb updates eos modules to use socket (#21197)
* updates eos modules to use persistent connection socket
* removes split eos shared module and combines into one
* adds singular eos doc frag (eos_local to be removed after module updates)
* updates unit test cases
2017-02-13 20:22:10 -05:00
Peter Sprygada 9937e604f5 fixes ios_facts that was returning no values (#21357)
* adds conversion to command dict into ios shared lib
* fixes hardware commands in ios_facts
* adds network_os value to play_context in ios action plugin

fixes #21190
2017-02-13 20:20:44 -05:00
Peter Sprygada 8adb108aa9 updates vyos modules to use socket connection (#21228)
* updates all vyos modules to use socket connection
* adds vyos local action handler
* adds exec_command() to vyos
* updates vyos_config local action
* update unit test cases
* add base class for testing vyos modules
2017-02-13 10:41:22 -05:00
Peter Sprygada 7f1c43e597 updates ios modules to support persistent socket (#21258)
* updates all ios modules to support persistent socket
* adds ios action plugin to connect to device
* adds exec_command() to ios shared module
* fixes ios_config and ios_template local action
* update all unit test cases
* adds base test module for ios module testing
2017-02-13 15:22:14 +00:00
Peter Sprygada 92aa92ebd2 updates netconf to use persistent socket (#21307) 2017-02-12 09:05:23 -05:00
Toshio Kuratomi 98541b7c8b Fix to bytes surrogate and nonencodable chars (#21180)
* Add a surrogate_then_replace error strategy to keep to_bytes from tracebacking by default
* Port all code that explicitly used surrogate_or_replace to surrogate_then_replace
2017-02-09 17:13:40 -08:00
Ssawa 149dd9ca86 Update winrm to allow fetch_file to work on files in use by processes (#18174) 2017-02-09 16:43:52 -08:00
Peter Sprygada 138051540e updates the code path for network modules (#21193)
* replaces persistent connection digest with _create_control_path()
* adds _ansible_socket to _legal_inputs in basic.py
* adds connection_user to play_context
* maps remote_user to connection_user when connection is local
* maps ansible_socket in task_vars to module_args _ansible_socket if exists
2017-02-09 14:05:54 -05:00
Pilou 4e5c9c3d0d default and minimal callback: display warnings in a consistent manner (#21144)
* default/minimal callback: don't display warnings twice

* minimal callback: display warnings raised by MODULE_NO_JSON modules
2017-02-08 10:58:39 -05:00
Brian Coca 82337cf52a remove warning/deprecation keys after usage 2017-02-08 09:47:10 -05:00
Peter Sprygada d8e57cc60a starts cli if root user login is used for junos terminal (#20761)
fixes #20131
2017-02-07 10:57:44 +00:00
Brian Coca c86a17b7a0 refactoring async
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
2017-02-06 19:32:00 -05:00
Toshio Kuratomi f75ffe46db Make configparser imports python3 ready
the ConfigParser module was renamed to configparser in Python3.  Use
six.moves to import it so that the modules will function on Python3.
2017-02-03 21:15:34 -08:00
Toshio Kuratomi d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
2017-02-03 08:23:50 -08:00
Toshio Kuratomi 1df7d95cec Module utils default path (#20913)
* Make the module_utils path configurable
* Add a config value to define the path site module_utils files
* Handle module_utils that do not have source as an error
* Make an integration test for module_utils envvar working
* Add documentation for the ANSIBLE_MODULE_UTILS config option/envvar
* Add it to the sample ansible.cfg
* Add it to intro_configuration.
* Also modify intro_configuration to place envvars on equal footing with
  the config options (will need to document the envvar names in the
  future)
* Also add the ANSIBLE_LIBRARY use case from
  https://github.com/ansible/ansible/issues/15432 so we can close out
  that bug.
2017-02-02 17:48:53 -08:00
Toshio Kuratomi fee6e2953b Split on newlines when checking for prompt matches (#20945)
* Check for the prompt as a substring of the output

sudo sometimes spits out warnings to stdout before getting to the
password prompt.  Account for that when trying to match a password
prompt.

Fixes #20858
2017-02-02 14:31:11 -08:00
Toshio Kuratomi 62ba084003 Do not substitute ssh_exeuctable until we need to
We need to use ssh_executable instead of hardcoding ssh in the command
we run but we need to use "ssh" when we lookup the value of the
{command}_extra_args variable.  Do this by leaving binary as "ssh" and
only expanding when we place it into b_command.

Fixes #20862
2017-02-02 13:43:39 -08:00
Brian Coca 6ca5fb49c3 allow debug to be 'changed' (#20957)
* allow debug to be 'changed'

fixes #14430

* removed test that checked against this
2017-02-02 13:46:16 -05:00
René Moser 541a51ddf7 module_utils: implement deprecation warning for params (#20884)
* module_utils: implement deprecation warning for params

* rename deprecated_version to removed_in_version

* fix pep8 E121
2017-02-01 13:00:22 -05:00
jctanner ac78347f2b Use a -short- custom hash for controlpersist path by default (#20843)
* A method to validate and alter the ssh control path automatically.
* First tries %C to use the shortened hash
* On further failure, it removes section by section from the original path
* Fix hostname
* Implement bcoca's suggested changes
* Remove unused option
* Remove unused class var
* Use to_string to avoid unicode error
* Switch from to_text to to_bytes
* Update the example config for the new controlpath feature
2017-02-01 10:39:40 -05:00
Matt Clay cdc56afe01 PEP 8 fixes. 2017-01-31 15:18:24 -08:00
Brian Coca 16615811b4 clean up empty warnings/deprecations from modules 2017-01-31 18:13:26 -05:00
Samuel Boucher 488acc750f Feature/lookup keyring (#18379)
* Add keyring lookup

* Better error control

* Update keyring.py

* Update keyring.py

typo
2017-01-31 17:26:19 -05:00
Matt Clay 524e5d2c39 PEP 8 cleanup. 2017-01-31 10:16:59 -08:00
Toshio Kuratomi 5c38f3cea2 Combine jimi-c and bcoca's ideas and work on hooking module-utils into PluginLoader.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.

Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).

Rename snippet* variables to module_utils*

Add a small number of unittests for recursive_finder

Add a larger number of integration tests to demonstrate that
module_utils is working.

Whitelist module-style shebang in test target library dirs

Prefix module_data variable with b_ to be clear that it holds bytes data
2017-01-31 09:41:24 -08:00
Doug Bridgens 65f561e496 added ldap auth capability to hashi_vault plugin (#20244)
* added ldap authentication capability
2017-01-31 11:25:50 -05:00
Brian Coca 7e6758873c added warnings list to module and autoadd
added better way of adding warnings to return data
backwards compatible if warnings key already exists
added deprecations made iface more generic
changed to enforce type per item
added logging of warnings/deprecations
also display deprecations by default
2017-01-31 10:03:09 -05:00
Matt Clay cb76200c7d PEP 8 E111 & E114 cleanup. (#20838) 2017-01-30 15:01:47 -08:00
Dag Wieers bbaab6ee5b Show remote host in error message. (#20823)
* Show remote host in error message.

* Typos :-/
2017-01-30 11:16:13 -05:00
hloeffler 8b14935521 add META Output for -vv, better than nothing 2017-01-30 09:14:50 -05:00
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
2017-01-29 07:28:53 +00:00
Matt Clay e2c0b375d3 PEP 8 cleanup. (#20790)
* PEP 8 E115 cleanup.
* PEP 8 E131 cleanup.
2017-01-28 01:39:40 -08:00
Matt Clay d0d1158c5e PEP 8 cleanup. (#20789)
* PEP 8 E703 cleanup.
* PEP 8 E701 cleanup.
* PEP 8 E711 cleanup.
* PEP 8 W191 and E101 cleanup.
2017-01-28 00:12:11 -08:00
Matt Clay d913f69ba1 PEP 8 W291 whitespace cleanup. 2017-01-27 17:08:02 -08:00
Matt Clay 95789f3949 PEP 8 whitespace cleanup. (#20783)
* PEP 8 E271 whitespace cleanup.
* PEP 8 W293 whitespace cleanup.
* Fix whitespace issue from recent PR.
2017-01-27 15:45:23 -08:00
Matt Clay 63b1e0c277 Fix infrequent PEP 8 issues. 2017-01-27 14:06:21 -08:00
Peter Sprygada 6c89c587cc refactors eos_config module to use network_cli (#20741)
* update eos_config to use eapi exclusively and remove cli transport
* add unit test cases for eos_config
* updates action plugin to handle both eapi and network_cli connections
2017-01-27 16:23:18 -05:00
Peter Sprygada e8a00377ae additional exception handling
* catches TypeError when trying to load json data
2017-01-26 22:59:32 -05:00
Brian Coca 607ef0ce80 only ignore become for making tmpdir
(cherry picked from commit 65ee9d2e4629cbb31178b54831612846ca7b3b69)
2017-01-26 15:18:32 -05:00
Brian Coca 0a8b856102 fix improper setting become user for tmpdir
fixes #20706
2017-01-26 14:39:33 -05:00
Peter Sprygada 43d0505bde fixes issue in network_cli where _connected wasn't set (#20708)
Adds the code to set and unset the _connected propery in network_cli
on calls to open() and close()
2017-01-26 14:00:08 -05:00
Brian Coca 9e0fd313d7 fix powershell mkdtemp 2017-01-25 13:19:10 -05:00
Brian Coca 10fa2cd0ef make sure tmpdir resolvs user dirs (#20486)
* make sure tmpdir resolvs user dirs

fixes #20332
supercedes #20484

* typo fix
2017-01-25 13:09:36 -05:00
Peter Sprygada 8ddffcb1a6 new connection plugin netconf (#20636)
* adds connection plugin for creating modules that use netconf
* adds basic unit test cases for connection plugin
2017-01-25 10:15:26 -05:00
John R Barker e1b1df3cb1 Detect VyOS based on /etc/issue (support 1.0.5) (#20221)
* Detect VyOS based on /etc/issue (support 1.0.5)

VyOS didn't contain `vyos` in `/proc/version`

* Update vyos.py
2017-01-25 14:47:56 +00:00
jonathanbouvier 8257ee3e25 Fix issue with cc and bcc getting added to mail if unset (#20610) 2017-01-25 11:47:37 +00:00
Matt Davis 06353c055a winrm managed kinit (#20416) 2017-01-24 15:04:11 -08:00
Peter Sprygada 83fe2170ff updates ensure_connect decorator function
only calls _connect() if _connected is False
2017-01-24 09:52:20 -05:00
Nathaniel Case b3662fdad1 vyos_command 2.3 (#18994)
* Peter's new module_utils/vyos
* Update vyos_command
* Restore `set terminal length $ANSIBLE_VYOS_TERMINAL_LENGTH` in vyos
* vyos_command tests
* Remove provider mentions from vyos_command
* Extend get_config to take (and cache) various `show configuration` commands
2017-01-23 15:30:19 -05:00
David Barroso 61243f0d99 First release of the callback selective.py (#17309) 2017-01-23 11:54:47 -08:00
Marcos Diez 1839c8699d fix code typo (#20478) 2017-01-19 16:47:01 -08:00
Brian Coca 221771c0c9 fixed fact gathering verbosity hint
also removed redundant cleanup in normal plugin as module execution already removes _ansible_* from results
fixes #20378
2017-01-19 15:36:15 -05:00
Matt Martz 537b3b75a6 Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes #20098 (#20362)
* Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes #20098

* Address some of the requested changes

* Quoting

* Print the python path and version

* Be less explicitly verbose, rely on implicit verbosity
2017-01-19 11:39:59 -08:00
Andrew Gaffney ac51266e8f Add pipeline-ish method using dd for file transfer over SSH (#18642) 2017-01-19 12:31:14 -05:00
Brian Coca 58b6027a75 remote user should rely on context, not direct var
this can cause issues with non-ssh connections and/or ansible_user
2017-01-19 12:15:45 -05:00
Brian Coca d8cefee68c include source file in error message 2017-01-18 22:24:35 -05:00
Matt Davis 883f7ed197 ensure win_reboot warnings value is a list 2017-01-16 14:28:18 -08:00
Matt Davis c8970fc4e6 Merge branch 'win_reboot-fix' of git://github.com/dagwieers/ansible into HEAD 2017-01-16 14:11:27 -08:00
Matt Davis b773ae4291 Preserve exit code in winrm exec (#20166)
Raw winrm exec discards the exit code from external processes- this change preserves the exit code if present.
2017-01-16 11:21:48 -08:00
Peter Sprygada efda0323be fixes the timeout source in network_cli (#20302)
Timeout should be taken from the play context not the constants.  this change
updates network_cli to source timeout from the correct place
2017-01-16 09:15:46 -05:00
Peter Sprygada 2c197343f3 adds timeout check when network_cli run without persistence (#20299)
* checks if signal hander is set and sets it if not (will be set if coming
  from ansible-connection)
* will now timeout long running commands based on DEFAULT_TIMEOUT setting
2017-01-16 08:32:45 -05:00
Brian Coca 08ef0aee25 removed bad iteration from execute meta (#19958)
* removed bad iteration from execute meta

most of the tasks should not be iterated over, others needed to include unreachable hosts
fixes #19673

* corrected host var
2017-01-13 14:51:39 -05:00
Brian Coca 1880027da1 better cleaning of module return, also warn
disallow any internal _ansible_ vars and also warn when those or any on the restricted list are attempted
harden and parameterize key cleaning
2017-01-13 12:12:56 -05:00
Peter Sprygada fec773a2b4 better error handling and log messages in network_cli (#20204) 2017-01-12 19:31:35 -05:00
Brian Coca 8c6b5621f8 deal with remote_src and tmp dirs properly
fixes #20128
2017-01-12 14:10:11 -05:00
Brian Coca 74421f42e1 ansible doc does not need plugin deprecation error 2017-01-12 13:09:15 -05:00
Brian Coca e35a757ee7 fixed typo 2017-01-11 21:11:12 -05:00
Toshio Kuratomi eeebd51f21 Rename the type filter to type_debug
Because we add the names of all filters to the callable whitelist used
by safe_eval, adding a filter named type makes it so code calling "type()"
gets eval'd.  We can't think of a way to exploit this but it's
sufficiently sketchy that we're renaming it in case someone smarter than
us can think of a problem.
2017-01-11 14:48:21 -08:00
Jamie Lennox 27d218f85d Don't use rsync-path in synchronize with docker
When you become: with synchronize and docker it sets the rsync-path to
"sudo rsync" to launch rsync on the server as root. Unfortunately due to
docker exec doing stricter argument parsing than ssh this fails to
launch rsync on the server and the sync fails.

For docker though we don't need to launch rsync with sudo we can simply
docker exec -u <user> and rsync as normal to get around the problem.

Closes #20117
2017-01-11 14:01:11 -08:00
Dag Wieers 239e2ae7e9 win_reboot: survive an already scheduled shutdown
These changes ensure win_reboot can survive an already scheduled
shutdown by pre-empting it.

This fixes #19982
2017-01-11 00:07:30 +01:00
Dag Wieers b7594070b3 win_reboot: display message before rebooting (#19986)
Show a message to users when rebooting the system.

TODO: Look if this message is logged in the EventLog properly, if not,
fix that :-)
2017-01-10 13:59:39 -08:00
Adrian Likins cf7b1108ff Fix 'No module named network' net_template error (#20060) 2017-01-09 23:45:52 -05:00
Ryan Brown a2b4ad9da0 (docs) Remove/update mentions of `git submodule` in docs and error me… (#19941)
* (docs) Remove/update mentions of `git submodule` in docs and error messages.

* Remove parenthetical per @dharmabumstead
2017-01-09 12:11:39 -05:00
Marius Gedminas 0a7f2c202b Improve --diff output when files lack trailing newlines
The behavior now matches GNU diff.

Fixes #14094.

Example of output before this change:

    TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
    changed: [ranka]
    --- before: /etc/cron.hourly/mg-healthchecks-dot-io
    +++ after: /tmp/tmpOTvXTw
    @@ -1,2 +1,2 @@
     #!/bin/sh
    -curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null+curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null

after this change:

    TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
    changed: [ranka]
    --- before: /etc/cron.hourly/mg-healthchecks-dot-io
    +++ after: /tmp/tmpOTvXTw
    @@ -1,2 +1,2 @@
     #!/bin/sh
    -curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
    \ No newline at end of file
    +curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null

The added unit tests contain more examples.

This commit also takes care to avoid "no newline at EOF" warnings when
no_log is in effect, and also when modules return dicts rather than
strings.  (It also removes trailing whitespace from using json
serialization when diffing dicts, because I hate trailing whitespace in
Python source files, even if they're test files.)
2017-01-09 11:49:03 -05:00
James Cammarata a6fff93967 Fixing security bugs for CVE-2016-9587 2017-01-09 10:43:03 -06:00
Peter Sprygada 258c6ada52 refactors ios_config to use network_cli plugin (#20042)
* updates the ios_config module to use the network_cli plugin
* updates the local action plugin to derive from network
* add unit test cases for ios_config
2017-01-09 11:19:25 -05:00
Peter Sprygada cba66dfedc update the _ios_template module to use the network_cli plugin (#19933)
* updates the deprecated ios_template module to use network_cli
* adds unit test cases for ios_template
* adds check for provider argument and displays warning message
2017-01-06 20:22:17 -05:00
Rene Moser 40c6191da6 win_template: fix key errors when --diff is used 2017-01-06 11:55:54 -05:00
Brian Coca 08e0f6ada5 allow modules to set custom stats (#18946)
can be per run or per host, also aggregate or not
set_stats action plugin as reference implementation
added doc stub
display stats in calblack
made custom stats showing configurable
2017-01-05 16:38:36 -05:00
Peter Sprygada 630d10a27a updates action plugins for network_cli connection (#19849)
* net_config now subclasses action plugin network
* net_template now subclasses action plugin network

This will break existing modules until those modules have been refactored.
2017-01-04 21:52:46 -05:00
Peter Sprygada 4937dd67e4 adds new feature to network_cli (#19848)
In some cases it is desirable to have a send only function that doesn't
wait for the response from the CLI (such as reloading a device).  This
adds a new key to the command json string sendonly that will
achieve this behavior.
2017-01-03 22:33:02 -05:00
Marcos Diez 024e40d5f4 new lookup module: mongodb (#15057)
* new lookup module: mongodb lookup

* fix versionadded for MongoDB Lookup

* tests should run again

* removed use of basestring

* we don't use iteritems anymore

* run tests again

* run tests again2

* run tests again3

* run tests again4
2017-01-03 11:40:02 -05:00
Alexander Gubin fce9f4f679 shuffle filter: added optional 'seed' parameter 2017-01-03 11:08:36 -05:00
Matt Davis f96ac8bff0 fix multiple handler notifications (#19655)
Fixes #19647
Adds integration test to catch multiple handler notifications
2016-12-22 21:33:24 -08:00
James Cammarata 78d4f6bbc1 Removing print debug statements 2016-12-22 19:14:20 -06:00
Tom Booth a37bc3788b runner_on_async_failed missing arg
missing and argument from CallbackModule
2016-12-22 16:55:46 -08:00
Timur Vasyunin 5365b81d14 Fix for import 'display' in lib/ansible/plugins/lookup/filetree.py 2016-12-22 15:51:39 -08:00
slohse 27cce305e6 iocage connector that wraps around jail connector. (#17986)
Added iocage connector that extends the jail connector. Uses iocage to translate iocage tags or UUIDs/partial UUIDs to the actual jail name and then uses the jail connector for actual functionality.
2016-12-22 15:34:47 -08:00
azenk f2458140f6 Lastpass lookup plugin (#16285)
This plugin can be used with the lpass cli interface for lastpass.
[lastpass-cli](https://github.com/lastpass/lastpass-cli)

Example:
Add a lookup to your playbooks/variables somewhere:
```
some_variable: "{{ lookup('lastpass','Some Lastpass entry name or ID', field='username') }}"
```
Usage:
* start a lpass session prior to using ansible
* run ansible
* logout when finished

```
lpass login user@domain.com
ansible-playbook foo.yml
lpass logout
```
2016-12-22 14:47:33 -08:00
Sorin Sbarnea 4ebd763de0 py3 compatibility fix reported on #17038 (#19569)
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2016-12-22 11:56:46 -08:00
James Cammarata d70d279c4e Also fix default callback to use ignore_errors param
Rather than the value in the task, which may not be templated.

Related to #18289
2016-12-21 11:37:06 -06:00
James Cammarata dd0257b995 Template "original_task" fields in _process_pending_results
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.

This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.

Fixes #18289
2016-12-21 10:09:06 -06:00
Peter Sprygada f831ef692c update signature in network plugin (#19545)
Fix call to _update_module_args() to include module_name in the args
2016-12-20 07:07:42 -05:00
Dag Wieers 93cfe73a76 Report detailed error when internal remote functions fail
This is a redesign in how plugins call _remote_checksum().

- _remote_stat() has been modified to report the real error as
  AnsiblError
- Action plugin **unarchive** calls _remote_stat() directly instead of
   _remote_checksum()
- Action plugin **unarchive** also handles the exceptions directly
- Ensure get_exception() returns native text

Two other action plugins, **template** and **fetch**, also do a remote checksum.
In **template** we already call _remote_stat(), just like we now do for
unarchive, in **fetch** we do call _remote_checksum() and we make the
exact same mistake as the unarchive plugin. So that one could use a
redesign as well.

This fixes #19494

Before:
```
[dag@moria ansible.testing]$ ansible-playbook -v test137.yml
Using /home/dag/home-made/ansible.testing/ansible.cfg as config file

PLAY [localhost]
******************************************************************************************************

TASK [unarchive]
******************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"python isn't present on the system.  Unable to compute checksum"}

PLAY RECAP
******************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0
failed=1
```

After:
```
[dag@moria ansible.testing]$ ansible-playbook -v test137.yml
Using /home/dag/home-made/ansible.testing/ansible.cfg as config file

PLAY [localhost]
*************************************************************************************************************

TASK [unarchive]
*************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"Failed to get information on remote file (/tmp/): sudo: unknown user:
foobar\nsudo: unable to initialize policy plugin\n"}

PLAY RECAP
*******************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0
failed=1
```
2016-12-19 21:04:48 -05:00
Peter Sprygada c546c12b60 fixes incorrect module name for action plugin (#19535)
This fixes a bug introduced in the action plugin for setting the
module name value for logging
2016-12-19 16:54:31 -05:00
Chris Houseknecht 71685b3258 Update module_args['rsync_opts'] when connection type is docker (#19531) 2016-12-19 16:08:31 -05:00
Peter Sprygada 8f97ddd98a removes network os guess static method (#19467)
For devices that do not support mutliplexing, we cannot automatically
determine the network os.  This removes the os guess static method
from the terminal plugin.  For this devices, the network_os
value must be configured
2016-12-19 07:47:29 -05:00
Dag Wieers 39c9c6b942 Ensure that command/shell errors are displayed
This fix ensures that if there are specific module errors (in our case
the python interpreter was not found) then command and shell returns a
proper error.

It also fixes a few other imperfections that we noticed during
troubleshooting:

- Return the real RC if it were available
- Improve a dictionary evaluation using .get()
- Return an RC of -1 if it is unknown (instead of returning 0)

This fixes #18846
2016-12-16 21:46:02 -05:00
Nathaniel Case 7900319fc3 Assorted python3 fixes for network code. (#18777) 2016-12-15 16:25:03 -05:00
Peter Sprygada ad99d52679 adds feature to allow connection to set action plugin (#18762)
Connection plugin can define default action plugin to use by providing
action_handler instance variable.  This will override the default
action plugin normal
2016-12-15 15:47:39 -05:00
Peter Sprygada 0cef38cf02 adds feature to all modules to be run locally (#18763)
* adds new error AnsibleModuleExit to handle module returns
* adds new action plugin network for attaching connection to network modules
* adds new shared module local to receive connection
* splits out function to update task_args with common updates

This commit provides a mechansim for running local modules that require
a connection object for interative commands tyically implemented for
network devices.  It provides a way to locally import modules (post fork)
and run them using exception handling to exit.
2016-12-15 15:47:29 -05:00
Brian Coca f550b4541f bad/missing fact plugin is a warning (#19350) 2016-12-14 14:25:19 -05:00
Brian Coca ed933421fe correct template lookup path
now all paths get 'templates/'
2016-12-14 12:25:08 -05:00
Andrea Tartaglia 59227d8c31 Removed dict.iteritems() in several other files.
This is for py3 compatibility #18506
2016-12-14 08:01:45 -08:00
Adrian Likins 6c4f554f5a Provide slightly better msg on fact cache error (#18759)
If the configured fact_cache plugin (fact_caching config)
fails, raise a fatal error instead of failing mysteriously
later.

Fixes #18751
2016-12-13 14:34:58 -05:00
Carlos E. Garcia 0b8011436d minor spelling changes 2016-12-13 13:51:13 -05:00
Clark Boylan a65e34ce77 Fix synchronize retries (#18535)
* Fix synchronize retries

The synchronize module munges its task args on every invocation of
run(). This was problematic because the munged data was not fit for use
by a second pass of the synchronize module. Correct this by using a copy
of the task args on every invocation of run() so that the original args
are not affected.

Local testing using this playbook seems to confirm that things work as
expected:

  - hosts: all
  tasks:
    - delay: 2
      register: task_result
      retries: 1
      until: task_result.rc == 0
      synchronize:
        dest: /tmp/out
        mode: pull
        src: /tmp/nonexistent/

fixes #18281

* Update synchroncization fixture assertions

When we started operating on a copy of the task args the test assertions
were no longer asserting things about the munged state but of the
pristine state. Convert the copy of task args to a class member so that
it can be compared against later in testing and update the assertions to
check this munged copy.
* Shuffle objects around for cleaner testing

Attach the temporary args dict to the task rather than the action as
this makes updating the existing tests cleaner.
2016-12-12 13:33:30 -08:00
Chris Houseknecht 65491cc839 For docker add --blocking-io only when missing (#19171) 2016-12-10 10:51:03 -05:00
Chris Houseknecht b02491ba70 Add --blocking-io option when docker connection (#19140) 2016-12-10 09:20:35 -05:00
Peter Sprygada dc23667cc2 add back reverted change to network_cli (#18761)
This adds back the change to the network_cli plugin.  Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.

The issue was due to the addition of the decorator that will call
_connect() when there is no connection.  The _connect() method should
have been mocked in the test case.  This commit fixes the test
case as well

Change was originally reverted in c414ded69a
2016-12-05 21:42:09 -05:00
Toshio Kuratomi c414ded69a Revert "minor updates to network_cli connection plugin (#18742)"
This reverts commit 8e375913b0.

This breaks testing as the PlayContext is not mocked out for the test.
Temporarily reverting.
2016-12-05 10:54:42 -08:00
Peter Sprygada 8e375913b0 minor updates to network_cli connection plugin (#18742)
* removes superfluous timeout kwargs from open_shell()
* cleans up play_context become check
* adds check for ssh session and calls _connect() if needed
2016-12-04 22:50:01 -05:00
Peter Sprygada 8e562018ed adds py3 compat changes to network_cli plugin (#18735)
now calculates StringIO receive buffer and if received data is less than
buffer size, resets offset to 0
2016-12-03 20:05:33 -05:00
Jesse Keating 8e38f7475f Do not set docker use to None. (#18706)
The user variable defaults to None, and was being passed in as a user
named None. This was breaking rsync unless a specific user was set.

Fixes 16306
2016-12-02 10:29:24 -05:00
Benoît Allard 56dcf2cc04 with_sequence: pass AnsibleError through
The parsing methods try as hard as possible to generate meaningful error messages that are all ignored and immediately overwritten by a new AnsibleError instance. Better use the original one instead.
2016-11-30 21:26:23 -08:00
Benoît Allard 7db4ed02ee with_sequence: Fix indentation
This doesn't need to run for every parameters. Once is enough.
2016-11-30 21:13:13 -08:00
Alberto Murillo 7542dae26b Fix fetching files with scp (#18673)
Commit ec2521f intended to fix the scp command to fetch files
from a remote machine but it has src and dest swapped.

This change correctly treats src as the location in the remote machine
and dest as the location in the local machine.

Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
2016-11-30 20:10:49 -08:00
Peter Sprygada 8137c7207d adds feature to try to auto determine network_os (#18674)
This updates the network_cli connection plugin to attempt to automatically
determine the remote device os.  The device network os discovery can
be overridden by setting the ansible_network_os value.
2016-11-30 16:28:47 -05:00
Yannig 900b3ffcba Implement docker support for synchronize module. (#18145)
* Implement docker support for synchronize module.

Note : you need rsync installation on your docker container.
Have a look at https://github.com/ansible/ansible/issues/16306 for more details.
Support Ansible options for remote access.

* Give user name to docker command.
2016-11-30 15:56:27 -05:00
Virgil Dupras bf48383610 Fix regression in jinja2 include search path (#18617)
* Fix regression in jinja2 include search path

Since commit 3c39bb5, the 'ansible_search_path' variable is used to set
jinja2's search path for {% include %} directives. However, this path is
the the proper one because our templates live in 'templates' subdirs in
our search path.

This is a regression because previously, our include search path would
include the dirname of the currently interpreted file, which worked most
of the time.

fixes #18526

* Fix template lookup search path

Improve fix in commit c96c853 so that the search path contain both
template-suffixed paths as well as original paths.

ref PR #18617

* Add integration test for template lookups

Tests regression at #18526

This test fails on current devel branch and succeeds on PR #18617
2016-11-29 16:23:06 -05:00
Brian Coca ca1514cf2a unified boolean function
optimized boolean function
fixes #17815
2016-11-29 12:34:30 -05:00
Adrian Likins 83676e9b40 Fix 'cancled' misspelling in --step debug 2016-11-29 09:48:19 -05:00
Peter Sprygada 9aa8547016 adds two new plugins that use ansible-connection for persistence (#18572)
* adds new connection plugin `network_cli` which builds on paramiko
* adds new plugin `terminal` used for manipulating network_cli terminals
* adds new field to play_context `network_os` settable as ansible_network_os

This commit adds the plugins necesary to establish a persistent cli connection
to network devices of ssh.  It builds on the paramiko connection plugin
to create a shell environment that will persistent through ansible-connection.
The `newtork_cli` plugin then uses the network_os in the instance of
PlayContext to load the appropriate network OS environment plugin for
handling opening and closing of shells as well as privilege escalation.
2016-11-28 12:49:40 -05:00
Peter Sprygada 7df5a0abd0 adds config option to auto add keys when using paramiko (#18598)
* updates paramiko_ssh to auto add keys
* updates constants with new config options

This commit adds a new feature that will allow paramiko to automatically
accept and save a host ssh key.  This feature is controlled by the
`host_key_auto_add` config setting in the paramiko section.  The default
is False to maintain current functionality.  It also includes a new
setting `look_for_keys` with the default to False for maintaining current the
current setting.
2016-11-28 11:31:12 -05:00
Alberto Murillo ec2521f6af Fix ssh plugin to correctly fetch files when using scp (#18614)
Fetch module uses fetch_file() from plugin/connection/ssh.py to
retrieve files from the remote hosts which in turns uses
_file_transport_command(self, in_path, out_path, sftp_action) being
sftp_action = 'get'

When using scp rather than sftp, sftp_action variable is not used
and the scp command is formed in a way that the file is always
sent to the remote machine

This patch fixes _file_transport_command() to correctly form the scp
swaping src and dest if sftp_action is 'get'

Bug introduced at 8e47b9b
Fixes #18603

Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
2016-11-26 16:55:38 -06:00
Brian Coca 778c983ef9 ansible_playbook_python (#18530)
* ansible_playbook_python

fixes #18471

* fix tests

* removed dupe
2016-11-23 16:30:46 -05:00
Matt Clay d61b2ed0a3 Support script interpreters for async_wrapper. (#18592) 2016-11-22 16:50:08 -08:00
Toshio Kuratomi 255a5b5d75 Fix the Solaris POSIX acl fix
For setfacl on Solaris we need to specify permissions like r-x.
For chmod, we need to specify them as rx (r-x means to make the file
readable and *not* executable)
2016-11-22 12:36:43 -08:00
koralsky 4e194d71bd import reduce from six label:python3 (#18561)
* import 'reduce' from six
* import reduce in facts fix
2016-11-22 11:22:24 -08:00
Michael Noseworthy bb5d8fb476 Fix unicode handling in fixup_perms2 errorhandling (#18565)
The _fixup_perms2 method checks to see if the user that is being sudo'd
is an unprivileged user or root. If it is an unprivileged user, some
checks are done to see if becoming this user would lock the ssh user out
of temp files, among other things. If this check fails, an error prints
telling the user to check the documentation for becoming an unprivileged
user.

On some systems, the stderr prints out the unprivileged user the ssh
user was trying to become contained in smartquotes. These quotes aren't
in the ASCII range, and so when we're trying to call `str.format()` to
combine the stderr message with the error text we get a
UnicodeEncodeError as python can't coerce the smartquotes using the
system default encoding. By calling `to_native()` on the error message
we can ensure that the error message is a native string for the
`Exception` handling, as `Exception` messages need to be native strings
to avoid errors (byte strings in python2, and text strings in python3)

Fixes: #18444
2016-11-22 11:19:47 -08:00
Nathaniel Case 8d0418f7c1 net_template doesn't need BOOLEANS
Note #17815
2016-11-22 11:49:21 -05:00
Toshio Kuratomi d90638ad40 Fix setfacl for Solaris with POSIX acl support.
Tested on Linux and freebsd.

Fixes #17919
2016-11-21 13:19:41 -08:00
James Cammarata 7e2305f953 Cache dynamically included blocks for later lookup via uuid
Fixes #18357
2016-11-21 13:02:24 -06:00
James Cammarata 26ec2ecfce Adding a persistent connection utility 2016-11-21 12:35:27 -06:00
Andrea Tartaglia 62697ad77f Replaced iterkeys with 'for key in dict' for #18507 (#18538) 2016-11-21 07:18:52 -08:00
jamessewell b91d4d884d Moved the _inventory.clear_group_dict_cache() from creating a group w… (#17766)
* Moved the _inventory.clear_group_dict_cache() from creating a group which doesn't exist, to adding members to the group.

* Update __init__.py

Update to use changed: block to catch all changes for cache clear as suggested
2016-11-18 13:27:17 -06:00
Andrew Haines 200d6bdb23 Default include_role results to empty list in linear strategy plugin
Fixes #18544.

When a loop is over an empty list, the result is set to

    {'skipped_reason': u'No items in the list', 'skipped': True, 'changed': False}

which means that accessing `hr._result['results']` throws a `KeyError`.
2016-11-18 12:13:53 -05:00
Brian Coca aff5d9160a disable 'dense' callback in py26 2016-11-18 11:41:56 -05:00
James Cammarata 937d872f4b Return failed instead of raising an error when an include_vars file is missing
Fixes #18508
2016-11-18 09:28:44 -06:00
Dag Wieers b7425b59ee Adapt callback functions to changes in 13f3cbaf3b 2016-11-17 19:30:42 -05:00
Dag Wieers 0036108c59 Implement destructor to ensure newline and terminal attributes
Also fixed a typo
2016-11-17 19:30:42 -05:00
Dag Wieers 2a8914e496 Reset VT100 attributes in case we get unexpected output 2016-11-17 19:30:42 -05:00
Dag Wieers cd6f577d91 Rename ansi to the more correct name vt100 2016-11-17 19:30:42 -05:00
Dag Wieers 893dc81b5e Implement our own detailed task output
Also:
- Streamline ANSI calls
- Implement v2_playbook_on_include
2016-11-17 19:30:42 -05:00
Dag Wieers 221a4e7014 Ensure we end with a newline in normal mode 2016-11-17 19:30:42 -05:00
Dag Wieers 7148023f59 Small fix so diff output starts on a new line 2016-11-17 19:30:42 -05:00
Dag Wieers 9a507b1f07 Don't try to be smart with cmd output ! 2016-11-17 19:30:42 -05:00
Dag Wieers 84e0420389 Make everything work magically !
- Support item-loops correctly
- Support notification handlers
2016-11-17 19:30:42 -05:00
Dag Wieers 589953c79b Assorted improvements
- Better switch between *dense* and *default*
- Reimplement C.COLOR* out of necessity (help!)
- Make verbose output more dense (clean up result)
- Implement our own dumper
- Improve delegation support
2016-11-17 19:30:42 -05:00
Dag Wieers 223c0011e0 Assorted fixes
- Implement delegate support
- Implement preliminary handler support
- Implement our own recap/summary (still fails)
2016-11-17 19:30:42 -05:00
Dag Wieers 1bf7e22756 Implement support for itemized tasks 2016-11-17 19:30:42 -05:00
Dag Wieers 14cfb2b230 Preliminary support for items, WIP 2016-11-17 19:30:42 -05:00
Dag Wieers 67c57a9b6b Improve readability 2016-11-17 19:30:42 -05:00
Dag Wieers 326bb24a89 Various improvements, now supports different verbosity levels 2016-11-17 19:30:42 -05:00
Dag Wieers fe6e4f8286 New "dense" callback plugin
The goal for the "dense" output is to only show changes and failures on-screen (the Unix-way).
However, since we still want to have a sense of progress, we use terminal capabilities to display progress.

 - On screen there should only be relevant stuff
    - How far are we ? (during run, last line)
    - What issues occured
    - What changes occured
    - Diff output

 - If verbosity increases, act as default output
   So that users can easily switch to default for troubleshooting

 - Leave previous task output on screen
   - If we would clear the line at the start of a task, there would often
     be no information at all
   - We use the cursor to indicate where in the task we are.
     Output after the prompt is the output of the previous task
   - Use the same color-conventions of Ansible

This is still a work in progress.
It was released to give a glimpse of what would be possible.

The Ansible callback mechanism currently does not have all the functionality we need to do this efficiently.
2016-11-17 19:30:42 -05:00
Brano Zarnovican 96ddd29c7e random filter: added optional 'seed' parameter
implements #15621
2016-11-17 15:02:06 -08:00
Toshio Kuratomi ed00741a01 Mcsalgado's change to use shlex.quote instead of pipes.quote (#18534)
* Replace pipes.quote for shlex_quote

* More migration of pipes.quote to shlex_quote

Note that we cannot yet move module code over.  Modules have six-1.4
bundled which does not have shlex_quote.  This shouldn't be a problem as
the function is still importable from pipes.quote.  It's just that this
has become an implementation detail that makes us want to import from
shlex instead.

Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
2016-11-17 13:18:29 -08:00
Brian Coca 5d043b65d3 fix for filter fix 2016-11-17 13:41:57 -05:00
Brian Coca 7c960d440f remove rsync path from returned facts 2016-11-17 13:20:46 -05:00
Dag Wieers 1ca4add91c Performance improvement using in-operator on dicts
Just a small cleanup for the existing occurrences.

Using the in-operator for hash lookups is faster than using .keys()
http://stackoverflow.com/questions/29314269/why-do-key-in-dict-and-key-in-dict-keys-have-the-same-output
2016-11-17 12:33:04 -05:00
James Cammarata ca5b361ad8 Reworking iterator logic regarding failed states during always
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.

This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.

Fixes #17983
2016-11-16 10:21:46 -06:00
Brian Coca afaec3da82 corrected service filtered option to singular 2016-11-16 10:12:47 -05:00
Brian Coca bd70397e24 always template when called from template (#18490)
* Have template action plugin call do_template

Avoids all the magic done for 'inline templating' for ansible plays.
renamed _do_template to do_template in templar to make externally accessible.
fixes #18192

* added backwards compat as per feedback
2016-11-15 15:16:46 -05:00
Brian Coca f4391d34e4 added alias to argumetns 'args' to blacklist
this should fix https://github.com/ansible/ansible-modules-core/issues/5584
2016-11-14 14:49:30 -05:00
Chris Church 534bd12ae9 Increment changed stat for a failed task if changed. (#18014) 2016-11-14 01:23:02 -06:00
James Cammarata 4f06a86161 Alternately track listening handlers by uuid if no name is set
Fixes #17846
2016-11-13 15:24:44 -06:00
James Cammarata 5b87951d6c Don't copy the parent block of TaskIncludes when loading statically
When loading an include statically, we previously were simply doing a
copy() of the TaskInclude object, which recurses up the parents creating
a new lineage of objects. This caused problems when used inside load_list_of_blocks
as the new parent Block of the new TaskInclude was not actually in the list
of blocks being operated on. In most circumstances, this did not cause a
problem as the new parent block was a proper copy, however when used in
combination with PlaybookInclude (which copies conditionals to the list of
blocks loaded) this untracked parent was not being properly updated, leading
to tasks being run improperly.

Fixes #18206
2016-11-11 08:09:43 -06:00
Ievgen Khmelenko 619f2fd210 ansible-logstash-callback (#18282)
* ansible-logstash-callback

* GPL v3 license preamble, ImportError

* Update logstash.py
2016-11-08 11:17:05 -05:00
James Tanner 20fb74b1b1 ini lookup: add 'default' to the list of parsed keys
Fixes #18369
2016-11-08 11:07:33 -05:00
Brian Coca 5dd195b52f restore play_hosts variables to not show removed
Also adds ansible_play_hosts_all with original list of hosts the play targeted
2016-11-08 10:51:27 -05:00
Toshio Kuratomi 6a3893c518 Remove direct calls to print and cleanup imports
All display of information should go through display instead of through print.
2016-11-07 17:12:36 -05:00
Adrian Likins dd0189839e Fix bug (#18355) where encrypted inventories fail 18355 (#18373)
* Fix bug (#18355) where encrypted inventories fail

This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes

The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).

So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().

The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.

Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.

_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.

Fixes #18355
2016-11-07 10:07:26 -05:00
Andrea Tartaglia b18263cf36 ANSIBLE_SSH_CONTROL_PATH_DIR option added (#18342)
* ANSIBLE_SSH_CONTROL_PATH_DIR option added

This removes the hardcoded value ( $HOME/.ansible/cp ) from ssh.py.
User is able to change the ControlPath directory ( the one that replaces %(directory)s ).

 Fixes #18325

* Added config option in ansible.cfg
2016-11-03 15:19:59 -07:00
Brian Coca 32a7b4ce71 only validate extensions when using dir loading
fixes #18223
2016-11-03 07:30:32 -07:00
Matt Clay 0d46805979 Clean up shebangs for various files.
- Remove shebangs from:
  - ini files
  - unit tests
  - module_utils
  - plugins
  - module_docs_fragments
  - non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.

Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.

A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
2016-11-02 17:00:27 -07:00
Samuel Boucher 911a602c79 Update syslog_json.py
Add SYSLOG_FACILITY environment variable to set syslog facility
2016-11-02 10:02:56 -04:00
Matt Martz cb1e3dab0d Add 'type' filter for display the underlying python type of a variable (#18242)
* Add 'type' filter for display the underlying python type of a variable

* Update playbooks_filters.rst

Minor copyedit.
2016-10-31 13:36:24 -07:00
jasdeep-hundal 679da00236 Fix OpenSSH-related ssh process exit race
Mitigate the effects of observing the ssh process still running
after seeing an EOF on stdout when using OpenSSH with
ControlPersist, since it does not close the stderr file descriptor
in this case.
2016-10-27 15:47:24 -07:00
Brian Coca 680cade77a simplified the code by removing repeats
(cherry picked from commit 84380b0ee4029212fc1637c008e07bb9958305c3)
2016-10-26 20:49:55 -04:00
jctanner 5502da3cf8 copy: Use the local file's mode for the argument if not explicitly given. (#17780)
* Use the local file's mode to for the argument if not explicitly given.

Fixes https://github.com/ansible/ansible-modules-core/issues/1124

* Fix octal mode for py3

* Implement preserve instead of null

* Remove duplicate line

* Update comment

* Use stat module per toshia's suggestion
2016-10-24 23:57:50 -04:00
Thomas Quinot 236c923c25 Filter out internal magic and connection variables from facts returns
Fixes #15925
2016-10-24 17:27:43 -05:00
Rene Moser e69d26270f handler: notify a handler by name _and_ listen
Before we only allowed either notify by name or listen and name had precedence.
2016-10-24 10:59:05 -04:00
Adrian Likins c0331d50dc Remove callback.CallbackBase._copy_result_exclude
Nothing seems to use this now.

Was added originally added in2d11cfab92f9d26448461b4bc81f466d1910a15e
but the code that used it was removed in
e02b98274b
2016-10-23 13:36:20 +02:00
Matt Robinson 692bfa872a Make bcrypt + passlib work in password_hash filter
If hashtype for the password_hash filter is 'blowfish' and passlib is
available, hashing fails as the hash function for this is named 'bcrypt'
(and not 'blowfish_crypt').  Special case this so that the correct
function is called.
2016-10-23 01:46:05 +02:00
Ssawa 8e47b9bc70 Handle 'smart' scp_if_ssh option for fetch (#18125) 2016-10-21 09:59:56 -04:00
Toshio Kuratomi 6d9f780937 Now that we convert salt inside of do_encryptas needed, keep salt as text type until then. 2016-10-20 22:49:30 -07:00
Brian Coca 99220a5f6c properly propagate loop vars
fixes #17966
2016-10-18 15:27:16 -04:00
James Cammarata 5be2a3a9e0 Break out of linear strategy if all hosts are failed and the result is not OK
Fixes #18021
2016-10-17 16:00:04 -05:00
Brian Coca 9bdde13126 fixes to ansible_search_path
now gets basedir (no need to frontload as dwim already does that)
added comment about basedir to search_path usage to avoid dupes
2016-10-17 11:21:46 -04:00
Toshio Kuratomi f24c10c32b Fixes to handle non-ascii become passwords
Fixes for non-ascii passwords on
* both python2 and python3,
* local and paramiko_ssh (ssh tested working with these changes)
* sudo and su

Fixes #16557
2016-10-15 16:25:19 -07:00
Toshio Kuratomi efc5dac52c Fix become password using non-ascii for local connection
Fixes #18029
2016-10-15 11:26:17 -07:00
James Cammarata 6bdcb3a392 Make sure free strategy is returning proper TQM constants
Fixes #18023
2016-10-14 09:23:35 -05:00
Brian Coca b169a61c20 toggle missing handler errors/warnings via config 2016-10-13 16:54:02 -04:00
Brian Coca cdb5a222c5 restored 'results' filters
tests do not work the same, restoring old filters to keep backwards compat
tests now only implment the new normalized 'tense'
2016-10-13 13:21:40 -04:00
Bruno Rocha b06fb2022c Fix unbound method call for JSONEncoder (#17970)
* Fix unbound method call for JSONEncoder

The way it is currently it will lead to unbound method error

```python
In [1]: import json

In [2]: json.JSONEncoder.default('object_here')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-872fdacfda50> in <module>()
----> 1 json.JSONEncoder.default('object_here')

TypeError: unbound method default() must be called with JSONEncoder instance as first argument (got str instance instead)

```

But what is really wanted is to let the json module to raise the "is not serializable error" which demands a bounded instance of `JSONEncoder()`

```python
In [3]: json.JSONEncoder().default('object_here')
---------------------------------------------------------------------------
TypeError: 'object_here' is not JSON serializable 

```


BTW: I think it would try to call `.to_json` of object before raising as it is a common pattern.

* Calling JSONEncoder bounded `default` method using super()
2016-10-11 08:31:53 -07:00
Pavlo Shchelokovskyy aa1ec8af17 Make interprocess polling interval configurable (#16560)
As recently there was back-and-forth with this hardcoded value
(0.001 -> 0.01 -> 0.005), obviousely the optimal value for it depends on
Ansible usage scanario and is better to be configurable.

This patch adds a new config option in DEFAULT section,
`internal_poll_interval`, with default of 0.001 corresponding to the
value hardcoded in Ansible v2.1.
This config option is then used instead of hardcoded values where
needed.

Related GH issue: 14219
2016-10-06 14:30:20 -05:00
James Cammarata e26bce5221 Sleep briefly while waiting for pending results to reduce CPU churn 2016-10-06 08:50:17 -05:00
Brian Coca d9d7e413a5 fixed storing of cwd 2016-10-04 14:24:45 -04:00
Brian Coca 125a8d3c65 no need for warnings in first_found 2016-10-03 20:23:33 -04:00
Adrian Likins 2addc09050 cast/copy keys() to list to avoid py3 errors
In py3, dict.keys() is a view and not a copy of the
dicts keys, so attempting to delete items from the dict
while iterating over the keys results int

RuntimeError: dictionary changed size during iteration

Resolve by casting .keys() to a list() type.
2016-10-03 13:10:00 -07:00
Brian Coca 49ce0c8bac only change dir to playdir if local
fixes #17869
fixes #17770
2016-10-03 08:44:25 -04:00
Toshio Kuratomi 5bc3cb278c Remove unicode escape (#17866)
* Remove unicode-escape which is not present on python3

Alternative fix for #17305

* Enable the assemble test on python3

* Fix other problems with assemble on python3
2016-10-02 22:12:51 -07:00
Guido Günther 14a9bd6a1c Add foreman callback plugin (#17141) 2016-10-03 00:12:12 -04:00
Toshio Kuratomi f72b123584 On python3, subprocess needs another arg to pass extra file descriptors 2016-10-02 15:29:54 -07:00
Toshio Kuratomi 64c446d9c0 Normalize text and byte type in the ssh plugin helper method that builds up an ssh command (#17860)
Mostly cleanups to make the code more efficient, more pythonic, and obey
the unicode sandwich strategy more but also Fixes #17832
2016-10-02 14:55:55 -07:00
Matt Davis aa0ad073b8 bugfixes to JSON junk filter, added unit/integration tests to exercise (#17834) 2016-10-02 08:03:42 -07:00
Senthil Kumar Ganesan 7e0074263d Remove the dellosX_template module (#17836) 2016-10-01 14:31:51 -04:00
Brian Coca 24e81ddd1c add ssh error message to failure 2016-09-30 17:58:19 -04:00
Makc c6fa701ab1 jabber callback (#16981)
* jabber callback

* delete empty lines

* grammar fix

* fix: do not print error when jabber callback disabled

* change callback version 1 -> 2
2016-09-30 12:06:53 -04:00
jctanner fff161f2f6 Smart mode for sftp+scp (#17813)
If the sftp fails, roll over to scp by default. This saves users
from having to know about the scp_if_ssh method when sftp is broken
on the remote host.
2016-09-29 17:44:54 -04:00
Matt Clay 670536f685 Revert "Clear the plugin path cache when adding new directories" (#17785)
This reverts commit 5a57313dd7.
2016-09-27 16:17:35 -07:00
James Cammarata 5a57313dd7 Clear the plugin path cache when adding new directories 2016-09-27 14:48:35 -05:00
Brian Coca 3550f73837 removed deprecated first available file from tasks (#17643)
https://docs.ansible.com/ansible/porting_guide_2.0.html#deprecated
2016-09-27 12:31:46 -04:00
Brian Coca f63b8878fd fix delegate_facts (#17736)
code was misplaced when we changed result processing
also cleaned up import and clarified item/label
fixes #17582
2016-09-27 11:51:16 -04:00
peter.jang 0d94d39689 fix for rsync protocol support (#16756) 2016-09-23 16:12:07 -04:00
James Cammarata 6666d13654 Add max_passes to the debug strategy override of _process_pending_results
Fixes #17520
2016-09-22 10:49:30 -05:00
jctanner a2547db5b5 ini lookup: add 'type' to the list of known arguments (#17707)
Fixes #16556
2016-09-22 11:32:14 -04:00
Brian Coca 1e4e188318 changed missing file error to warning for lookups (#16800)
* changed missing file error to warning for lookups

* changed plugins that expected exception

warning will still be displayed, they now work with None value
2016-09-20 15:05:11 -04:00
James Cammarata 4dc2bf4815 Take ITERATING_ALWAYS into account when setting failed state in PlayIterator
Fixes #15963
2016-09-20 11:11:52 -05:00
Timothy Appnel 1975a545bd Implements verbose always to assert action plugin module (#17654) 2016-09-20 10:52:21 -05:00
James Cammarata 1b54d3b6dc Merge branch 'threaded_receiver' into devel 2016-09-20 09:18:26 -05:00
James Cammarata 1d2e1a5648 Do not set run_once for meta noop tasks
Fixes #17581
2016-09-19 14:27:02 -05:00
Toshio Kuratomi 5e9a8d9202 Fix password lookup py3 plus alikins unittest additions refactoring (#17626)
* Improve unit testing of 'password' lookup

The tests showed some UnicodeErrors for the
cases where the 'chars' param include unicode,
causing the 'getattr(string, c, c)' to fail.
So the candidate char generation code try/excepts
UnicodeErrors there now.

Some refactoring of the password.py module to make
it easier to test, and some new tests that cover more
of the password and salt generation.

* More refactoring and fixes.

* manual merge of text enc fixes from pr17475

* moving methods to module scope

* more refactoring

* A few more text encoding fixes/merges

* remove now unused code

* Add test cases and data for _gen_candidate_chars

* more test coverage for password lookup

* wip

* More text encoding fixes and test coverage

* cleanups

* reenable text_type assert

* Remove unneeded conditional in _random_password

* Add docstring for _gen_candidate_chars

* remove redundant to_text and list comphenesion

* Move set of 'chars' default in _random_password

on py2, C.DEFAULT_PASSWORD_CHARS is a regular str
type, so the assert here fails. Move setting the
default into the method and to_text(DEFAULT_PASSWORD_CHARS)
if it's needed.

* combine _random_password and _gen_password

* s/_create_password_file/_create_password_file_dir

* native strings for exception msgs

* move password to_text to _read_password_file

* move to_bytes(content) to _write_password_file

* add more test assertions about genned pw's

* Some cleanups to alikins and abadger's password lookup refactoring:

* Make DEFAULT_PASSWORD_CHARS into a text string in constants.py
  - Move this into the nonconfigurable section of constants.
* Make utils.encrypt.do_encrypt() return a text string because all the
  hashes in passlib should be returning ascii-only strings and they are
  text strings in python3.
* Make the split up of functions more sane:
   - Don't split such that conditionals have to occur in two separate functions.
   - Don't go overboard: Good to split file system manipulation from parsing
     but we don't need to do every file manipulation in a separate
     function.
  - Don't split so that creation of the password store happens in two
    parts.
  - Don't split in such a way that no decisions are made in run.
* Organize functions by when it gets called from run().
* Run all potential characters through the gen_candidate_chars function
  because it does both normalization and validation.
* docstrings for functions
* Change when we store salt slightly.  Store it whenever it was already
  present in the file as well as when encrypt is requested.  This will
  head of potential idempotence bugs where a user has two playbook tasks
  using the same password and in one they need it encrypted but in the
  other they need it plaintext.
* Reorganize tests to follow the order of the functions so it's easier
  to figure out if/where a function has been tested.
* Add tests for the functions that read and write the password file.
* Add tests of run() when the password has already been created.
* Test coverage currently at 100%
2016-09-19 11:37:57 -07:00
Michael Scherer 968101f611 Fix #17624 (#17627)
Even if the size of the network is 0, it should have a netmask.
2016-09-19 11:58:20 -04:00
Peter Sprygada ff52e01a11 Merge pull request #17625 from skg-net/dellos6_template
Added support for template module for Dell Networking OS6 devices
2016-09-18 22:26:20 -04:00
Joshua Elsasser fa5f8a7543 Fix slaac filter (#17551) 2016-09-17 23:52:10 +02:00
Senthil Kumar Ganesan 463349c0d1 Added support template module for Dell Networking OS6 devices 2016-09-17 10:55:40 -07:00
James Cammarata 5a57c66e3c Moving result reading to a background thread 2016-09-17 08:12:52 -05:00
Peter Sprygada 2a7c87a3b7 fixes AttributeError: 'Task' object has no attribute '_block'
This addresses a problem when *_config or *_template network modules are
being used in roles.  The module will error with the above message.  This
fixes that problem

fixed ansible/ansible-modules-core#4840
2016-09-16 22:04:22 -04:00
Toshio Kuratomi ce4c0fd644 Use text strings when using display() to avoid UnicodeError tracebacks 2016-09-16 16:24:19 -07:00
Senthil Kumar Ganesan bb9ed50441 Renamed the Modules from dnos* -> dellos*, updated copyright, removed… (#17617)
* Renamed the Modules from dnos* -> dellos*, updated copyright, removed dnos6_template

* Addressed @gundalow comments
2016-09-16 22:45:08 +01:00
Adrian Likins 3e754086b6 Fix error using jsonfile with incomplete config (#17567)
If 'fact_caching=jsonfile' was configured, but
'fact_caching_connection' was not configured, jsonfile
would fail and ansible-playbook would exit with a traceback.

Fixes #17566
2016-09-16 15:08:02 -04:00
jctanner 29fda4be1e copy action plugin: recurse into sub folders of the source (#17614)
* copy action plugin: recurse into sub folders of the source

Fixes #13013

* Fix python3 bytes/strings

* Fix py3 again

* test
2016-09-16 14:26:19 -04:00
James Cammarata dfb1c0647e Revert "Move queuing tasks to a background thread"
This reverts commit b71957d6e6.
2016-09-15 17:00:06 -05:00
Toshio Kuratomi 70e63ddf6c Fix cow mode to work with unicode task and play names (#17576)
Fixes #17560
2016-09-14 20:02:35 -07:00
Senthil Kumar Ganesan 68f1705d18 Add action plugin for dnos6_config module (#17553) 2016-09-14 20:55:45 +01:00
Senthil Kumar Ganesan 7d9b07382b Add action pluging for dnos9_template module (#17549) 2016-09-14 20:40:55 +01:00
Senthil Kumar Ganesan 564e02c3c3 Add action plugin for dnos6_template module (#17554) 2016-09-14 20:37:56 +01:00
Senthil Kumar Ganesan daba7fa057 Moved the dnos10_template.py action plugin to the correct directory (#17550) 2016-09-14 20:28:01 +01:00
Adrian Likins 8438da2a34 Make jsonfile cache plugin errors less vague. (#17568) 2016-09-14 14:10:12 -04:00
Adrian Likins c633022fca [wip] Let jsonfile and memcached cache plugins understand fact_caching_timeout=0 (#17565)
* Add support for no-expiration to jsonfile cache

* Let memcached cache use fact_caching_timeout=0

If fact_cache=memcached and fact_caching_timeout=0
memcached would hit a NameError on _expire_keys
2016-09-14 13:27:42 -04:00
Brian Coca 4656b6a846 added msg info to unreachable for oneline callback
fixes #17367
2016-09-14 11:33:01 -04:00
Senthil Kumar Ganesan c6acf44a7c Added support for dnos9_config module 2016-09-13 10:42:12 -07:00
Brian Coca 7450629733 better display for free strategy (#17355) 2016-09-13 12:59:32 -04:00
Abhijit Menon-Sen 07756a4265 Remove extraneous . from VALID_FILE_EXTENSIONS
Otherwise if you «include_vars: vars.json», you get this error:

    "message": "/path/to/vars.json does not have a valid extension: yaml, yml, .json"
2016-09-13 19:41:26 +05:30
Peter Sprygada 16d6274276 raise exception when src file is not found in net_config
This will now raise an exception if the file path specified in src is not
found and the module will gracefully error.

ref #4797
2016-09-12 10:00:35 -04:00
James Cammarata f5cff2ced5 Revert "Flush the queued items in linear periodically"
This reverts commit d3418fd658.
2016-09-11 14:28:49 -05:00
Robin Schneider d76dd56ca3
Use addresses and names reserved for documentation
Trying to preserve the meaning of the examples. Not all occurrences in
`docsite/rst/playbooks_lookups.rst` have been changed for instance to
allow the unchanged examples to be used for testing.

Related to: #17479
2016-09-11 17:31:41 +02:00
James Cammarata d3418fd658 Flush the queued items in linear periodically 2016-09-10 09:43:49 -05:00
James Cammarata 6b206f5d00 Re-adding processing of results inline with queuing tasks for linear 2016-09-10 09:43:49 -05:00
Brian Coca 0aaee0272a using delegated host's facts when delegating (#17489)
fixes #17313
2016-09-09 17:26:47 -04:00
Alvaro Aleman e8897a9b91 Fix from_yaml for non-string_types. This resolves #17397 (#17398) 2016-09-09 13:21:58 -04:00
Brian Coca cd2f60e11a take delegated vars before removing them
fixes #17455
2016-09-09 09:55:24 -04:00
Dag Wieers 2daf527e63 Fix fileglob filter to work just like fileglob lookup plugin (#17480)
The fileglob lookup plugin only returns files, not directories.
This is to be expected, as a mixed list would not be very useful in with_fileglob.
However the fileglob filter does return anything glob.glob() returns.

This change fixes this, so that fileglob returns files (as the name indicates).

PS We could also offer a glob filter for thos that would need it ?

This relates to comments in issue #17136 and fixes confusion in #17269.
2016-09-09 09:37:29 -04:00
Dag Wieers 1df924e1d5 Clean up unarchive action plugin (#17429) 2016-09-09 09:27:38 -04:00
Maciej Delmanowski f5a4677680 Allow for no prefix in the comment filter (#17421)
In the 'comment' filter, if the 'prefix' parameter is set as empty,
don't add an empty line before the comment. To get the previous
behaviour (empty line before comment), set the prefix to '\n'.
2016-09-09 09:23:39 -04:00
Tobias Wolf 9838d6420a Re-fix the per-item diff output, (#17458)
which got lost in recent big 'performance improvements' merge by @jimi-c.

I had made a previous PR to fix this, then @bcoca had committed an
improved fix. Now it's lost again.

cf: d2b3b2c03e (lost here)
cf:  25e9b5788b (previous fix)

Earlier PR #14849
Earlier issue #14843

Please note that jimi-c broke this last time as well ... seeing a
pattern here.
2016-09-08 11:59:35 -04:00
Andrea Tartaglia ba28f1a2da Make ssh_executable available as ansible_ssh_* variable (#17450) 2016-09-07 14:13:11 -07:00
Brian Coca 65c373c55a fixed bad condition hiding results 2016-09-07 12:55:03 -04:00
@skg_net e5b8a177d7 Support files for dnos10_template module (#17399) 2016-09-07 17:35:02 +01:00
Brian Coca a4abf8da79 Flush cache (#17445)
* transfer module based on pipelining, not tmp

fixes #17381

* added missing flush cache option back
2016-09-07 12:12:41 -04:00
Andrea Tartaglia dd71469bb7 Added option to change ssh executable path (#17377) 2016-09-07 08:41:43 -07:00
Toshio Kuratomi 4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
Matt Davis f239e1e61f windows async changes and tests (#17400) 2016-09-06 17:38:12 -07:00
Matt Clay 94a0d2afb4 Add partially backwards compatible version of _fixup_perms. (#17427)
Also added a deprecation notice for _fixup_perms.

Resolves issue #17352 (assumes custom actions use recursive=False).
2016-09-06 16:49:59 -07:00
Toshio Kuratomi 1d412059a0 Fix UnicodeError loading plugins with non-ascii in them on python3 2016-09-06 16:35:25 -07:00
Matt Davis b860b2d258 windows environment support (#17402) 2016-09-06 13:00:21 -07:00
James Cammarata 7a0b25d5fa Fixing incorrect 'task' variable bug in strategy/__init__.py 2016-09-05 23:41:21 -05:00
Marc-André Gatien d56b0aa813 typo fix for variable ANSIBLE_ETCD_VERSION (#17373) 2016-09-05 23:00:56 -04:00
Brian Coca ff34f5548d Dynamic role include (#17401)
* dynamic role_include

* more fixes for dynamic include roles

* set play yfrom iterator when dynamic

* changes from jimi-c

* avoid modules that break ad hoc

TODO: should really be a config
2016-09-05 20:07:58 -04:00
Peter Sprygada 439aa353f1 fixes bug when template is missing src argument
adds AttributeError to exception handling to change missing src argument

fixes #17391
2016-09-05 08:54:14 -04:00
Peter Sprygada 9c5bcda4fe Merge pull request #17388 from privateip/asa
fix up asa shared module
2016-09-04 20:59:20 -04:00
Peter Sprygada d976c80a22 roll up of updates to junos shared module
* adds implementation for Network methods
* adds action plugin junos_config
* removes _log() and raises exceptions instead to be handled by the module
2016-09-04 08:37:33 -04:00
Peter Sprygada 7aa1220b96 fix up asa shared module
* add authorize() method to handle authorization
* move terminal commands to after authorization completed
* add save_config() method to handling writing config to disk
* fix minor issues with get_config
* adds action plugin asa_config
2016-09-04 08:31:40 -04:00
Toshio Kuratomi f7b22a5eaa Fix paramiko's exec_command() to return bytes on python3 (#17372)
* Fix paramiko's exec_command() to return bytes on python3

* Run test_connection for python3 now too

* Fix atomic_move for problem in shippable's testing

* Python-2.4 needs to use b()
2016-09-02 20:32:14 -07:00
Matt Clay cfacc1fd53 Pass '_' to async_wrapper for no argsfile. (#17374)
This provides compatibility with changes made to the async_wrapper module.
2016-09-02 18:33:33 -07:00
Peter Sprygada c8431e334b Merge pull request #17296 from skg-net/devel
Add support for Dell Networking OS10 Command and Config Modules
2016-09-01 10:26:30 -04:00
Tobias Wolf c23b11d212 [Inventory] Cache the result of enumerating groups and host names
for `VariableManager._get_magic_variables()`.

This saves a lot of time re-iterating the nearly always constant global
list of groups and their members.

Generate once and cache, and invalidate cache in case `add_host:` or
`group_by:` are used.
2016-09-01 06:19:49 -05:00
Roman Belyakovsky d80d986a38 Added to_datetime filter (#17145)
* Added to_datetime filter

* Added to_datetime filter documentation
2016-09-01 01:51:03 +02:00
James Cammarata 78b8f81cab New feature: add new meta action `end_play`
This feature also cleans up and extends the meta subsystem:
* Allows for some meta actions (noop, clear_facts, clear_host_errors,
  and end_play) to operate on a per-host basis, meaning they can work
  with the free strategy as expected.
* Allows for conditionals on meta tasks.
* Fixes a bug where (for the linear strategy) metas were not treated
  as a run_once task, meaning every host in inventory would run the
  meta task.

Fixes #1476
2016-08-31 13:59:58 -05:00
James Cammarata b71957d6e6 Move queuing tasks to a background thread 2016-08-31 13:33:01 -05:00
Michael Scherer acd69bcc77 Fix url lookup for python 3 (#17295)
* Use six instead of urllib2, for python 3 compat

* Open the certificate file using binary mode

On python3, os.write requires 'bytes'. Also avoid
using a too broad exception, since the issue was hard
to spot due to it.

* Do not add the header User-agent if not set

Python3 module do raise a exception if a header is
not a string-like object, and the default value is None.
2016-08-31 07:03:20 -07:00
Andrew Gaffney f65a3ce547 Support for specifying item label in a loop (#17294) 2016-08-31 09:59:43 -04:00
Peter Sprygada 61e7c3af1a Merge pull request #17315 from privateip/openswitch
minor bug fixes in openswitch shared module
2016-08-31 09:59:30 -04:00
Peter Sprygada ec3dcefd39 minor bug fixes in openswitch shared module
* fix setting cookie after successful login
* raise NotImplementedError if run_commands is called in Rest
* return header msg key if status is not 2xx
* add action plugin ops_config
2016-08-30 23:04:41 -04:00
Allen Sanabria 03132041fb Include vars updated to work with directories (#17207)
* New features for include_vars

include_vars.py now allows you to include an entire directory and its nested directories of variable files.

Added Features..

* Ignore by default *.md, *.py, and *.pyc
* Ignore any list of files.
* Only include files nested by depth (default=unlimited)
* Match only files matching (valid regex)
* Sort files alphabetically and load in that order.
* Sort directories alphabetically and load in that order.

```
    - include_vars: 'vars/all.yml'

    - name: include all.yml
      include_vars:
        file: 'vars/all.yml'

    - name: include all yml files in vars/all and all nested directories
      include_vars:
        dir: 'vars/all'

    - name: include all yml files in vars/all and all nested directories and save the output in test.
      include_vars:
        dir: 'vars/all'
        name: test

    - name: include all yml files in vars/services
      include_vars:
        dir: 'vars/services'
        depth: 1

    - name: include only bastion.yml files
      include_vars:
        dir: 'vars'
        files_matching: 'bastion.yml'

    - name: include only all yml files exception bastion.yml
      include_vars:
        dir: 'vars'
        ignore_files: 'bastion.yml'
```

* Added whitelist for file extensisions (yaml, yml, json)

* Removed unit tests in favor of integration tests
2016-08-30 14:34:31 -07:00
Michael Scherer 65f1eb8e5c Reduce no longer exist in python3, so we have to use six (#17304) 2016-08-30 10:58:22 -07:00
Michael Scherer adcb87f781 file() function in python3 no longer exist, so use open() (#17303) 2016-08-30 10:47:41 -07:00
Michael Scherer 9245c786db Fix the lines lookup to work on python3 (#17291)
Since stdout is (on python3) of type 'bytes', callbacks
plugins fail in the test suite, because calls backs expect
a string.
2016-08-30 10:38:55 -07:00
Michael Scherer 8f364b549b Fix indexed_items.py to run on python3 (#17292)
On python3, zip is a iterator so we need
to explictily create the list from that.
2016-08-30 09:49:58 -07:00
Senthil Kumar Ganesan f6aa730c3c Utilities to support DELL Networking OS10 command and config modules 2016-08-29 21:29:26 -07:00
Toshio Kuratomi fa804125b5 Python3 fixes and porting (#17271)
* Fix to_native call in selinux_context and selinux_default_context to
  use the error handler correctly.
* Port set_mode_if_different to work on python3
* Port atomic_move to work on python3
* Fix check_password_prompt variable which wasn't renamed properly
2016-08-29 09:11:40 -07:00
Brian Coca bd9094c925 include_role (role revamp implementation) (#17232)
* attempt #11 to role_include

* fixes from jimi-c

* do not override load_data, move all to load

* removed debugging

* implemented tasks_from parameter, must break cache

* fixed issue with cache and tasks_from

* make resolution of from_tasks prioritize literal

* avoid role dependency dedupe when include_role

* fixed role deps and handlers are now loaded

* simplified code, enabled k=v parsing

used example from jimi-c

* load role defaults for task when include_role

* fixed issue with from_Tasks overriding all subdirs

* corrected priority order of main candidates

* made tasks_from a more generic interface to roles

* fix block inheritance and handler order

* allow vars: clause into included role

* pull vars already processed vs from raw data

* fix from jimi-c blocks i broke

* added back append for dynamic includes

* only allow for basename in from parameter

* fix for docs when no default

* fixed notes

* added include_role to changelog
2016-08-26 13:42:13 -04:00
Michael Scherer a30f545a62 Do not convert Nonetype to "None" (#17261)
If someone use a task with a empty name like this:

  - name:
    command: true

This will result in displaying 'None' as a task name instead of
'command'.
2016-08-26 09:48:27 -07:00
Abhijit Menon-Sen b4a035718e Make _display_plugin_load much less noisy
There was general consensus that displaying every plugin load on -vvv
was *way* too noisy. This commit reformats the log message to be less
verbose, and drops it down to debugging-only level.
2016-08-26 20:28:58 +05:30
Toshio Kuratomi f57f33a8e7 Fix fetch idempotence (#17255)
Fetch always follows symlinks when downloading so it needs to always
follow symlinks when getting the checksum of the file as well.
2016-08-25 21:45:38 -07:00
Toshio Kuratomi 0a39700b36 Fix octal output in a few more places (#17250)
Fix filetree lookup plugin for python3 (octal output and selinux API
takes native strings)
2016-08-25 14:58:35 -07:00
Toshio Kuratomi bd68c324ce Get the ssh plugin working with python3 (#17234) 2016-08-25 10:57:55 -07:00
Chris Faulkner dbab23e68f Fix context_demo callback plugin. (#17235) 2016-08-25 10:22:27 -07:00
Abhijit Menon-Sen 3ab9dddb3a Make the extract filter return Undefined on KeyError
groups['x']|map('extract', hostvars, 'somevar') would break if any host
didn't have 'somevar' set. With this change, it will return Undefined
instead. This change permits |map('extract', …)|map('default', 42) to
set a default value in such cases.
2016-08-25 22:45:20 +05:30
Peter Sprygada 24e8e3b03f initial add of shared modules for supporting Nokia SROS devices
This adds a cli transport, netcfg, and netcli implementations for working
with devices running Nokia SROS.  There is also an update to netcfg
to support the sros config file format.
2016-08-25 11:47:00 -04:00
Matt Clay 178292d2cd Fix file and copy modules on py3 and enable tests. (#17239)
- Fix octal formatting of file mode in module response on py3.
- Convert file path to unicode in copy action.
- Enable file and copy module tests for py3 now that they pass.
2016-08-25 07:44:31 -07:00
Abhijit Menon-Sen 1a62fe3874 Merge pull request #13739 from chrrrles/ipaddr_empty_string
ipaddr filter properly handle addresses on /31 networks
2016-08-25 15:47:08 +05:30
Toshio Kuratomi 7b4f808a21 One more izip_longest => zip_longest fix (#17229) 2016-08-24 13:50:14 -07:00
Toshio Kuratomi 51ec35378d xrange and izip_longest aren't available in vanilla python3 (#17226)
Fixes for these are either rewriting to get rid of the need for the
functions or using six.moves to get equivalent functions for both
python2 and python3
2016-08-24 12:28:02 -07:00
Yannig 27b0f3241b new filter human_bytes: convert a string (ex: 1Mo, 1K) into bytes (#12074)
* Rework human_readable and human_to_bytes.
New filter human_to_bytes.

* Fix for python 3.
2016-08-24 12:04:20 -07:00
Toshio Kuratomi 5d865ec1ef Cleanup debug.py (#17222)
* Use isinstance instead of comparing to type.
* Change check against unicode type to check against six.string_types
  for python3 compatibility.
2016-08-24 10:37:15 -07:00
Peter Sprygada fa0d39a1b4 updates iosxr shared module
This completes the refactor of the iosxr 2.2 shared module.  It also
includes the iosxr_config action plugin to be implemented by the
iosxr_config module for 2.2
2016-08-24 11:03:01 -04:00
Toshio Kuratomi a22909c226 Migrate basestring to a python3 compatible type (#17199) 2016-08-23 13:13:44 -07:00
Adrian Likins 8bbbe16d31 try/except xmltodict import, misc cleanups (#16287)
The 'import xmltodict' was causing import
errors when generating documentation. Since
xmltodict is a required but not stdlib module,
throw AnsibleError if unable to import.

Remove unused combine_vars.

Replace a use of 'stdin_iterator == None' with
idiomatic 'stdin_iterat is None'

Misc pep8 cleanups.
2016-08-23 13:07:25 -04:00
Adrian Likins 71118b2720 Show where plugins were loaded in vvv/vvvv (#15757)
Make the plugin loading info displayed by callback plugins
match.

In debug mode (ANSIBLE_DEBUG=1 env), log all requests for
plugins including already cached plugins and class_only
requests.
2016-08-23 11:54:14 -04:00
Toshio Kuratomi 76f9935634 Add some missing imports from last night's py3 fixes (#17196) 2016-08-23 08:06:20 -07:00
Marius Gedminas 73caff58e8 Add missing to_unicode import 2016-08-23 09:24:02 +03:00
Marius Gedminas 483c83b088 There's no ansible.module_utils_unicode 2016-08-23 09:22:03 +03:00
Toshio Kuratomi 313d4b2c9e Move a path being passed around as a byte string to being passed around as a text string. (#17190)
This is enough to get minimal copy module working on python3

We have t omodify dataloader's path_dwim_relative_stack and everything
that calls it to use text paths instead of byte string paths
2016-08-22 21:55:30 -07:00
Peter Sprygada 20bde8f549 Merge pull request #17189 from privateip/eos
updates eos shared module
2016-08-22 20:52:18 -04:00
Peter Sprygada b5bbac29e5 updates eos shared module
* adds support for netcli methods
* adds support for netcfg methods
* Cli class now derives from CliBase
* adds eos_config action plugin
2016-08-22 20:26:16 -04:00
Adrian Likins a4785c2691 Fix docker connection plugin version tests and py2.6 compat (#16841)
* Rm py2.7+ code in docker connection plugin

The docker connection plugin was using subprocess.check_output
which only exists in python 2.7 and later. Connection plugins
need to support python2.6 so this replaces it with Popen/communicate()

* Handle docker ver errors in docker connection

Add unit tests for DockerConnection

Fixes #16971
2016-08-22 10:39:38 -04:00
Peter Sprygada 4ab4b6698d updates nxos shared module refactor
This commit updates the nxos transport shared plugins for
2.2.  This includes updates to both Cli and Nxapi.  This commit
also includes the nxos_config action plugin
2016-08-21 08:57:49 -04:00
Peter Sprygada d50ef4446f action plugin net_config will now remove any private result key
This update will now remove any keys from results that are created using
the private names.  Private names are identified as double underscore (__)
on either side of the key name
2016-08-19 11:15:36 -04:00
Dusty Mabe b817f1f3ea actions/unarchive: fix unarchive from remote url (#17126)
* actions/unarchive: fix unarchive from remote url

Currently unarchive from remote url does not work because the core
unarchive module was updated to support 'remote_src' [1], but the
unarchive action plugin was not updated for this. This causes failures
because the action plugin assumes it needs to copy a file to the
remote server, but in the case of downloading a file from a remote
url a local file does not exist, so an error occurs when the file is
not found.

[1] https://github.com/ansible/ansible-modules-core/commit/467516e

* test_unarchive: fix test with wrong remote_src use

The non-ascii filenames test had improperly set remote_src=yes even
though it was actually copying the file from the local machine (i.e.
the file did not already exist remotely). This test was passing
until the remote_src behavior of unarchive was fixed in 276550f.
2016-08-19 08:16:21 -05:00
James Cammarata 732d722a7f Catch a missing include_vars filename
Related to ansible/ansible-modules-core#4445
2016-08-17 09:35:37 -05:00
Matt Davis e0bdb502e3 raw now returns changed: true (#17112)
for consistency w/ shell/command/script "non-idempotent" modules.

Updated tests, changelog.
2016-08-16 20:39:23 -07:00
James Cammarata f4237b2151 Rework the way params are assigned to TaskIncludes when they're dynamic
Copying the TaskInclude task (which is the parent) before loading the blocks
makes the code much more simple and clean, and fixes a bug introduced during
the performance improvement changes (and specifically the change which moved
things to a single-parent model).

Fixes #17064
2016-08-16 15:45:24 -05:00
Brian Coca 5fe9d3c3d5 make parsed param private and explicit (#17104)
* make parsed param private and explicit

* fixed missed parsed
2016-08-16 11:59:30 -04:00
James Cammarata f722d41eab Allow notifies to be sent to the top level includes when they were static
Since we introduced static includes in 2.1, this broke the functionality
where a notify could be sent to a named include statement, triggering all
handlers contained within the include. This patch fixes that by adding a
search through the parents of a handler for any TaskIncludes which match.

Fixes #15915
2016-08-15 14:22:35 -05:00
James Cammarata c669a381d1 Don't immediately return failed for any_errors_fatal tasks
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.

Fixes #16937
2016-08-12 14:18:50 -05:00
Dag Wieers 1268f4778d Introduce new 'filetree' lookup plugin (#14332)
* Introduce new 'filetree' lookup plugin

The new "filetree" lookup plugin makes it possible to recurse over a tree of files within the task loop. This makes it possible to e.g. template a complete tree of files to a target system with little effort while retaining permissions and ownership.

The module supports directories, files and symlinks.

The item dictionary consists of:
 - src
 - root
 - path
 - mode
 - state
 - owner
 - group
 - seuser
 - serole
 - setype
 - selevel
 - uid
 - gid
 - size
 - mtime
 - ctime

EXAMPLES:
Here is an example of how we use with_filetree within a role:

```yaml
 - name: Create directories
   file:
     path: /web/{{ item.path }}
     state: directory
     mode: '{{ item.mode }}'
     owner: '{{ item.owner }}'
     group: '{{ item.group }}'
     force: yes
   with_filetree: web/
   when: item.state == 'directory'

 - name: Template complete tree
   file:
     src: '{{ item.src }}'
     dest: /web/{{ item.path }}
     state: 'link'
     mode: '{{ item.mode }}'
     owner: '{{ item.owner }}'
     group: '{{ item.group }}'
   with_filetree: web/
   when: item.state == 'link'

 - name: Template complete tree
   template:
     src: '{{ item.src }}'
     dest: /web/{{ item.path }}
     mode: '{{ item.mode }}'
     owner: '{{ item.owner }}'
     group: '{{ item.group }}'
     force: yes
   with_filetree: web/
   when: item.state == 'file'
```

SPECIAL USE:
The following properties also have its special use:

 - root: Makes it possible to filter by original location
 - path: Is the relative path to root
 - uid, gid: Makes it possible to force-create by exact id, rather than by name
 - size, mtime, ctime: Makes it possible to filter out files by size, mtime or ctime

TODO:
 - Add snippets to documentation

* Small fixes for Python 3

* Return the portion of the file’s mode that can be set by os.chmod()

And remove the exists=True, which is redundant.

* Use lstat() instead of stat() since we support symlinks

* Avoid a few possible stat() calls

* Bring in line with v1.9 and hybrid plugin

* Remove glob module since we no longer use it

* Included suggestions from @RussellLuo

- Two blank lines will be better. See PEP 8
- I think if props is not None is more conventional 😄

* Support failed pwd/grp lookups

* Implement first-found functionality in the path-order
2016-08-11 23:33:54 -04:00
jctanner 39d764c7ad Restore previous behavior of ignoring missing files via with_fileglob (#17053)
Fixes #16801
2016-08-11 19:55:21 -04:00
James Cammarata 925b0ff9e9 cleaning up some debug statements in _queue_task() 2016-08-11 14:26:28 -05:00
James Cammarata 1c7e0c73c9 Several fixes for includes
* when including statically, make sure that all parents were also included
  statically (issue #16990)
* properly resolve nested static include paths
* print a message when a file is statically included

Fixes #16990
2016-08-11 14:07:49 -05:00
James Cammarata 68d44e48ba Remove sleeps and rely on regular queue options to process results 2016-08-11 09:11:32 -05:00
Dag Wieers 718f431466 Allow to make the jsonfile cache files pretty (indented and sorted) (#17000)
* Allow to make the jsonfile cache files pretty (indented and sorted)

Since the json cache files are condensed, it is not very practical to look for something in them. Having indented/sorted cache files makes debugging and playbook/inventory development a lot easier to do.

I made it configurable in case people would object to the performance hit this would have, but to be honest, then they probably should be looking at other cache plugins instead IMO.

* Removed the config option and documentation changes
2016-08-08 17:15:19 -04:00
James Cammarata 06d4f4ad0e Move tasks/blocks to a single parent model 2016-08-08 15:58:46 -05:00
James Cammarata d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
Filipe Niero Felisbino e54a9d3a51 Add generic data structures querying (#13684)
* Query lookup plugin

* Add license and docstrings

* Add python3-ish imports

* Change query plugin type from lookup to filter

* Switch from dq to jsonpath_rw

* Add integration test for query filter

* Rename query filter to json_query

* Add jsonpath-rw

* Rename query filter to json_query

* Switch query implementation from jsonpath-rw to jmespath
2016-08-08 11:55:59 -04:00
Adrian Likins f21df311bc Fix hipchat callback "has no attribute display'" (#16998)
Use self._display for access to display object for
the hipchat callback plugin.

Fixes #16946
2016-08-08 11:04:06 -04:00
Matt Clay 72cca01cd4 Use file list, not recursion, in _fixup_perms. (#16924)
Run setfacl/chown/chmod on each temp dir and file.

This fixes temp file permissions handling on platforms such as FreeBSD
which always return success when using find -exec. This is done by
eliminating the use of find when setting up temp files and directories.

Additionally, tests that now pass on FreeBSD have been enabled for CI.
2016-08-05 18:40:28 -07:00
Matt Clay e07fbba0a5 Add missing boilerplate code. (#16980) 2016-08-05 12:22:52 -07:00
James Pic d2438b6b6b Debug output callback (#16839)
* Added debug output callback

* Better import

* Comment on line
2016-08-05 13:57:13 -04:00
James Cammarata 1714279b5e Tweak the way the debug strategy imports the linear strategy parent
Due to the way we load plugins, internally to Python there can be issues when
the debug strategy is loaded after the linear strategy. To work around this,
we're changing the import line for the linear strategy to avoid the problem.

Related to #16825
2016-08-05 10:06:50 -05:00
Brian Coca bced8715cd add transparent vault use to other plugins (#16957)
assemble, script and unarchive (copy already had it).
2016-08-04 18:35:30 -04:00
ovcharenko f326e49d75 Fix for issue @synchronize doesn't substitute variables properly #16347 (#16349)
* Fix for issue @synchronize doesn't substitute variables properly #16347
2016-08-04 11:27:44 -07:00
victoru 14901b65d9 raise AnsibleError in hashi_vault lookup plugin when hvac module is not installed (#16859) 2016-08-04 10:06:12 -07:00
Brian Coca b0786a1845 actually skip bad/broken module files found 2016-08-04 11:32:08 -04:00
Adrian Likins 36bf1e6b7e Fix funcd to at least import without errors. (#16288)
This plugin was using very old api, so was
updated to newer api.

Also misc style/pep8 cleanups.
2016-07-29 16:19:38 -04:00
Brian Coca e2f17f8d9b set cwd to task's basedir (#16805)
* switch cwd to basedir of task

This restores previous behaviour in pre 2.0 and allows for 'local type' plugins
and actions to have a more predictable relative path.

fixes #14489

* removed FIXME since prev commit 'fixes' this

* fix tests, now they need a loader (thanks jimi!)
2016-07-25 08:11:45 -04:00
Brian Coca 7fdbfd490e allow include_vars to namespace imports 2016-07-24 21:31:11 -04:00
Matt Martz 4065acc37d indent callback output if using 3verbosity or higher (#16231) 2016-07-23 18:10:53 -04:00
Brian Coca 71c97685a1 fixes for service module interaction with systemd (#16720)
now systemd will run even if service module is inovked with parameters that it does not support
these will be removed before invoking systemd and issue a warning.

this facility will work for any new service modules.
2016-07-22 20:35:20 -04:00
Peter Sprygada e4b3032231 removes methods to write to dest from net_config 2016-07-15 17:34:46 -04:00
Peter Sprygada e5e387562d add vyos action plugin for handling local operations
This plugin extends the functions of the net_config plugin to vyos
2016-07-15 08:30:21 -04:00
jctanner 238c6461f6 Add a function to check for killed processes in all strategies (#16684)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
2016-07-14 16:37:35 -04:00
Brian Coca 3c39bb5633 fixed lookup search path (#16630)
* fixed lookup search path

added ansible_search_path var that contains the proper list and in order
removed roledir var which was only used by first_found, rest used role_path
added needle function for lookups that mirrors the action plugin one, now
both types of plugins use same pathing.

* added missing os import

* renamed as per feedback

* fixed missing rename in first_found

* also fixed first_found

* fixed import to match new error class

* fixed getattr ref
2016-07-13 10:06:34 -04:00
Brian Coca ed7623ecde J2 test docs (#16646)
* moved tests from filters to actual jinja2 tests

also removed some unused declarations and imports

* split tests into their own docs

removed isnan as existing jinja2's 'number' already covers same
added missing docs for several tests

* updated as per feedback
2016-07-12 10:13:00 -04:00
James Cammarata 245ce9461d Fix unreachable host/any_errors_fatal bug in linear strategy
2e003adb added the ability for tasks using any_errors_fatal to fail
when there were unreachable hosts. However that patch used the running
unreachable hosts data rather than the results from the current task,
which causes failures when any run_once or BYPASS_HOST_LOOP task is hit
after an unreachable host causes a failure. This patch corrects that by
using the current set of results to determine if any hosts were
unreachable during the last task only.

Fixes ansible/ansible-modules-core#4160
2016-07-12 03:01:47 -05:00
Peter Sprygada 075a03d511 adds action plugin for working with network configuration files
This adds a action plugin that will allow config and template modules
to be merged into a single module.  Once completed this will supercede
the net_template action plugin.
2016-07-11 06:12:31 -07:00
Brian Coca 7657f75f75 removed defined/undefined as jinja2 already has em 2016-07-08 12:11:22 -04:00
Brian Coca 7f28bcacaa added defined/undefined tests 2016-07-08 11:28:37 -04:00