Commit Graph

1737 Commits (64a9794997ed631ab4346171bf10aacc0a7e219e)

Author SHA1 Message Date
Toshio Kuratomi c033e5111f Graceful error message for abstract base classes in PluginLoader 2017-02-22 10:27:29 -05:00
Brian Coca 374af06cbf added pickle and yaml cache plugins
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
2017-02-22 10:27:29 -05:00
Toshio Kuratomi 2dde7f7768 Fix a case where we mixed text and bytes in the local connection plugin
Fixes #20710
2017-02-21 14:19:08 -08:00
James Cammarata 3d65482927 Stash post-validated fields of the task in the TaskResult
This allows us to have a snapshot of the fields, which we can restore
on the pre-fork side so as to avoid having to re-template fields.
2017-02-21 14:35:13 -06:00
Peter Sprygada 6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 2017-02-21 09:21:41 -05:00
Peter Sprygada 2567e8157f fixes bug where handle_prompts failed if list of prompts provided (#21711)
This will now automatically convert the prompts to a list and iterate
over the regexp looking for a prompt match before supplying the answer.
2017-02-21 08:27:33 -05:00
Peter Sprygada e4a2c804be partial revert of changes introduced in d5f7a0181b (#21688) 2017-02-20 15:37:14 -05:00
Marc Abramowitz e2e4a69425 hashi_vault: Get token from env var or file
This allows getting the Vault token from the `VAULT_TOKEN` env var or
from the file `$HOME/.vault-token`, as both of these are understood by
the Vault CLI and are a common place to put Vault tokens. This allows
avoiding hard-coding a Vault token into playbooks or having to include
lookups.

`HOME/.vault-token` is nice because a user can authenticate with the CLI
using `vault auth` and then the token will be stored in
`$HOME/.vault-token`. If we read this file, then we allow someone to do
`vault auth` "out of band" to set up Vault access.
2017-02-20 10:10:22 -05:00
René Moser 576ff0728d service: deprecate state=running (#21499)
* service: deprecate state=running

* deprecated with version 2.7
2017-02-20 09:34:52 -05:00
Peter Sprygada bd036c15e0 minor fixes to junos action (#21643)
* adds instrumentation messages
* checks cli context is not in config mode
2017-02-19 18:04:00 -05:00
Peter Sprygada ce9180b3a1 roll up of vyos action fixes (#21642)
* check cli context is not in config mode
* add display messages for more details
2017-02-19 17:37:16 -05:00
Peter Sprygada 8472ed640b fixes minor bugs in nxos action (#21641)
* checks cli context is correct
* adds display messages
2017-02-19 17:13:14 -05:00
Peter Sprygada d5f7a0181b roll up of eos fixes (#21629)
* fixes error where eos would close the cli shell
* fixes network_cli connection plugin to check before calling open_shell()
* fixes json commands being sent over eapi
2017-02-19 08:46:14 -05:00
Peter Sprygada 2f10bdf0c7 roll up of fixes in junos action plugin (#21624)
* calls open_shell() or open_session() depending on connection type
* closes shell after module completion
* adds open_session() to netconf
2017-02-18 17:20:03 -05:00
Peter Sprygada 920f9f4815 fixes nxos nxapi implementation (#21615)
* correctly maps play_context to nxapi values
* fixes bug in nxos_nxapi module detecting nxapi feature
* updates nxos shared lib provider values
* fixes missing ssh_keyfile in nxos shared lib
2017-02-18 11:20:26 -05:00
Peter Sprygada c875393916 roll up of fixes for vyos base (#21616)
* adds ssh_keyfile to vyos
* fixes play_context mapping to provider
* adds missing network_os to play_context
* executes close_shell() after module completes
2017-02-18 09:29:17 -05:00
Peter Sprygada e6dfbf63f6 fixes eos action mapping of play_context values (#21614)
* adds mapping for ssh_keyfile
* adds correct mappings for eapi
* fixes missing no_log value for provider arg
2017-02-18 09:29:06 -05:00
Brian Coca ee17b91455 Revert "add systemd-nspawn connection driver"
This reverts commit 3597ca082b.
2017-02-17 16:36:46 -05:00
Brian Coca e84713c76c Revert "Remove the --nspawn-extra-args cli option"
This reverts commit b8125ac1a6.
2017-02-17 16:35:47 -05:00
Brian Coca 2d2874fb3c Revert "Add safety checks to nspawn connection plugin"
This reverts commit 60bb677154.

this connection cannot work as is, systemd-nspawn creates instance on each command
2017-02-17 16:35:47 -05:00
Toshio Kuratomi 275be702a8 Switch from using inspect to globals() 2017-02-17 12:57:26 -08:00
Peter Sprygada 8716a5bc67 maps ios provider ssh_keyfile to play_context (#21597)
This allows the key file to be specified in the playbook for the ios
module provider argument
2017-02-17 18:56:00 +00:00
Thomas Szymanski 60bb677154 Add safety checks to nspawn connection plugin
This patch adds some checks on the path that is accessed as a container,
making sure it looks like one. It implements the connection method and
add adaptations to the modern way of writing connections for Ansible.
It also rewords docs and vars to use the nspawn terminology instead of
chroot.
2017-02-17 12:39:48 -05:00
Thomas Szymanski b8125ac1a6 Remove the --nspawn-extra-args cli option
Providing extra arguments to pass to systemd-nspawn is still possible
through the `nspawn_connection` of the `ansible.cfg` file.
2017-02-17 12:39:48 -05:00
Lars Kellogg-Stedman 3597ca082b add systemd-nspawn connection driver
This commit adds a connection driver built on top of systemd-nspawn.
This is similar to the existing `chroot` driver, except that nspawn
offers a variety of additional services. For example, it takes care of
automatically mounting `/proc` and `/sys` inside the chroot environment,
which will make a variety of tools work correctly that would otherwise
fail.

You can take advantage of other system-nspawn features to perform more
complicated tasks.  For example, on my x86_64 system I have a Raspberry
Pi disk image mounted on `/rpi`.  I can't use `chroot` with this because
the binaries contained in the image are for the wrong architecture.
However, I can use the systemd-nspawn `--bind` option to automatically
insert the appropriate qemu-arm binary into the container using an
inventory file like this:

    pi ansible_host=/rpi ansible_nspawn_extra_args='--bind /usr/bin/qemu-arm --bind /lib64'

See http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
for more information about systemd-nspawn itself.
2017-02-17 12:39:48 -05:00
Peter Sprygada a6cecef6bc clean up of terminal plugins (#21549)
* removes unneeded supports_multiplexing var
* refactors terminal_prompts_re to terminal_stdout_re
* refactors terminal_errors_re to terminal_stderr_re
* updates network_cli unit test cases
2017-02-17 10:00:23 -05:00
Ricardo Carrillo Cruz c5452eef6f Removes default kwarg in iosxr_argument_spec for timeout (#21569) 2017-02-17 15:59:24 +01:00
Peter Sprygada 4cbbed0b37 fixes issue with prompt detection in network_cli (#21574)
The network_cli plugin would return immediately if an error was
detected.  This patch will force the connection plugin to still try to
detect the current prompt even if an error is found.
2017-02-17 14:13:26 +00:00
Peter Sprygada b0abbb5f8b removes the default kwarg in ios_argument_spec for timeout (#21552)
Uses the configured timeout setting instead of the arg_spec

fixes #21520
2017-02-17 13:12:12 +01:00
Matt Davis 8527013fbe Complete rewrite of Windows exec wrapper (#21510)
* supports pipelining for faster execution
* supports become (runas), creates interactive subsession under WinRM batch logon
* supports usage of arbitrary module_utils files
* modular exec wrapper payload supports easier extension
* integrates async wrapper behavior for pipelined/become'd async
* module_utils are loaded as true Powershell modules, no more runtime modifications to module code
2017-02-17 00:09:56 -08:00
James Cammarata 7bf56ceee3 Relocate creation of Templar in process_pending_results
Moving it to after the blocks where per-item results are calculated,
as it's not used there and causes quite a performance hit being there.

Fixes #21340
2017-02-17 00:26:09 -06:00
Peter Sprygada c9f6a2b740 fixes issue with cli shell left open (#21548)
The nxos action plugin will now close the shell connection once the
module has completely run
2017-02-16 22:11:32 -05:00
Peter Sprygada 20c5a1adc1 verifies cli context for iosxr (#21550)
Checks cli context and exits config mode if needed
2017-02-16 22:11:21 -05:00
Peter Sprygada 9d4a3599b8 bug fixes and updates for eos connections (#21534)
* refactors supports_sessions to a property
* exposes supports_sessions as a toplevel function
* adds open_shell() to network_cli
* implements open_shell() in eos action plugin
2017-02-16 20:26:48 -05:00
Peter Sprygada a01288859d check ios cli context and exits config mode if there (#21544) 2017-02-16 20:26:29 -05:00
Toshio Kuratomi eeaec56ed5 Use isinstance instead of type() comparisons. isinstance is more robust 2017-02-16 16:34:43 -08:00
Peter Sprygada 02d2b753db refactors junos modules to support persistent socket connections (#21365)
* updates junos_netconf module
* updates junos_command module
* updates junos_config module
* updates _junos_template module
* adds junos_rpc module
* adds junos_user module
2017-02-16 10:53:03 -05:00
Brian Coca cc0bb54d2c the return of reset_connection
allows user to force persistent connection to close, needed for when
you want to benefit from changes applied to the current play but persistent connections
prevent them from being realized.
2017-02-16 10:49:57 -05:00
Peter Sprygada 34e6cc788f check cli context to be sure out of config mode in ios (#21493)
This change will now check the cli context after a module runs and if
the cli is still in config mode it will exit config mode.  Also fixes a
minor issue with converting list of commands to a dict

fixes #21481
2017-02-16 12:54:33 +00:00
James 87646595e3 Remove the end of the task_data.path which causes #21461 2017-02-15 17:35:16 -08:00
Michael Scherer 6fae1d2bbf Make password lookup treat /dev/null as a special case
I have from time to time a need of random password without
wanting to write them down (one example is mailman list creation,
that requires a password to be given to be sent to the list owner).

But using /dev/null do not return null, but the empty string, which
doesn't generate a password at all and so do not achieve my use case.
2017-02-15 12:15:09 -08:00
Matt Clay 09f4242ce4 Fix junit callback handling of surrogate escapes. 2017-02-15 10:51:55 -08:00
Toshio Kuratomi 99fd2328af Fix hash filter for non-ascii strings and Python3
hashlib hashes operate on byte strings.  When given a text string on
Python3, hashlib backtraces.  When given a text string on Python2,
hashlib will backtrace if the string contains non-ascii characters.
Encode the text string to utf-8 prior to hashing to avoid this problem.

Fixes #21452
2017-02-15 10:50:10 -08:00
Peter Sprygada 48b02336ab code clean of old code from network modules updates (#21469)
* removes unused code
* removes module_utils/local.py
* removes plugins/action/network.py
* removes action_handler from connection plugins
* removes code to use action_handler in task_executor
* updates action plugins to subclass from normal
2017-02-15 12:46:30 -05:00
Peter Sprygada 21d993a4b8 refactors nxos module to use persistent connections (#21470)
This completes the refactor of the nxos modules to use the persistent
connection.  It also updates all of the nxos modules to use the
new connection module and preserves use of nxapi as well.
2017-02-15 11:43:09 -05:00
Peter Sprygada eb1453a366 updates iosxr modules to support socket (#21231)
* updates all iosxr modules to support persistent socket
* adds iosxr action plugin to connect to device
* adds exec_command() to iosxr shared module
* fixes iosxr_config and iosxr_template local action
* update all unit test cases
* adds base test module for iosxr module testing
2017-02-15 10:47:02 -05:00
Peter Sprygada 381a045089 updates network local actions to check for connection=local (#21437)
* updates vyos local action
* updates ios local action
2017-02-15 11:27:54 +00:00
Peter Sprygada 8e4f0ec162 checks connection type is local and fails if not in eos (#21429) 2017-02-14 18:42:15 -05:00
Peter Sprygada e1a2c6e1d3 roll up of fixes for eos modules (#21406)
* fixes issue with load_provider() not checking for an existing key
* adds updates to eos_config results key
* lots of minor syntax fixes in eos shared module
* adds eos_argument_spec to eos_eapi

fixes #21402
2017-02-14 18:47:29 +00:00
Patrick Deelman 65cd21e9a8 passwordstore (pass) lookup module (#21314)
* passwordstore (pass) lookup module

* removed shebang
changed licensing
2017-02-14 11:12:39 -05:00
Peter Sprygada d43eac93bc ios local action creates circular reference (#21404)
When loading the params in the local action, the provider key was
inadvertently being loaded as well.  This created the circular
reference.  The load_provider() method will now check for the provider
key and skip it when encountered.

fixes #21399
2017-02-14 09:56:52 -05:00
Peter Sprygada 14b942f3fb updates eos modules to use socket (#21197)
* updates eos modules to use persistent connection socket
* removes split eos shared module and combines into one
* adds singular eos doc frag (eos_local to be removed after module updates)
* updates unit test cases
2017-02-13 20:22:10 -05:00
Peter Sprygada 9937e604f5 fixes ios_facts that was returning no values (#21357)
* adds conversion to command dict into ios shared lib
* fixes hardware commands in ios_facts
* adds network_os value to play_context in ios action plugin

fixes #21190
2017-02-13 20:20:44 -05:00
Peter Sprygada 8adb108aa9 updates vyos modules to use socket connection (#21228)
* updates all vyos modules to use socket connection
* adds vyos local action handler
* adds exec_command() to vyos
* updates vyos_config local action
* update unit test cases
* add base class for testing vyos modules
2017-02-13 10:41:22 -05:00
Peter Sprygada 7f1c43e597 updates ios modules to support persistent socket (#21258)
* updates all ios modules to support persistent socket
* adds ios action plugin to connect to device
* adds exec_command() to ios shared module
* fixes ios_config and ios_template local action
* update all unit test cases
* adds base test module for ios module testing
2017-02-13 15:22:14 +00:00
Peter Sprygada 92aa92ebd2 updates netconf to use persistent socket (#21307) 2017-02-12 09:05:23 -05:00
Toshio Kuratomi 98541b7c8b Fix to bytes surrogate and nonencodable chars (#21180)
* Add a surrogate_then_replace error strategy to keep to_bytes from tracebacking by default
* Port all code that explicitly used surrogate_or_replace to surrogate_then_replace
2017-02-09 17:13:40 -08:00
Ssawa 149dd9ca86 Update winrm to allow fetch_file to work on files in use by processes (#18174) 2017-02-09 16:43:52 -08:00
Peter Sprygada 138051540e updates the code path for network modules (#21193)
* replaces persistent connection digest with _create_control_path()
* adds _ansible_socket to _legal_inputs in basic.py
* adds connection_user to play_context
* maps remote_user to connection_user when connection is local
* maps ansible_socket in task_vars to module_args _ansible_socket if exists
2017-02-09 14:05:54 -05:00
Pilou 4e5c9c3d0d default and minimal callback: display warnings in a consistent manner (#21144)
* default/minimal callback: don't display warnings twice

* minimal callback: display warnings raised by MODULE_NO_JSON modules
2017-02-08 10:58:39 -05:00
Brian Coca 82337cf52a remove warning/deprecation keys after usage 2017-02-08 09:47:10 -05:00
Peter Sprygada d8e57cc60a starts cli if root user login is used for junos terminal (#20761)
fixes #20131
2017-02-07 10:57:44 +00:00
Brian Coca c86a17b7a0 refactoring async
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
2017-02-06 19:32:00 -05:00
Toshio Kuratomi f75ffe46db Make configparser imports python3 ready
the ConfigParser module was renamed to configparser in Python3.  Use
six.moves to import it so that the modules will function on Python3.
2017-02-03 21:15:34 -08:00
Toshio Kuratomi d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
2017-02-03 08:23:50 -08:00
Toshio Kuratomi 1df7d95cec Module utils default path (#20913)
* Make the module_utils path configurable
* Add a config value to define the path site module_utils files
* Handle module_utils that do not have source as an error
* Make an integration test for module_utils envvar working
* Add documentation for the ANSIBLE_MODULE_UTILS config option/envvar
* Add it to the sample ansible.cfg
* Add it to intro_configuration.
* Also modify intro_configuration to place envvars on equal footing with
  the config options (will need to document the envvar names in the
  future)
* Also add the ANSIBLE_LIBRARY use case from
  https://github.com/ansible/ansible/issues/15432 so we can close out
  that bug.
2017-02-02 17:48:53 -08:00
Toshio Kuratomi fee6e2953b Split on newlines when checking for prompt matches (#20945)
* Check for the prompt as a substring of the output

sudo sometimes spits out warnings to stdout before getting to the
password prompt.  Account for that when trying to match a password
prompt.

Fixes #20858
2017-02-02 14:31:11 -08:00
Toshio Kuratomi 62ba084003 Do not substitute ssh_exeuctable until we need to
We need to use ssh_executable instead of hardcoding ssh in the command
we run but we need to use "ssh" when we lookup the value of the
{command}_extra_args variable.  Do this by leaving binary as "ssh" and
only expanding when we place it into b_command.

Fixes #20862
2017-02-02 13:43:39 -08:00
Brian Coca 6ca5fb49c3 allow debug to be 'changed' (#20957)
* allow debug to be 'changed'

fixes #14430

* removed test that checked against this
2017-02-02 13:46:16 -05:00
René Moser 541a51ddf7 module_utils: implement deprecation warning for params (#20884)
* module_utils: implement deprecation warning for params

* rename deprecated_version to removed_in_version

* fix pep8 E121
2017-02-01 13:00:22 -05:00
jctanner ac78347f2b Use a -short- custom hash for controlpersist path by default (#20843)
* A method to validate and alter the ssh control path automatically.
* First tries %C to use the shortened hash
* On further failure, it removes section by section from the original path
* Fix hostname
* Implement bcoca's suggested changes
* Remove unused option
* Remove unused class var
* Use to_string to avoid unicode error
* Switch from to_text to to_bytes
* Update the example config for the new controlpath feature
2017-02-01 10:39:40 -05:00
Matt Clay cdc56afe01 PEP 8 fixes. 2017-01-31 15:18:24 -08:00
Brian Coca 16615811b4 clean up empty warnings/deprecations from modules 2017-01-31 18:13:26 -05:00
Samuel Boucher 488acc750f Feature/lookup keyring (#18379)
* Add keyring lookup

* Better error control

* Update keyring.py

* Update keyring.py

typo
2017-01-31 17:26:19 -05:00
Matt Clay 524e5d2c39 PEP 8 cleanup. 2017-01-31 10:16:59 -08:00
Toshio Kuratomi 5c38f3cea2 Combine jimi-c and bcoca's ideas and work on hooking module-utils into PluginLoader.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.

Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).

Rename snippet* variables to module_utils*

Add a small number of unittests for recursive_finder

Add a larger number of integration tests to demonstrate that
module_utils is working.

Whitelist module-style shebang in test target library dirs

Prefix module_data variable with b_ to be clear that it holds bytes data
2017-01-31 09:41:24 -08:00
Doug Bridgens 65f561e496 added ldap auth capability to hashi_vault plugin (#20244)
* added ldap authentication capability
2017-01-31 11:25:50 -05:00
Brian Coca 7e6758873c added warnings list to module and autoadd
added better way of adding warnings to return data
backwards compatible if warnings key already exists
added deprecations made iface more generic
changed to enforce type per item
added logging of warnings/deprecations
also display deprecations by default
2017-01-31 10:03:09 -05:00
Matt Clay cb76200c7d PEP 8 E111 & E114 cleanup. (#20838) 2017-01-30 15:01:47 -08:00
Dag Wieers bbaab6ee5b Show remote host in error message. (#20823)
* Show remote host in error message.

* Typos :-/
2017-01-30 11:16:13 -05:00
hloeffler 8b14935521 add META Output for -vv, better than nothing 2017-01-30 09:14:50 -05:00
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
2017-01-29 07:28:53 +00:00
Matt Clay e2c0b375d3 PEP 8 cleanup. (#20790)
* PEP 8 E115 cleanup.
* PEP 8 E131 cleanup.
2017-01-28 01:39:40 -08:00
Matt Clay d0d1158c5e PEP 8 cleanup. (#20789)
* PEP 8 E703 cleanup.
* PEP 8 E701 cleanup.
* PEP 8 E711 cleanup.
* PEP 8 W191 and E101 cleanup.
2017-01-28 00:12:11 -08:00
Matt Clay d913f69ba1 PEP 8 W291 whitespace cleanup. 2017-01-27 17:08:02 -08:00
Matt Clay 95789f3949 PEP 8 whitespace cleanup. (#20783)
* PEP 8 E271 whitespace cleanup.
* PEP 8 W293 whitespace cleanup.
* Fix whitespace issue from recent PR.
2017-01-27 15:45:23 -08:00
Matt Clay 63b1e0c277 Fix infrequent PEP 8 issues. 2017-01-27 14:06:21 -08:00
Peter Sprygada 6c89c587cc refactors eos_config module to use network_cli (#20741)
* update eos_config to use eapi exclusively and remove cli transport
* add unit test cases for eos_config
* updates action plugin to handle both eapi and network_cli connections
2017-01-27 16:23:18 -05:00
Peter Sprygada e8a00377ae additional exception handling
* catches TypeError when trying to load json data
2017-01-26 22:59:32 -05:00
Brian Coca 607ef0ce80 only ignore become for making tmpdir
(cherry picked from commit 65ee9d2e4629cbb31178b54831612846ca7b3b69)
2017-01-26 15:18:32 -05:00
Brian Coca 0a8b856102 fix improper setting become user for tmpdir
fixes #20706
2017-01-26 14:39:33 -05:00
Peter Sprygada 43d0505bde fixes issue in network_cli where _connected wasn't set (#20708)
Adds the code to set and unset the _connected propery in network_cli
on calls to open() and close()
2017-01-26 14:00:08 -05:00
Brian Coca 9e0fd313d7 fix powershell mkdtemp 2017-01-25 13:19:10 -05:00
Brian Coca 10fa2cd0ef make sure tmpdir resolvs user dirs (#20486)
* make sure tmpdir resolvs user dirs

fixes #20332
supercedes #20484

* typo fix
2017-01-25 13:09:36 -05:00
Peter Sprygada 8ddffcb1a6 new connection plugin netconf (#20636)
* adds connection plugin for creating modules that use netconf
* adds basic unit test cases for connection plugin
2017-01-25 10:15:26 -05:00
John R Barker e1b1df3cb1 Detect VyOS based on /etc/issue (support 1.0.5) (#20221)
* Detect VyOS based on /etc/issue (support 1.0.5)

VyOS didn't contain `vyos` in `/proc/version`

* Update vyos.py
2017-01-25 14:47:56 +00:00
jonathanbouvier 8257ee3e25 Fix issue with cc and bcc getting added to mail if unset (#20610) 2017-01-25 11:47:37 +00:00
Matt Davis 06353c055a winrm managed kinit (#20416) 2017-01-24 15:04:11 -08:00
Peter Sprygada 83fe2170ff updates ensure_connect decorator function
only calls _connect() if _connected is False
2017-01-24 09:52:20 -05:00
Nathaniel Case b3662fdad1 vyos_command 2.3 (#18994)
* Peter's new module_utils/vyos
* Update vyos_command
* Restore `set terminal length $ANSIBLE_VYOS_TERMINAL_LENGTH` in vyos
* vyos_command tests
* Remove provider mentions from vyos_command
* Extend get_config to take (and cache) various `show configuration` commands
2017-01-23 15:30:19 -05:00
David Barroso 61243f0d99 First release of the callback selective.py (#17309) 2017-01-23 11:54:47 -08:00
Marcos Diez 1839c8699d fix code typo (#20478) 2017-01-19 16:47:01 -08:00
Brian Coca 221771c0c9 fixed fact gathering verbosity hint
also removed redundant cleanup in normal plugin as module execution already removes _ansible_* from results
fixes #20378
2017-01-19 15:36:15 -05:00
Matt Martz 537b3b75a6 Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes #20098 (#20362)
* Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes #20098

* Address some of the requested changes

* Quoting

* Print the python path and version

* Be less explicitly verbose, rely on implicit verbosity
2017-01-19 11:39:59 -08:00
Andrew Gaffney ac51266e8f Add pipeline-ish method using dd for file transfer over SSH (#18642) 2017-01-19 12:31:14 -05:00
Brian Coca 58b6027a75 remote user should rely on context, not direct var
this can cause issues with non-ssh connections and/or ansible_user
2017-01-19 12:15:45 -05:00
Brian Coca d8cefee68c include source file in error message 2017-01-18 22:24:35 -05:00
Matt Davis 883f7ed197 ensure win_reboot warnings value is a list 2017-01-16 14:28:18 -08:00
Matt Davis c8970fc4e6 Merge branch 'win_reboot-fix' of git://github.com/dagwieers/ansible into HEAD 2017-01-16 14:11:27 -08:00
Matt Davis b773ae4291 Preserve exit code in winrm exec (#20166)
Raw winrm exec discards the exit code from external processes- this change preserves the exit code if present.
2017-01-16 11:21:48 -08:00
Peter Sprygada efda0323be fixes the timeout source in network_cli (#20302)
Timeout should be taken from the play context not the constants.  this change
updates network_cli to source timeout from the correct place
2017-01-16 09:15:46 -05:00
Peter Sprygada 2c197343f3 adds timeout check when network_cli run without persistence (#20299)
* checks if signal hander is set and sets it if not (will be set if coming
  from ansible-connection)
* will now timeout long running commands based on DEFAULT_TIMEOUT setting
2017-01-16 08:32:45 -05:00
Brian Coca 08ef0aee25 removed bad iteration from execute meta (#19958)
* removed bad iteration from execute meta

most of the tasks should not be iterated over, others needed to include unreachable hosts
fixes #19673

* corrected host var
2017-01-13 14:51:39 -05:00
Brian Coca 1880027da1 better cleaning of module return, also warn
disallow any internal _ansible_ vars and also warn when those or any on the restricted list are attempted
harden and parameterize key cleaning
2017-01-13 12:12:56 -05:00
Peter Sprygada fec773a2b4 better error handling and log messages in network_cli (#20204) 2017-01-12 19:31:35 -05:00
Brian Coca 8c6b5621f8 deal with remote_src and tmp dirs properly
fixes #20128
2017-01-12 14:10:11 -05:00
Brian Coca 74421f42e1 ansible doc does not need plugin deprecation error 2017-01-12 13:09:15 -05:00
Brian Coca e35a757ee7 fixed typo 2017-01-11 21:11:12 -05:00
Toshio Kuratomi eeebd51f21 Rename the type filter to type_debug
Because we add the names of all filters to the callable whitelist used
by safe_eval, adding a filter named type makes it so code calling "type()"
gets eval'd.  We can't think of a way to exploit this but it's
sufficiently sketchy that we're renaming it in case someone smarter than
us can think of a problem.
2017-01-11 14:48:21 -08:00
Jamie Lennox 27d218f85d Don't use rsync-path in synchronize with docker
When you become: with synchronize and docker it sets the rsync-path to
"sudo rsync" to launch rsync on the server as root. Unfortunately due to
docker exec doing stricter argument parsing than ssh this fails to
launch rsync on the server and the sync fails.

For docker though we don't need to launch rsync with sudo we can simply
docker exec -u <user> and rsync as normal to get around the problem.

Closes #20117
2017-01-11 14:01:11 -08:00
Dag Wieers 239e2ae7e9 win_reboot: survive an already scheduled shutdown
These changes ensure win_reboot can survive an already scheduled
shutdown by pre-empting it.

This fixes #19982
2017-01-11 00:07:30 +01:00
Dag Wieers b7594070b3 win_reboot: display message before rebooting (#19986)
Show a message to users when rebooting the system.

TODO: Look if this message is logged in the EventLog properly, if not,
fix that :-)
2017-01-10 13:59:39 -08:00
Adrian Likins cf7b1108ff Fix 'No module named network' net_template error (#20060) 2017-01-09 23:45:52 -05:00
Ryan Brown a2b4ad9da0 (docs) Remove/update mentions of `git submodule` in docs and error me… (#19941)
* (docs) Remove/update mentions of `git submodule` in docs and error messages.

* Remove parenthetical per @dharmabumstead
2017-01-09 12:11:39 -05:00
Marius Gedminas 0a7f2c202b Improve --diff output when files lack trailing newlines
The behavior now matches GNU diff.

Fixes #14094.

Example of output before this change:

    TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
    changed: [ranka]
    --- before: /etc/cron.hourly/mg-healthchecks-dot-io
    +++ after: /tmp/tmpOTvXTw
    @@ -1,2 +1,2 @@
     #!/bin/sh
    -curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null+curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null

after this change:

    TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
    changed: [ranka]
    --- before: /etc/cron.hourly/mg-healthchecks-dot-io
    +++ after: /tmp/tmpOTvXTw
    @@ -1,2 +1,2 @@
     #!/bin/sh
    -curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
    \ No newline at end of file
    +curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null

The added unit tests contain more examples.

This commit also takes care to avoid "no newline at EOF" warnings when
no_log is in effect, and also when modules return dicts rather than
strings.  (It also removes trailing whitespace from using json
serialization when diffing dicts, because I hate trailing whitespace in
Python source files, even if they're test files.)
2017-01-09 11:49:03 -05:00
James Cammarata a6fff93967 Fixing security bugs for CVE-2016-9587 2017-01-09 10:43:03 -06:00
Peter Sprygada 258c6ada52 refactors ios_config to use network_cli plugin (#20042)
* updates the ios_config module to use the network_cli plugin
* updates the local action plugin to derive from network
* add unit test cases for ios_config
2017-01-09 11:19:25 -05:00
Peter Sprygada cba66dfedc update the _ios_template module to use the network_cli plugin (#19933)
* updates the deprecated ios_template module to use network_cli
* adds unit test cases for ios_template
* adds check for provider argument and displays warning message
2017-01-06 20:22:17 -05:00
Rene Moser 40c6191da6 win_template: fix key errors when --diff is used 2017-01-06 11:55:54 -05:00
Brian Coca 08e0f6ada5 allow modules to set custom stats (#18946)
can be per run or per host, also aggregate or not
set_stats action plugin as reference implementation
added doc stub
display stats in calblack
made custom stats showing configurable
2017-01-05 16:38:36 -05:00
Peter Sprygada 630d10a27a updates action plugins for network_cli connection (#19849)
* net_config now subclasses action plugin network
* net_template now subclasses action plugin network

This will break existing modules until those modules have been refactored.
2017-01-04 21:52:46 -05:00
Peter Sprygada 4937dd67e4 adds new feature to network_cli (#19848)
In some cases it is desirable to have a send only function that doesn't
wait for the response from the CLI (such as reloading a device).  This
adds a new key to the command json string sendonly that will
achieve this behavior.
2017-01-03 22:33:02 -05:00
Marcos Diez 024e40d5f4 new lookup module: mongodb (#15057)
* new lookup module: mongodb lookup

* fix versionadded for MongoDB Lookup

* tests should run again

* removed use of basestring

* we don't use iteritems anymore

* run tests again

* run tests again2

* run tests again3

* run tests again4
2017-01-03 11:40:02 -05:00
Alexander Gubin fce9f4f679 shuffle filter: added optional 'seed' parameter 2017-01-03 11:08:36 -05:00
Matt Davis f96ac8bff0 fix multiple handler notifications (#19655)
Fixes #19647
Adds integration test to catch multiple handler notifications
2016-12-22 21:33:24 -08:00
James Cammarata 78d4f6bbc1 Removing print debug statements 2016-12-22 19:14:20 -06:00
Tom Booth a37bc3788b runner_on_async_failed missing arg
missing and argument from CallbackModule
2016-12-22 16:55:46 -08:00
Timur Vasyunin 5365b81d14 Fix for import 'display' in lib/ansible/plugins/lookup/filetree.py 2016-12-22 15:51:39 -08:00
slohse 27cce305e6 iocage connector that wraps around jail connector. (#17986)
Added iocage connector that extends the jail connector. Uses iocage to translate iocage tags or UUIDs/partial UUIDs to the actual jail name and then uses the jail connector for actual functionality.
2016-12-22 15:34:47 -08:00
azenk f2458140f6 Lastpass lookup plugin (#16285)
This plugin can be used with the lpass cli interface for lastpass.
[lastpass-cli](https://github.com/lastpass/lastpass-cli)

Example:
Add a lookup to your playbooks/variables somewhere:
```
some_variable: "{{ lookup('lastpass','Some Lastpass entry name or ID', field='username') }}"
```
Usage:
* start a lpass session prior to using ansible
* run ansible
* logout when finished

```
lpass login user@domain.com
ansible-playbook foo.yml
lpass logout
```
2016-12-22 14:47:33 -08:00
Sorin Sbarnea 4ebd763de0 py3 compatibility fix reported on #17038 (#19569)
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2016-12-22 11:56:46 -08:00
James Cammarata d70d279c4e Also fix default callback to use ignore_errors param
Rather than the value in the task, which may not be templated.

Related to #18289
2016-12-21 11:37:06 -06:00
James Cammarata dd0257b995 Template "original_task" fields in _process_pending_results
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.

This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.

Fixes #18289
2016-12-21 10:09:06 -06:00
Peter Sprygada f831ef692c update signature in network plugin (#19545)
Fix call to _update_module_args() to include module_name in the args
2016-12-20 07:07:42 -05:00
Dag Wieers 93cfe73a76 Report detailed error when internal remote functions fail
This is a redesign in how plugins call _remote_checksum().

- _remote_stat() has been modified to report the real error as
  AnsiblError
- Action plugin **unarchive** calls _remote_stat() directly instead of
   _remote_checksum()
- Action plugin **unarchive** also handles the exceptions directly
- Ensure get_exception() returns native text

Two other action plugins, **template** and **fetch**, also do a remote checksum.
In **template** we already call _remote_stat(), just like we now do for
unarchive, in **fetch** we do call _remote_checksum() and we make the
exact same mistake as the unarchive plugin. So that one could use a
redesign as well.

This fixes #19494

Before:
```
[dag@moria ansible.testing]$ ansible-playbook -v test137.yml
Using /home/dag/home-made/ansible.testing/ansible.cfg as config file

PLAY [localhost]
******************************************************************************************************

TASK [unarchive]
******************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"python isn't present on the system.  Unable to compute checksum"}

PLAY RECAP
******************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0
failed=1
```

After:
```
[dag@moria ansible.testing]$ ansible-playbook -v test137.yml
Using /home/dag/home-made/ansible.testing/ansible.cfg as config file

PLAY [localhost]
*************************************************************************************************************

TASK [unarchive]
*************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"Failed to get information on remote file (/tmp/): sudo: unknown user:
foobar\nsudo: unable to initialize policy plugin\n"}

PLAY RECAP
*******************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0
failed=1
```
2016-12-19 21:04:48 -05:00
Peter Sprygada c546c12b60 fixes incorrect module name for action plugin (#19535)
This fixes a bug introduced in the action plugin for setting the
module name value for logging
2016-12-19 16:54:31 -05:00
Chris Houseknecht 71685b3258 Update module_args['rsync_opts'] when connection type is docker (#19531) 2016-12-19 16:08:31 -05:00
Peter Sprygada 8f97ddd98a removes network os guess static method (#19467)
For devices that do not support mutliplexing, we cannot automatically
determine the network os.  This removes the os guess static method
from the terminal plugin.  For this devices, the network_os
value must be configured
2016-12-19 07:47:29 -05:00
Dag Wieers 39c9c6b942 Ensure that command/shell errors are displayed
This fix ensures that if there are specific module errors (in our case
the python interpreter was not found) then command and shell returns a
proper error.

It also fixes a few other imperfections that we noticed during
troubleshooting:

- Return the real RC if it were available
- Improve a dictionary evaluation using .get()
- Return an RC of -1 if it is unknown (instead of returning 0)

This fixes #18846
2016-12-16 21:46:02 -05:00
Nathaniel Case 7900319fc3 Assorted python3 fixes for network code. (#18777) 2016-12-15 16:25:03 -05:00
Peter Sprygada ad99d52679 adds feature to allow connection to set action plugin (#18762)
Connection plugin can define default action plugin to use by providing
action_handler instance variable.  This will override the default
action plugin normal
2016-12-15 15:47:39 -05:00
Peter Sprygada 0cef38cf02 adds feature to all modules to be run locally (#18763)
* adds new error AnsibleModuleExit to handle module returns
* adds new action plugin network for attaching connection to network modules
* adds new shared module local to receive connection
* splits out function to update task_args with common updates

This commit provides a mechansim for running local modules that require
a connection object for interative commands tyically implemented for
network devices.  It provides a way to locally import modules (post fork)
and run them using exception handling to exit.
2016-12-15 15:47:29 -05:00
Brian Coca f550b4541f bad/missing fact plugin is a warning (#19350) 2016-12-14 14:25:19 -05:00
Brian Coca ed933421fe correct template lookup path
now all paths get 'templates/'
2016-12-14 12:25:08 -05:00
Andrea Tartaglia 59227d8c31 Removed dict.iteritems() in several other files.
This is for py3 compatibility #18506
2016-12-14 08:01:45 -08:00
Adrian Likins 6c4f554f5a Provide slightly better msg on fact cache error (#18759)
If the configured fact_cache plugin (fact_caching config)
fails, raise a fatal error instead of failing mysteriously
later.

Fixes #18751
2016-12-13 14:34:58 -05:00
Carlos E. Garcia 0b8011436d minor spelling changes 2016-12-13 13:51:13 -05:00
Clark Boylan a65e34ce77 Fix synchronize retries (#18535)
* Fix synchronize retries

The synchronize module munges its task args on every invocation of
run(). This was problematic because the munged data was not fit for use
by a second pass of the synchronize module. Correct this by using a copy
of the task args on every invocation of run() so that the original args
are not affected.

Local testing using this playbook seems to confirm that things work as
expected:

  - hosts: all
  tasks:
    - delay: 2
      register: task_result
      retries: 1
      until: task_result.rc == 0
      synchronize:
        dest: /tmp/out
        mode: pull
        src: /tmp/nonexistent/

fixes #18281

* Update synchroncization fixture assertions

When we started operating on a copy of the task args the test assertions
were no longer asserting things about the munged state but of the
pristine state. Convert the copy of task args to a class member so that
it can be compared against later in testing and update the assertions to
check this munged copy.
* Shuffle objects around for cleaner testing

Attach the temporary args dict to the task rather than the action as
this makes updating the existing tests cleaner.
2016-12-12 13:33:30 -08:00
Chris Houseknecht 65491cc839 For docker add --blocking-io only when missing (#19171) 2016-12-10 10:51:03 -05:00
Chris Houseknecht b02491ba70 Add --blocking-io option when docker connection (#19140) 2016-12-10 09:20:35 -05:00
Peter Sprygada dc23667cc2 add back reverted change to network_cli (#18761)
This adds back the change to the network_cli plugin.  Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.

The issue was due to the addition of the decorator that will call
_connect() when there is no connection.  The _connect() method should
have been mocked in the test case.  This commit fixes the test
case as well

Change was originally reverted in c414ded69a
2016-12-05 21:42:09 -05:00
Toshio Kuratomi c414ded69a Revert "minor updates to network_cli connection plugin (#18742)"
This reverts commit 8e375913b0.

This breaks testing as the PlayContext is not mocked out for the test.
Temporarily reverting.
2016-12-05 10:54:42 -08:00
Peter Sprygada 8e375913b0 minor updates to network_cli connection plugin (#18742)
* removes superfluous timeout kwargs from open_shell()
* cleans up play_context become check
* adds check for ssh session and calls _connect() if needed
2016-12-04 22:50:01 -05:00
Peter Sprygada 8e562018ed adds py3 compat changes to network_cli plugin (#18735)
now calculates StringIO receive buffer and if received data is less than
buffer size, resets offset to 0
2016-12-03 20:05:33 -05:00
Jesse Keating 8e38f7475f Do not set docker use to None. (#18706)
The user variable defaults to None, and was being passed in as a user
named None. This was breaking rsync unless a specific user was set.

Fixes 16306
2016-12-02 10:29:24 -05:00
Benoît Allard 56dcf2cc04 with_sequence: pass AnsibleError through
The parsing methods try as hard as possible to generate meaningful error messages that are all ignored and immediately overwritten by a new AnsibleError instance. Better use the original one instead.
2016-11-30 21:26:23 -08:00
Benoît Allard 7db4ed02ee with_sequence: Fix indentation
This doesn't need to run for every parameters. Once is enough.
2016-11-30 21:13:13 -08:00
Alberto Murillo 7542dae26b Fix fetching files with scp (#18673)
Commit ec2521f intended to fix the scp command to fetch files
from a remote machine but it has src and dest swapped.

This change correctly treats src as the location in the remote machine
and dest as the location in the local machine.

Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
2016-11-30 20:10:49 -08:00
Peter Sprygada 8137c7207d adds feature to try to auto determine network_os (#18674)
This updates the network_cli connection plugin to attempt to automatically
determine the remote device os.  The device network os discovery can
be overridden by setting the ansible_network_os value.
2016-11-30 16:28:47 -05:00
Yannig 900b3ffcba Implement docker support for synchronize module. (#18145)
* Implement docker support for synchronize module.

Note : you need rsync installation on your docker container.
Have a look at https://github.com/ansible/ansible/issues/16306 for more details.
Support Ansible options for remote access.

* Give user name to docker command.
2016-11-30 15:56:27 -05:00
Virgil Dupras bf48383610 Fix regression in jinja2 include search path (#18617)
* Fix regression in jinja2 include search path

Since commit 3c39bb5, the 'ansible_search_path' variable is used to set
jinja2's search path for {% include %} directives. However, this path is
the the proper one because our templates live in 'templates' subdirs in
our search path.

This is a regression because previously, our include search path would
include the dirname of the currently interpreted file, which worked most
of the time.

fixes #18526

* Fix template lookup search path

Improve fix in commit c96c853 so that the search path contain both
template-suffixed paths as well as original paths.

ref PR #18617

* Add integration test for template lookups

Tests regression at #18526

This test fails on current devel branch and succeeds on PR #18617
2016-11-29 16:23:06 -05:00
Brian Coca ca1514cf2a unified boolean function
optimized boolean function
fixes #17815
2016-11-29 12:34:30 -05:00
Adrian Likins 83676e9b40 Fix 'cancled' misspelling in --step debug 2016-11-29 09:48:19 -05:00
Peter Sprygada 9aa8547016 adds two new plugins that use ansible-connection for persistence (#18572)
* adds new connection plugin `network_cli` which builds on paramiko
* adds new plugin `terminal` used for manipulating network_cli terminals
* adds new field to play_context `network_os` settable as ansible_network_os

This commit adds the plugins necesary to establish a persistent cli connection
to network devices of ssh.  It builds on the paramiko connection plugin
to create a shell environment that will persistent through ansible-connection.
The `newtork_cli` plugin then uses the network_os in the instance of
PlayContext to load the appropriate network OS environment plugin for
handling opening and closing of shells as well as privilege escalation.
2016-11-28 12:49:40 -05:00
Peter Sprygada 7df5a0abd0 adds config option to auto add keys when using paramiko (#18598)
* updates paramiko_ssh to auto add keys
* updates constants with new config options

This commit adds a new feature that will allow paramiko to automatically
accept and save a host ssh key.  This feature is controlled by the
`host_key_auto_add` config setting in the paramiko section.  The default
is False to maintain current functionality.  It also includes a new
setting `look_for_keys` with the default to False for maintaining current the
current setting.
2016-11-28 11:31:12 -05:00
Alberto Murillo ec2521f6af Fix ssh plugin to correctly fetch files when using scp (#18614)
Fetch module uses fetch_file() from plugin/connection/ssh.py to
retrieve files from the remote hosts which in turns uses
_file_transport_command(self, in_path, out_path, sftp_action) being
sftp_action = 'get'

When using scp rather than sftp, sftp_action variable is not used
and the scp command is formed in a way that the file is always
sent to the remote machine

This patch fixes _file_transport_command() to correctly form the scp
swaping src and dest if sftp_action is 'get'

Bug introduced at 8e47b9b
Fixes #18603

Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
2016-11-26 16:55:38 -06:00
Brian Coca 778c983ef9 ansible_playbook_python (#18530)
* ansible_playbook_python

fixes #18471

* fix tests

* removed dupe
2016-11-23 16:30:46 -05:00
Matt Clay d61b2ed0a3 Support script interpreters for async_wrapper. (#18592) 2016-11-22 16:50:08 -08:00
Toshio Kuratomi 255a5b5d75 Fix the Solaris POSIX acl fix
For setfacl on Solaris we need to specify permissions like r-x.
For chmod, we need to specify them as rx (r-x means to make the file
readable and *not* executable)
2016-11-22 12:36:43 -08:00
koralsky 4e194d71bd import reduce from six label:python3 (#18561)
* import 'reduce' from six
* import reduce in facts fix
2016-11-22 11:22:24 -08:00
Michael Noseworthy bb5d8fb476 Fix unicode handling in fixup_perms2 errorhandling (#18565)
The _fixup_perms2 method checks to see if the user that is being sudo'd
is an unprivileged user or root. If it is an unprivileged user, some
checks are done to see if becoming this user would lock the ssh user out
of temp files, among other things. If this check fails, an error prints
telling the user to check the documentation for becoming an unprivileged
user.

On some systems, the stderr prints out the unprivileged user the ssh
user was trying to become contained in smartquotes. These quotes aren't
in the ASCII range, and so when we're trying to call `str.format()` to
combine the stderr message with the error text we get a
UnicodeEncodeError as python can't coerce the smartquotes using the
system default encoding. By calling `to_native()` on the error message
we can ensure that the error message is a native string for the
`Exception` handling, as `Exception` messages need to be native strings
to avoid errors (byte strings in python2, and text strings in python3)

Fixes: #18444
2016-11-22 11:19:47 -08:00
Nathaniel Case 8d0418f7c1 net_template doesn't need BOOLEANS
Note #17815
2016-11-22 11:49:21 -05:00
Toshio Kuratomi d90638ad40 Fix setfacl for Solaris with POSIX acl support.
Tested on Linux and freebsd.

Fixes #17919
2016-11-21 13:19:41 -08:00
James Cammarata 7e2305f953 Cache dynamically included blocks for later lookup via uuid
Fixes #18357
2016-11-21 13:02:24 -06:00
James Cammarata 26ec2ecfce Adding a persistent connection utility 2016-11-21 12:35:27 -06:00
Andrea Tartaglia 62697ad77f Replaced iterkeys with 'for key in dict' for #18507 (#18538) 2016-11-21 07:18:52 -08:00
jamessewell b91d4d884d Moved the _inventory.clear_group_dict_cache() from creating a group w… (#17766)
* Moved the _inventory.clear_group_dict_cache() from creating a group which doesn't exist, to adding members to the group.

* Update __init__.py

Update to use changed: block to catch all changes for cache clear as suggested
2016-11-18 13:27:17 -06:00
Andrew Haines 200d6bdb23 Default include_role results to empty list in linear strategy plugin
Fixes #18544.

When a loop is over an empty list, the result is set to

    {'skipped_reason': u'No items in the list', 'skipped': True, 'changed': False}

which means that accessing `hr._result['results']` throws a `KeyError`.
2016-11-18 12:13:53 -05:00
Brian Coca aff5d9160a disable 'dense' callback in py26 2016-11-18 11:41:56 -05:00
James Cammarata 937d872f4b Return failed instead of raising an error when an include_vars file is missing
Fixes #18508
2016-11-18 09:28:44 -06:00
Dag Wieers b7425b59ee Adapt callback functions to changes in 13f3cbaf3b 2016-11-17 19:30:42 -05:00
Dag Wieers 0036108c59 Implement destructor to ensure newline and terminal attributes
Also fixed a typo
2016-11-17 19:30:42 -05:00
Dag Wieers 2a8914e496 Reset VT100 attributes in case we get unexpected output 2016-11-17 19:30:42 -05:00
Dag Wieers cd6f577d91 Rename ansi to the more correct name vt100 2016-11-17 19:30:42 -05:00
Dag Wieers 893dc81b5e Implement our own detailed task output
Also:
- Streamline ANSI calls
- Implement v2_playbook_on_include
2016-11-17 19:30:42 -05:00
Dag Wieers 221a4e7014 Ensure we end with a newline in normal mode 2016-11-17 19:30:42 -05:00
Dag Wieers 7148023f59 Small fix so diff output starts on a new line 2016-11-17 19:30:42 -05:00
Dag Wieers 9a507b1f07 Don't try to be smart with cmd output ! 2016-11-17 19:30:42 -05:00
Dag Wieers 84e0420389 Make everything work magically !
- Support item-loops correctly
- Support notification handlers
2016-11-17 19:30:42 -05:00
Dag Wieers 589953c79b Assorted improvements
- Better switch between *dense* and *default*
- Reimplement C.COLOR* out of necessity (help!)
- Make verbose output more dense (clean up result)
- Implement our own dumper
- Improve delegation support
2016-11-17 19:30:42 -05:00
Dag Wieers 223c0011e0 Assorted fixes
- Implement delegate support
- Implement preliminary handler support
- Implement our own recap/summary (still fails)
2016-11-17 19:30:42 -05:00
Dag Wieers 1bf7e22756 Implement support for itemized tasks 2016-11-17 19:30:42 -05:00
Dag Wieers 14cfb2b230 Preliminary support for items, WIP 2016-11-17 19:30:42 -05:00
Dag Wieers 67c57a9b6b Improve readability 2016-11-17 19:30:42 -05:00
Dag Wieers 326bb24a89 Various improvements, now supports different verbosity levels 2016-11-17 19:30:42 -05:00
Dag Wieers fe6e4f8286 New "dense" callback plugin
The goal for the "dense" output is to only show changes and failures on-screen (the Unix-way).
However, since we still want to have a sense of progress, we use terminal capabilities to display progress.

 - On screen there should only be relevant stuff
    - How far are we ? (during run, last line)
    - What issues occured
    - What changes occured
    - Diff output

 - If verbosity increases, act as default output
   So that users can easily switch to default for troubleshooting

 - Leave previous task output on screen
   - If we would clear the line at the start of a task, there would often
     be no information at all
   - We use the cursor to indicate where in the task we are.
     Output after the prompt is the output of the previous task
   - Use the same color-conventions of Ansible

This is still a work in progress.
It was released to give a glimpse of what would be possible.

The Ansible callback mechanism currently does not have all the functionality we need to do this efficiently.
2016-11-17 19:30:42 -05:00
Brano Zarnovican 96ddd29c7e random filter: added optional 'seed' parameter
implements #15621
2016-11-17 15:02:06 -08:00
Toshio Kuratomi ed00741a01 Mcsalgado's change to use shlex.quote instead of pipes.quote (#18534)
* Replace pipes.quote for shlex_quote

* More migration of pipes.quote to shlex_quote

Note that we cannot yet move module code over.  Modules have six-1.4
bundled which does not have shlex_quote.  This shouldn't be a problem as
the function is still importable from pipes.quote.  It's just that this
has become an implementation detail that makes us want to import from
shlex instead.

Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
2016-11-17 13:18:29 -08:00
Brian Coca 5d043b65d3 fix for filter fix 2016-11-17 13:41:57 -05:00
Brian Coca 7c960d440f remove rsync path from returned facts 2016-11-17 13:20:46 -05:00
Dag Wieers 1ca4add91c Performance improvement using in-operator on dicts
Just a small cleanup for the existing occurrences.

Using the in-operator for hash lookups is faster than using .keys()
http://stackoverflow.com/questions/29314269/why-do-key-in-dict-and-key-in-dict-keys-have-the-same-output
2016-11-17 12:33:04 -05:00
James Cammarata ca5b361ad8 Reworking iterator logic regarding failed states during always
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.

This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.

Fixes #17983
2016-11-16 10:21:46 -06:00
Brian Coca afaec3da82 corrected service filtered option to singular 2016-11-16 10:12:47 -05:00
Brian Coca bd70397e24 always template when called from template (#18490)
* Have template action plugin call do_template

Avoids all the magic done for 'inline templating' for ansible plays.
renamed _do_template to do_template in templar to make externally accessible.
fixes #18192

* added backwards compat as per feedback
2016-11-15 15:16:46 -05:00
Brian Coca f4391d34e4 added alias to argumetns 'args' to blacklist
this should fix https://github.com/ansible/ansible-modules-core/issues/5584
2016-11-14 14:49:30 -05:00
Chris Church 534bd12ae9 Increment changed stat for a failed task if changed. (#18014) 2016-11-14 01:23:02 -06:00
James Cammarata 4f06a86161 Alternately track listening handlers by uuid if no name is set
Fixes #17846
2016-11-13 15:24:44 -06:00
James Cammarata 5b87951d6c Don't copy the parent block of TaskIncludes when loading statically
When loading an include statically, we previously were simply doing a
copy() of the TaskInclude object, which recurses up the parents creating
a new lineage of objects. This caused problems when used inside load_list_of_blocks
as the new parent Block of the new TaskInclude was not actually in the list
of blocks being operated on. In most circumstances, this did not cause a
problem as the new parent block was a proper copy, however when used in
combination with PlaybookInclude (which copies conditionals to the list of
blocks loaded) this untracked parent was not being properly updated, leading
to tasks being run improperly.

Fixes #18206
2016-11-11 08:09:43 -06:00
Ievgen Khmelenko 619f2fd210 ansible-logstash-callback (#18282)
* ansible-logstash-callback

* GPL v3 license preamble, ImportError

* Update logstash.py
2016-11-08 11:17:05 -05:00
James Tanner 20fb74b1b1 ini lookup: add 'default' to the list of parsed keys
Fixes #18369
2016-11-08 11:07:33 -05:00
Brian Coca 5dd195b52f restore play_hosts variables to not show removed
Also adds ansible_play_hosts_all with original list of hosts the play targeted
2016-11-08 10:51:27 -05:00
Toshio Kuratomi 6a3893c518 Remove direct calls to print and cleanup imports
All display of information should go through display instead of through print.
2016-11-07 17:12:36 -05:00
Adrian Likins dd0189839e Fix bug (#18355) where encrypted inventories fail 18355 (#18373)
* Fix bug (#18355) where encrypted inventories fail

This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes

The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).

So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().

The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.

Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.

_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.

Fixes #18355
2016-11-07 10:07:26 -05:00
Andrea Tartaglia b18263cf36 ANSIBLE_SSH_CONTROL_PATH_DIR option added (#18342)
* ANSIBLE_SSH_CONTROL_PATH_DIR option added

This removes the hardcoded value ( $HOME/.ansible/cp ) from ssh.py.
User is able to change the ControlPath directory ( the one that replaces %(directory)s ).

 Fixes #18325

* Added config option in ansible.cfg
2016-11-03 15:19:59 -07:00
Brian Coca 32a7b4ce71 only validate extensions when using dir loading
fixes #18223
2016-11-03 07:30:32 -07:00
Matt Clay 0d46805979 Clean up shebangs for various files.
- Remove shebangs from:
  - ini files
  - unit tests
  - module_utils
  - plugins
  - module_docs_fragments
  - non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.

Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.

A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
2016-11-02 17:00:27 -07:00
Samuel Boucher 911a602c79 Update syslog_json.py
Add SYSLOG_FACILITY environment variable to set syslog facility
2016-11-02 10:02:56 -04:00
Matt Martz cb1e3dab0d Add 'type' filter for display the underlying python type of a variable (#18242)
* Add 'type' filter for display the underlying python type of a variable

* Update playbooks_filters.rst

Minor copyedit.
2016-10-31 13:36:24 -07:00
jasdeep-hundal 679da00236 Fix OpenSSH-related ssh process exit race
Mitigate the effects of observing the ssh process still running
after seeing an EOF on stdout when using OpenSSH with
ControlPersist, since it does not close the stderr file descriptor
in this case.
2016-10-27 15:47:24 -07:00
Brian Coca 680cade77a simplified the code by removing repeats
(cherry picked from commit 84380b0ee4029212fc1637c008e07bb9958305c3)
2016-10-26 20:49:55 -04:00
jctanner 5502da3cf8 copy: Use the local file's mode for the argument if not explicitly given. (#17780)
* Use the local file's mode to for the argument if not explicitly given.

Fixes https://github.com/ansible/ansible-modules-core/issues/1124

* Fix octal mode for py3

* Implement preserve instead of null

* Remove duplicate line

* Update comment

* Use stat module per toshia's suggestion
2016-10-24 23:57:50 -04:00
Thomas Quinot 236c923c25 Filter out internal magic and connection variables from facts returns
Fixes #15925
2016-10-24 17:27:43 -05:00
Rene Moser e69d26270f handler: notify a handler by name _and_ listen
Before we only allowed either notify by name or listen and name had precedence.
2016-10-24 10:59:05 -04:00
Adrian Likins c0331d50dc Remove callback.CallbackBase._copy_result_exclude
Nothing seems to use this now.

Was added originally added in2d11cfab92f9d26448461b4bc81f466d1910a15e
but the code that used it was removed in
e02b98274b
2016-10-23 13:36:20 +02:00
Matt Robinson 692bfa872a Make bcrypt + passlib work in password_hash filter
If hashtype for the password_hash filter is 'blowfish' and passlib is
available, hashing fails as the hash function for this is named 'bcrypt'
(and not 'blowfish_crypt').  Special case this so that the correct
function is called.
2016-10-23 01:46:05 +02:00
Ssawa 8e47b9bc70 Handle 'smart' scp_if_ssh option for fetch (#18125) 2016-10-21 09:59:56 -04:00
Toshio Kuratomi 6d9f780937 Now that we convert salt inside of do_encryptas needed, keep salt as text type until then. 2016-10-20 22:49:30 -07:00
Brian Coca 99220a5f6c properly propagate loop vars
fixes #17966
2016-10-18 15:27:16 -04:00
James Cammarata 5be2a3a9e0 Break out of linear strategy if all hosts are failed and the result is not OK
Fixes #18021
2016-10-17 16:00:04 -05:00
Brian Coca 9bdde13126 fixes to ansible_search_path
now gets basedir (no need to frontload as dwim already does that)
added comment about basedir to search_path usage to avoid dupes
2016-10-17 11:21:46 -04:00
Toshio Kuratomi f24c10c32b Fixes to handle non-ascii become passwords
Fixes for non-ascii passwords on
* both python2 and python3,
* local and paramiko_ssh (ssh tested working with these changes)
* sudo and su

Fixes #16557
2016-10-15 16:25:19 -07:00
Toshio Kuratomi efc5dac52c Fix become password using non-ascii for local connection
Fixes #18029
2016-10-15 11:26:17 -07:00
James Cammarata 6bdcb3a392 Make sure free strategy is returning proper TQM constants
Fixes #18023
2016-10-14 09:23:35 -05:00
Brian Coca b169a61c20 toggle missing handler errors/warnings via config 2016-10-13 16:54:02 -04:00
Brian Coca cdb5a222c5 restored 'results' filters
tests do not work the same, restoring old filters to keep backwards compat
tests now only implment the new normalized 'tense'
2016-10-13 13:21:40 -04:00
Bruno Rocha b06fb2022c Fix unbound method call for JSONEncoder (#17970)
* Fix unbound method call for JSONEncoder

The way it is currently it will lead to unbound method error

```python
In [1]: import json

In [2]: json.JSONEncoder.default('object_here')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-872fdacfda50> in <module>()
----> 1 json.JSONEncoder.default('object_here')

TypeError: unbound method default() must be called with JSONEncoder instance as first argument (got str instance instead)

```

But what is really wanted is to let the json module to raise the "is not serializable error" which demands a bounded instance of `JSONEncoder()`

```python
In [3]: json.JSONEncoder().default('object_here')
---------------------------------------------------------------------------
TypeError: 'object_here' is not JSON serializable 

```


BTW: I think it would try to call `.to_json` of object before raising as it is a common pattern.

* Calling JSONEncoder bounded `default` method using super()
2016-10-11 08:31:53 -07:00
Pavlo Shchelokovskyy aa1ec8af17 Make interprocess polling interval configurable (#16560)
As recently there was back-and-forth with this hardcoded value
(0.001 -> 0.01 -> 0.005), obviousely the optimal value for it depends on
Ansible usage scanario and is better to be configurable.

This patch adds a new config option in DEFAULT section,
`internal_poll_interval`, with default of 0.001 corresponding to the
value hardcoded in Ansible v2.1.
This config option is then used instead of hardcoded values where
needed.

Related GH issue: 14219
2016-10-06 14:30:20 -05:00
James Cammarata e26bce5221 Sleep briefly while waiting for pending results to reduce CPU churn 2016-10-06 08:50:17 -05:00
Brian Coca d9d7e413a5 fixed storing of cwd 2016-10-04 14:24:45 -04:00
Brian Coca 125a8d3c65 no need for warnings in first_found 2016-10-03 20:23:33 -04:00