can be per run or per host, also aggregate or not
set_stats action plugin as reference implementation
added doc stub
display stats in calblack
made custom stats showing configurable
* net_config now subclasses action plugin network
* net_template now subclasses action plugin network
This will break existing modules until those modules have been refactored.
In some cases it is desirable to have a send only function that doesn't
wait for the response from the CLI (such as reloading a device). This
adds a new key to the command json string sendonly that will
achieve this behavior.
* new lookup module: mongodb lookup
* fix versionadded for MongoDB Lookup
* tests should run again
* removed use of basestring
* we don't use iteritems anymore
* run tests again
* run tests again2
* run tests again3
* run tests again4
Added iocage connector that extends the jail connector. Uses iocage to translate iocage tags or UUIDs/partial UUIDs to the actual jail name and then uses the jail connector for actual functionality.
This plugin can be used with the lpass cli interface for lastpass.
[lastpass-cli](https://github.com/lastpass/lastpass-cli)
Example:
Add a lookup to your playbooks/variables somewhere:
```
some_variable: "{{ lookup('lastpass','Some Lastpass entry name or ID', field='username') }}"
```
Usage:
* start a lpass session prior to using ansible
* run ansible
* logout when finished
```
lpass login user@domain.com
ansible-playbook foo.yml
lpass logout
```
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.
This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.
Fixes#18289
This is a redesign in how plugins call _remote_checksum().
- _remote_stat() has been modified to report the real error as
AnsiblError
- Action plugin **unarchive** calls _remote_stat() directly instead of
_remote_checksum()
- Action plugin **unarchive** also handles the exceptions directly
- Ensure get_exception() returns native text
Two other action plugins, **template** and **fetch**, also do a remote checksum.
In **template** we already call _remote_stat(), just like we now do for
unarchive, in **fetch** we do call _remote_checksum() and we make the
exact same mistake as the unarchive plugin. So that one could use a
redesign as well.
This fixes#19494
Before:
```
[dag@moria ansible.testing]$ ansible-playbook -v test137.yml
Using /home/dag/home-made/ansible.testing/ansible.cfg as config file
PLAY [localhost]
******************************************************************************************************
TASK [unarchive]
******************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"python isn't present on the system. Unable to compute checksum"}
PLAY RECAP
******************************************************************************************************
localhost : ok=0 changed=0 unreachable=0
failed=1
```
After:
```
[dag@moria ansible.testing]$ ansible-playbook -v test137.yml
Using /home/dag/home-made/ansible.testing/ansible.cfg as config file
PLAY [localhost]
*************************************************************************************************************
TASK [unarchive]
*************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg":
"Failed to get information on remote file (/tmp/): sudo: unknown user:
foobar\nsudo: unable to initialize policy plugin\n"}
PLAY RECAP
*******************************************************************************************************************
localhost : ok=0 changed=0 unreachable=0
failed=1
```
For devices that do not support mutliplexing, we cannot automatically
determine the network os. This removes the os guess static method
from the terminal plugin. For this devices, the network_os
value must be configured
This fix ensures that if there are specific module errors (in our case
the python interpreter was not found) then command and shell returns a
proper error.
It also fixes a few other imperfections that we noticed during
troubleshooting:
- Return the real RC if it were available
- Improve a dictionary evaluation using .get()
- Return an RC of -1 if it is unknown (instead of returning 0)
This fixes#18846
Connection plugin can define default action plugin to use by providing
action_handler instance variable. This will override the default
action plugin normal
* adds new error AnsibleModuleExit to handle module returns
* adds new action plugin network for attaching connection to network modules
* adds new shared module local to receive connection
* splits out function to update task_args with common updates
This commit provides a mechansim for running local modules that require
a connection object for interative commands tyically implemented for
network devices. It provides a way to locally import modules (post fork)
and run them using exception handling to exit.
* Fix synchronize retries
The synchronize module munges its task args on every invocation of
run(). This was problematic because the munged data was not fit for use
by a second pass of the synchronize module. Correct this by using a copy
of the task args on every invocation of run() so that the original args
are not affected.
Local testing using this playbook seems to confirm that things work as
expected:
- hosts: all
tasks:
- delay: 2
register: task_result
retries: 1
until: task_result.rc == 0
synchronize:
dest: /tmp/out
mode: pull
src: /tmp/nonexistent/
fixes#18281
* Update synchroncization fixture assertions
When we started operating on a copy of the task args the test assertions
were no longer asserting things about the munged state but of the
pristine state. Convert the copy of task args to a class member so that
it can be compared against later in testing and update the assertions to
check this munged copy.
* Shuffle objects around for cleaner testing
Attach the temporary args dict to the task rather than the action as
this makes updating the existing tests cleaner.
This adds back the change to the network_cli plugin. Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.
The issue was due to the addition of the decorator that will call
_connect() when there is no connection. The _connect() method should
have been mocked in the test case. This commit fixes the test
case as well
Change was originally reverted in c414ded69a
* removes superfluous timeout kwargs from open_shell()
* cleans up play_context become check
* adds check for ssh session and calls _connect() if needed
The parsing methods try as hard as possible to generate meaningful error messages that are all ignored and immediately overwritten by a new AnsibleError instance. Better use the original one instead.
Commit ec2521f intended to fix the scp command to fetch files
from a remote machine but it has src and dest swapped.
This change correctly treats src as the location in the remote machine
and dest as the location in the local machine.
Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
This updates the network_cli connection plugin to attempt to automatically
determine the remote device os. The device network os discovery can
be overridden by setting the ansible_network_os value.
* Implement docker support for synchronize module.
Note : you need rsync installation on your docker container.
Have a look at https://github.com/ansible/ansible/issues/16306 for more details.
Support Ansible options for remote access.
* Give user name to docker command.
* Fix regression in jinja2 include search path
Since commit 3c39bb5, the 'ansible_search_path' variable is used to set
jinja2's search path for {% include %} directives. However, this path is
the the proper one because our templates live in 'templates' subdirs in
our search path.
This is a regression because previously, our include search path would
include the dirname of the currently interpreted file, which worked most
of the time.
fixes#18526
* Fix template lookup search path
Improve fix in commit c96c853 so that the search path contain both
template-suffixed paths as well as original paths.
ref PR #18617
* Add integration test for template lookups
Tests regression at #18526
This test fails on current devel branch and succeeds on PR #18617
* adds new connection plugin `network_cli` which builds on paramiko
* adds new plugin `terminal` used for manipulating network_cli terminals
* adds new field to play_context `network_os` settable as ansible_network_os
This commit adds the plugins necesary to establish a persistent cli connection
to network devices of ssh. It builds on the paramiko connection plugin
to create a shell environment that will persistent through ansible-connection.
The `newtork_cli` plugin then uses the network_os in the instance of
PlayContext to load the appropriate network OS environment plugin for
handling opening and closing of shells as well as privilege escalation.
* updates paramiko_ssh to auto add keys
* updates constants with new config options
This commit adds a new feature that will allow paramiko to automatically
accept and save a host ssh key. This feature is controlled by the
`host_key_auto_add` config setting in the paramiko section. The default
is False to maintain current functionality. It also includes a new
setting `look_for_keys` with the default to False for maintaining current the
current setting.
Fetch module uses fetch_file() from plugin/connection/ssh.py to
retrieve files from the remote hosts which in turns uses
_file_transport_command(self, in_path, out_path, sftp_action) being
sftp_action = 'get'
When using scp rather than sftp, sftp_action variable is not used
and the scp command is formed in a way that the file is always
sent to the remote machine
This patch fixes _file_transport_command() to correctly form the scp
swaping src and dest if sftp_action is 'get'
Bug introduced at 8e47b9bFixes#18603
Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
For setfacl on Solaris we need to specify permissions like r-x.
For chmod, we need to specify them as rx (r-x means to make the file
readable and *not* executable)
The _fixup_perms2 method checks to see if the user that is being sudo'd
is an unprivileged user or root. If it is an unprivileged user, some
checks are done to see if becoming this user would lock the ssh user out
of temp files, among other things. If this check fails, an error prints
telling the user to check the documentation for becoming an unprivileged
user.
On some systems, the stderr prints out the unprivileged user the ssh
user was trying to become contained in smartquotes. These quotes aren't
in the ASCII range, and so when we're trying to call `str.format()` to
combine the stderr message with the error text we get a
UnicodeEncodeError as python can't coerce the smartquotes using the
system default encoding. By calling `to_native()` on the error message
we can ensure that the error message is a native string for the
`Exception` handling, as `Exception` messages need to be native strings
to avoid errors (byte strings in python2, and text strings in python3)
Fixes: #18444
* Moved the _inventory.clear_group_dict_cache() from creating a group which doesn't exist, to adding members to the group.
* Update __init__.py
Update to use changed: block to catch all changes for cache clear as suggested
Fixes#18544.
When a loop is over an empty list, the result is set to
{'skipped_reason': u'No items in the list', 'skipped': True, 'changed': False}
which means that accessing `hr._result['results']` throws a `KeyError`.
- Better switch between *dense* and *default*
- Reimplement C.COLOR* out of necessity (help!)
- Make verbose output more dense (clean up result)
- Implement our own dumper
- Improve delegation support
The goal for the "dense" output is to only show changes and failures on-screen (the Unix-way).
However, since we still want to have a sense of progress, we use terminal capabilities to display progress.
- On screen there should only be relevant stuff
- How far are we ? (during run, last line)
- What issues occured
- What changes occured
- Diff output
- If verbosity increases, act as default output
So that users can easily switch to default for troubleshooting
- Leave previous task output on screen
- If we would clear the line at the start of a task, there would often
be no information at all
- We use the cursor to indicate where in the task we are.
Output after the prompt is the output of the previous task
- Use the same color-conventions of Ansible
This is still a work in progress.
It was released to give a glimpse of what would be possible.
The Ansible callback mechanism currently does not have all the functionality we need to do this efficiently.
* Replace pipes.quote for shlex_quote
* More migration of pipes.quote to shlex_quote
Note that we cannot yet move module code over. Modules have six-1.4
bundled which does not have shlex_quote. This shouldn't be a problem as
the function is still importable from pipes.quote. It's just that this
has become an implementation detail that makes us want to import from
shlex instead.
Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.
This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.
Fixes#17983
* Have template action plugin call do_template
Avoids all the magic done for 'inline templating' for ansible plays.
renamed _do_template to do_template in templar to make externally accessible.
fixes#18192
* added backwards compat as per feedback
When loading an include statically, we previously were simply doing a
copy() of the TaskInclude object, which recurses up the parents creating
a new lineage of objects. This caused problems when used inside load_list_of_blocks
as the new parent Block of the new TaskInclude was not actually in the list
of blocks being operated on. In most circumstances, this did not cause a
problem as the new parent block was a proper copy, however when used in
combination with PlaybookInclude (which copies conditionals to the list of
blocks loaded) this untracked parent was not being properly updated, leading
to tasks being run improperly.
Fixes#18206
* Fix bug (#18355) where encrypted inventories fail
This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes
The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).
So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().
The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.
Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.
_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.
Fixes#18355
* ANSIBLE_SSH_CONTROL_PATH_DIR option added
This removes the hardcoded value ( $HOME/.ansible/cp ) from ssh.py.
User is able to change the ControlPath directory ( the one that replaces %(directory)s ).
Fixes#18325
* Added config option in ansible.cfg
- Remove shebangs from:
- ini files
- unit tests
- module_utils
- plugins
- module_docs_fragments
- non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.
Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.
A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
Mitigate the effects of observing the ssh process still running
after seeing an EOF on stdout when using OpenSSH with
ControlPersist, since it does not close the stderr file descriptor
in this case.
Brian Coca
Peter Sprygada
Marcos Diez
Alexander Gubin
Matt Davis
James Cammarata
Tom Booth
Timur Vasyunin
slohse
azenk
Sorin Sbarnea
Dag Wieers
Chris Houseknecht
Nathaniel Case
Brian Coca
Andrea Tartaglia
Adrian Likins
Carlos E. Garcia
Clark Boylan
Toshio Kuratomi
Jesse Keating
Benoît Allard
Alberto Murillo
Yannig
Virgil Dupras
Matt Clay
koralsky
Michael Noseworthy
Nathaniel Case
jamessewell
Andrew Haines
Brano Zarnovican
Chris Church
Ievgen Khmelenko
James Tanner
Samuel Boucher
Matt Martz
jasdeep-hundal