Using set_host_overrides() in the connection plugin to access the ssh
argument variables from the inventory didn't see group_vars/host_vars
settings, as noted earlier. Instead, we can set the correct values in
the PlayContext, which has access to all command-line options, task
settings, and variables.
The only downside of doing so is that the source of the settings is no
longer available in ssh.py, and therefore can't be logged. But the code
is simpler, and it actually works.
This change was suggested by @jimi-c in response to the FIXME in the
earlier commit.
Now we have the following ways to set additional arguments:
1. [ssh_connection]ssh_args in ansible.cfg: global setting, prepended to
every command line for ssh/scp/sftp. Overrides default ControlPersist
settings.
2. ansible_ssh_common_args inventory variable. Appended to every command
line for ssh/scp/sftp. Used in addition to ssh_args, if set above, or
the default settings.
3. ansible_{sftp,scp,ssh}_extra_args inventory variables. Appended to
every command line for the relevant binary only. Used in addition to
#1 and #2, if set above, or the default settings.
3. Using the --ssh-common-args or --{sftp,scp,ssh}-extra-args command
line options (which are overriden by #2 and #3 above).
This preserves backwards compatibility (for ssh_args in ansible.cfg),
but also permits global settings (e.g. ProxyCommand via _common_args) or
ssh-specific options (e.g. -R via ssh_extra_args).
Fixes#12576
Fixes another failing test.
(I don't want to do a global search/replace for 'basestring' because I
want to have unit tests covering each occurrence. When I run out of
existing failing tests, I'll try to write new ones.)
SSH pipelining can be a significant performance improvement, but it will
not work if sudoers is configured to requiretty. With this change, one
could have pipelining enabled in ansible.cfg, but use sudo to turn off
requiretty in a separate play (or task) where pipelining is disabled:
- hosts: foo
vars:
ansible_pipelining: no
tasks:
- lineinfile: dest=/etc/sudoers line='Defaults requiretty' state=absent
sudo_user: root
(Note that sudoers has a complicated syntax, so the above lineinfile
invocation may be too simplistic for production use; but the point is
that a separate play can do something to disable requiretty.)
* allow global no_log setting, no need to set at play or task level, but can be overriden by them
* allow turning off syslog only on task execution from target host (manage_syslog), overlaps with no_log functionality
* created log function for task modules to use, now we can remove all syslog references, will use systemd journal if present
* added debug flag to modules, so they can make it call new log function conditionally
* added debug logging in module's run_command
Due to the way we're now calculating delegate_to, if that value is based
on a loop variable ('item') we need to calculate all of the possible
delegated_to variables for that loop.
Fixes#12499
The event loop (even after it was brought into one place in _run in the
previous commit) was hard to follow. The states and transitions weren't
clear or documented, and the privilege escalation code was non-blocking
while the rest was blocking.
Now we have a state machine with four states: awaiting_prompt,
awaiting_escalation, ready_to_send (initial data), and awaiting_exit.
The actions in each state and the transitions between then are clearly
documented.
The check_incorrect_password() method no longer checks for empty strings
(since they will always match), and check_become_success() uses equality
rather than a substring match to avoid thinking an echoed command is an
indication of successful escalation. Also adds a check_missing_password
connection method to detect the error from sudo -n/doas -n.
Now, instead of relying on hostvars on the executor side, we compile
the vars for the delegated to host in a special internal variable and
have the PlayContext object look for things there when applying task/
var overrides, which is much cleaner and takes advantage of the code
already dealing with all of the magic variable variations.
Fixes#12127Fixes#12079
Use six.moves.range instead (aliased to xrange on Python 2, aliased to
range on Python 3).
Also I couldn't resist replacing the elaborate chr/ord/randrange dance
with the simpler random.choice(string.ascii_lowercase) that was already
used elsewhere in the Ansible codebase.
commit 9921bb9d2002e136c030ff337c14f8b7eab0fc72
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 20:19:44 2015 +0530
Document --ssh-extra-args command-line option
commit 8b25595e7b1cc3658803d0821fbf498c18ee608a
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Thu Aug 13 13:24:57 2015 +0530
Don't disable GSSAPI/Pubkey authentication when using --ask-pass
This commit is based on a bug report and PR by kolbyjack (#6846) which
was subsequently closed and rebased as #11690. The original problem was:
«The password on the delegated host is different from the one I
provided on the command line, so it had to use the pubkey, and the
main host doesn't have a pubkey on it yet, so it had to use the
password.»
(This commit is revised and included here because #11690 would conflict
with the changes in #11908 otherwise.)
Closes#11690
commit 119d0323892c65e8169ae57e42bbe8e3517551a3
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Thu Aug 13 11:16:42 2015 +0530
Be more explicit about why SSH arguments are added
This adds vvvvv log messages that spell out in detail where each SSH
command-line argument is obtained from.
Unfortunately, we can't be sure if, say, self._play_context.remote_user
is obtained from ANSIBLE_REMOTE_USER in the environment, remote_user in
ansible.cfg, -u on the command line, or an ansible_ssh_user setting in
the inventory or on a task or play. In some cases, e.g. timeout, we
can't even be sure if it was set by the user or just a default.
Nevertheless, on the theory that at five v's you can use all the hints
available, I've mentioned the possible sources in the log messages.
Note that this caveat applies only to the arguments that ssh.py adds by
itself. In the case of ssh_args and ssh_extra_args, we know where they
are from, and say so, though we can't say WHERE in the inventory they
may be set (e.g. in host_vars or group_vars etc.).
commit b605c285baf505f75f0b7d73cb76b00d4723d02e
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Tue Aug 11 15:19:43 2015 +0530
Add a FAQ entry about ansible_ssh_extra_args
commit 49f8edd035cd28dd1cf8945f44ec3d55212910bd
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 20:48:50 2015 +0530
Allow ansible_ssh_args to be set as an inventory variable
Before this change, ssh_args could be set only in the [ssh_connection]
section of ansible.cfg, and was applied to all hosts. Now it's possible
to set ansible_ssh_args as an inventory variable (directly, or through
group_vars or host_vars) to selectively override the global setting.
Note that the default ControlPath settings are applied only if ssh_args
is not set, and this is true of ansible_ssh_args as well. So if you want
to override ssh_args but continue to set ControlPath, you'll need to
repeat the appropriate options when setting ansible_ssh_args.
(If you only need to add options to the default ssh_args, you may be
able to use the ansible_ssh_extra_args inventory variable instead.)
commit 37c1a5b6794cee29a7809ad056a86365a2c0f886
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 19:42:30 2015 +0530
Allow overriding ansible_ssh_extra_args on the command-line
This patch makes it possible to do:
ansible somehost -m setup \
--ssh-extra-args '-o ProxyCommand="ssh -W %h:%p -q user@bouncer.example.com"'
This overrides the inventory setting, if any, of ansible_ssh_extra_args.
Based on a patch originally by @Richard2ndQuadrant.
commit b023ace8a8a7ce6800e29129a27ebe8bf6bd38e0
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date: Mon Aug 10 19:06:19 2015 +0530
Add an ansible_ssh_extra_args inventory variable
This can be used to configure a per-host or per-group ProxyCommand to
connect to hosts through a jumphost, e.g.:
inventory:
[gatewayed]
foo ansible_ssh_host=192.0.2.1
group_vars/gatewayed.yml:
ansible_ssh_extra_args: '-o ProxyCommand="ssh -W %h:%p -q bounceuser@gateway.example.com"'
Note that this variable is used in addition to any ssh_args configured
in the [ssh_connection] section of ansible.cfg (so you don't need to
repeat the ControlPath settings in ansible_ssh_extra_args).
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host). And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
The lock file is (a temporary file) opened in the parent process, whose
open fd is inherited by the workers after fork, and passed down through
the PlayContext. Connection grows lock/unlock methods which can be used
by individual connection plugins.
FieldAttributes will now by default not be post_validated unless a flag
is set on them in the class, as a large number of fields are really there
simply to be inherited by Task/PlayContext and shouldn't be templated too
early.
The other (unrelated to the base issue) in #12084 is also fixed here, where
the roles field is loaded before vars/vars_files, meaning there are no vars
yet loaded in the play when the templating occurs.
Fixes#12084
`if method in dir(self):` is very inefficient:
- it must construct a list object listing all the object attributes & methods
- it must then perform a O(N) linear scan of that list
Replace it with the idiomatic `if hasattr(self, method):`, which is a
O(1) expected time hash lookup.
Should fix#11981.
* Speed up serialization of hostvars by simply using the internal
dictionary used for cached lookups
* Use blocking gets/puts on queues instead of spin locking
* Merge sequential implicitly created blocks
* now it uses -n to get immediate error if no password is supplied and one is needed,
this should fix the issue with sudo hanging waiting for input.
* made -k configurable, this can break changing become_users in play if left out,
but opens up the possiblity of OTP support.
Fixes bugs related to creating Templar() objects on the fly, where
the shared loader objects (serialized to TaskExecutor) aren't used
so information loaded into plugin loaders after forking is lost.
Fixes#11815
* Don't reference __class__ when we can use the instance itself
* use isdisjoint() as it can stop once a match is found
* Remove a condtional that was taken care of in the conditonal just above
* added pattern matching to match v1 functionality
* check the task name, not the task+role name for matches
* make sure the input is unicode
Fixes#11692
Also making PlayContext a child class of the Playbook Base class,
which gives it access to all of the FieldAttribute code to ensure
field values are correctly typed after post_validation
Fixes#11381
There was a confusion between the valid values for defaults.gathering
(explicit/implicit/smart) and a play's gather_facts setting (boolean),
which resulted in gathering=explicit being ignored.
The tags field may contain bad data before it is post_validated, however
some methods assumed it would be a simple list or string. Using itertools
gets us around the problem of the data potentially not being hashable
Fixes#9380
Getting recursive errors otherwise, so this is probably not something
we want to do. This most likely only worked in v1 due to the fact that
module args were templated earlier than the point in Runner() when
they were fed into the templating engine.
The --force-handlers command line argument was not correctly running
handlers on hosts which had tasks that later failed. This corrects that,
and also allows you to specify force_handlers in ansible.cfg or in a
play.
- become constants inherit existing sudo/su ones
- become command line options, marked sudo/su as deprecated and moved sudo/su passwords to runas group
- changed method signatures as privlege escalation is collapsed to become
- added tests for su and become, diabled su for lack of support in local.py
- updated playbook,play and task objects to become
- added become to runner
- added whoami test for become/sudo/su
- added home override dir for plugins
- removed useless method from ask pass
- forced become pass to always be string also uses to_bytes
- fixed fakerunner for tests
- corrected reference in synchronize action plugin
- added pfexec (needs testing)
- removed unused sudo/su in runner init
- removed deprecated info
- updated pe tests to allow to run under sudo and not need root
- normalized become options into a funciton to avoid duplication and inconsistencies
- pushed suppored list to connection classs property
- updated all connection plugins to latest 'become' pe
- includes fixes from feedback (including typos)
- added draft docs
- stub of become_exe, leaving for future v2 fixes
Adds a special tag:
- always: always runs no matter what --tags, --skip-tags say
Adds 4 special keywords for --tags/skip-tags
- all: all tagged + untagged tasks
- tagged: only tagged tasks
- untagged: only untagged tasks
- always: only run tasks tagged 'always'
Adds new settings for managing retry files:
* retry_files_enabled, defaults to True
* retry_files_save_path, defaults to ~/.ansible-retry
This change was adapted from PR #5515.
James Cammarata
Abhijit Menon-Sen
Brian Coca
Brian Coca
Marius Gedminas
Toshio Kuratomi
Florian Apolloner
Damian Gerow
Thomas Quinot
Pierre-Louis Bonicoli
Benno Joy
Matt Martz
Carlos E. Garcia
Jesse Rusak
Hagai Kariti
Kishin Yagami
Maykel Moya