* Ensure play order is obeyed
it was being ignored depending on other options
also added tests for each order (except shuffle) both serial and not
fixes#49846
* [WIP] become plugins
Move from hardcoded method to plugins for ease of use, expansion and overrides
- load into connection as it is going to be the main consumer
- play_context will also use to keep backwards compat API
- ensure shell is used to construct commands when needed
- migrate settings remove from base config in favor of plugin specific configs
- cleanup ansible-doc
- add become plugin docs
- remove deprecated sudo/su code and keywords
- adjust become options for cli
- set plugin options from context
- ensure config defs are avaialbe before instance
- refactored getting the shell plugin, fixed tests
- changed into regex as they were string matching, which does not work with random string generation
- explicitly set flags for play context tests
- moved plugin loading up front
- now loads for basedir also
- allow pyc/o for non m modules
- fixes to tests and some plugins
- migrate to play objects fro play_context
- simiplify gathering
- added utf8 headers
- moved option setting
- add fail msg to dzdo
- use tuple for multiple options on fail/missing
- fix relative plugin paths
- shift from play context to play
- all tasks already inherit this from play directly
- remove obsolete 'set play'
- correct environment handling
- add wrap_exe option to pfexec
- fix runas to noop
- fixed setting play context
- added password configs
- removed required false
- remove from doc building till they are ready
future development:
- deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems
* cleanup
remove callers to removed func
removed --sudo cli doc refs
remove runas become_exe
ensure keyerorr on plugin
also fix backwards compat, missing method is attributeerror, not ansible error
get remote_user consistently
ignore missing system_tmpdirs on plugin load
correct config precedence
add deprecation
fix networking imports
backwards compat for plugins using BECOME_METHODS
* Port become_plugins to context.CLIARGS
This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
instead
* Refactor make_become_commands as asked for by alikins
* Typo in comment fix
* Stop loading values from the cli in more than one place
Both play and play_context were saving default values from the cli
arguments directly. This changes things so that the default values are
loaded into the play and then play_context takes them from there.
* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH
As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that
should be done all at one time rather than piecemeal.
* One to throw away
This is a set of hacks to get setting FieldAttribute defaults to command
line args to work. It's not fully done yet.
After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.
What we want to be able to do ideally is something like this:
class Base(FieldAttributeBase):
_check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])
class Play(Base):
# lambda so that we have a chance to parse the command line args
# before we get here. In the future we might be able to restructure
# this so that the cli parsing code runs before these classes are
# defined.
class Task(Base):
pass
And still have a playbook like this function:
---
- hosts:
tasks:
- command: whoami
check_mode: True
(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).
There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now. The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)
* Revert "One to throw away"
This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.
* Set FieldAttr defaults directly from CLIARGS
* Remove dead code
* Move timeout directly to PlayContext, it's never needed on Play
* just for backwards compat, add a static version of BECOME_METHODS to constants
* Make the become attr on the connection public, since it's used outside of the connection
* Logic fix
* Nuke connection testing if it supports specific become methods
* Remove unused vars
* Address rebase issues
* Fix path encoding issue
* Remove unused import
* Various cleanups
* Restore network_cli check in _low_level_execute_command
* type improvements for cliargs_deferred_get and swap shallowcopy to default to False
* minor cleanups
* Allow the su plugin to work, since it doesn't define a prompt the same way
* Fix up ksu become plugin
* Only set prompt if build_become_command was called
* Add helper to assist connection plugins in knowing they need to wait for a prompt
* Fix tests and code expectations
* Doc updates
* Various additional minor cleanups
* Make doas functional
* Don't change connection signature, load become plugin from TaskExecutor
* Remove unused imports
* Add comment about setting the become plugin on the playcontext
* Fix up tests for recent changes
* Support 'Password:' natively for the doas plugin
* Make default prompts raw
* wording cleanups. ci_complete
* Remove unrelated changes
* Address spelling mistake
* Restore removed test, and udpate to use new functionality
* Add changelog fragment
* Don't hard fail in set_attributes_from_cli on missing CLI keys
* Remove unrelated change to loader
* Remove internal deprecated FieldAttributes now
* Emit deprecation warnings now
* faster config loading
- used already loaded module var instead of doc functions
- add preload to populate config defs cache
- avoid debug work when not in debug
- generators, force consumption
* Once cli args are parsed, they're constant. So, save the parsed args
into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
* Got rid of the private self._options attribute
* Use context.CLIARGS in the individual callback plugins instead.
* Also output positional arguments in default and unixy plugins
* Code has been simplified since we're now dealing with a dict rather
than Optparse.Value
* Only template values in vars_prompt rather than all vars
This allows the use of variables in vars_prompt fields but allows variables entered in the prompt to affect play vars rather than throwing an undefined error.
Only post validate if there was a vars_prompt
* Add tests for vars_prompt
* Start of ansible config project
moved configuration definitions to external yaml file vs hardcoded
* updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
* prints green for default/unchanged and yellow for those that have been overriden
* added list action to show all configurable settings and their associated ini and env var names
* allows specifying config file to see what result would look like
* TBD update, edit and view options
removed test for functions that have been removed
env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling
updated as per feedback
* pep8
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
In 159aa26b36
the result of _get_serialized_branches when no hosts were matched
changed from [[]] to []. Thus, the v2_playbook_on_no_hosts_matched
callback would not fire. Change the check so we get the error message
again. Fixes#17706
* Pass the absolute path to dirname when assigning basedir
If no path is specified when calling the playbook, os.path.dirname(playbook_path) returns ''
This will cause failure when creating the retry file.
Fixes#17456
* Updated to use os.pathdirname(os.path.abspath())
* Add a new config option to cache the check for controlpersist on the
control machine.
Fixes#15844
* Remove the option and make the behavior the default
* Make the check for controlpersist cache its status per-ssh executable
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing. So once created it we had two sets of functions that did
the same things but had different implementations. To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
The calculation for max_fail_percentage was moved into the linear
strategy a while back, and works better there in the stategy layer
rather than at the PBE layer. This patch removes it from the PBE layer
and tweaks the logic controlling whether or not the next batch is run.
Fixes#15954
The flag new_pb_basedir is not being utilized in Inventory._get_hostgroup_vars,
leading to the situation where an inventory with no playbook basedir set will
read host/group vars from the $CWD, regardless of the inventory and/or playbook
relative location. This patch corrects that by not using the playbook basedir
if it is unset (None).
This patch also corrects a bug in which the VariableManager would accumulate
host/group vars files, which could lead to incorrect vars files being used when
playbooks are run from different directories containing their own group/host vars
directories.
Fixes#16953
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.
Fixes#16937
This feature changes the scalar value of `serial:` to a list, which
allows users to specify a list of values, so batches can be ramped
up (commonly called "canary" setups):
- hosts: all
serial: [1, 5, 10, "100%"]
tasks:
...
Again, as we're carrying failed/unreachable hosts forward from play to play via
internal structures, we need to remember which ones had previously failed so that
unrelated host failures don't inflate the numbers for a given serial batch in the
PlaybookExecutor causing a premature exit.
Fixes#16364
* Fix: create retry_files_save_path if it doesn't exist
Ansible documentation states that retry_files_save_path directory will be
created if it does not already exist. It currently doesn't, so this patch
fixes it :)
* Use makedirs_safe to ensure thread-safe dir creation
@bcoca suggested to use the makedirs_safe helper function :)
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.
Fixes#15523
rm _del_ as it might leak memory
renamed to tmp file cleanup
added exception handling when traversing file list, even if one fails try rest
added cleanup to finally to ensure removal in most cases
Ansible when there was a percentage that was calculated to be less than
1.0 would run all hosts as the value for a rolling update.
The error is due to the fact that Python will round a
float that is under 1.0 to 0, which will trigger the case of
0 hosts. The 0 host case tells ansible to run all hosts.
The fix will see if the percentage calculation after int
conversion is 0 and will else to 1 host.
- moved to base cli class to handle centrally and duplicate less code
- now avoids duplication and reiteration of signal handler by reassigning it
- left note on how to do non-graceful in case we add in future
as I won't remember everything i did here and don't want to 'relearn' it.
- adhoc now terminates gracefully
- avoid race condition on terminations by ignoring errors if
worker might have been reaped between checking if active and termination call
- ansible-playbook now properly exits on sigint/term
- adhoc and playbook now give exceptions that we should not normally capture
and rely on top level finally to reap children
- handle systemexit breaks in workers
- added debug to see at which frame we exit
partial fix for #14346
pushed it to use the existing propmpt from display and moved the vars prompt code there also for uniformity
changed vars_prompt to check extra vars vs the empty play.vars to restore 1.9 behaviour
sipmlified the code as it didn't need to check for syntax again (tqm is made none prior based on that)
fixes#13770
* Move self._tqm.load_callbacks() earlier to ensure that v2_on_playbook_start can fire
* Pass the playbook instance to v2_on_playbook_start
* Add a _file_name instance attribute to the playbook
Brian Coca
Matt Martz
Martin Krizek
Toshio Kuratomi
Sam Doran
Sam Doran
Brian Coca
Dag Wieers
stephane
Andrea Tartaglia
James Cammarata
J. Javier Maestro
Vincent Roy
Jonathon Klobucar
Toshio Kuratomi