* Major slow down due to using the wrong loader for plugin_types
* traceback due to adding httpapi and cliconf plugins to the plugins we
allow ansible-doc to process
* removed hardcoded loader/plugins list
* updated a few errors to keep orig object
* fix httpapi/cliconf listing
* ansible-doc fixes
* show undocumented as UNDOCUMENTEd
* added missing undoc
* Fix ansible-doc wrt removed modules
* Fix listing of modules ia ansible-doc to not complain about removed modules
Removed modules are marked as such in the metadata but nowhere else.
Need to retrieve the metadata when a module doesn't have a doc so that
we can tell if it falls under this case.
* omit removed modules from json dump
* Print an error that the module has been removed if attempting to run
ansible-doc on that specific module
* Get plugin_formatter to stop outputting removed modules
If there were fatal bugs in this portion of config, they would never be displayed
because config would fail to load and then every^U (Exageration... only
half of every) other part of the code which depended on config to be
loaded would fail before we ever got around to a section of code that
would process UNABLE.
Remove the try except from here so that we are able to debug this code
NOTE:
1. Use unsafe decorator but not builtin escape wrapper in jinja2
since ansible will try parse ssh password twice, the builtin
escape wrapper will be removed during the first parse.
2. Use class AnsibleUnsafeText but not '!unsafe' syntax since
passwords are not loaded by YAML env, '!unsafe' syntax doesn't
work for them.
* handle groups correctly when they are 'null'
- even if defined as mapping but having no keys, objects shoudl still be processed correctly
- also add ansilbe_verbosity to vars not to display in ansible-inventory
fixes#41692
* allow to load json marked as unsafe or vault
* centralized json code/decode, add vault support
* use generics to allow for more varied inputs
* allow inventory to dump vault w/o decrypting
* override simplejson also
* add entry for unsafe also
* load vaulted and unsafe json, support unvaulting if secrets provided
Add the 'localhost_warning' configuration option. When set to 'false',
this will prevent Ansible from issuing a warning when the inventory is
empty and it is using an implicit inventory with only 'localhost'.
Closes#17086
* centralize doc/config plugin lists
also update list for generation in docsite
added note to ensure they are in sync
* updated shell page to list plugins
added some more docs hinting at plugins being configurable
* fix edit link for plugins
* Fix 'New Vault password' on vault 'edit'
ffe0ddea96 introduce a
change on 'ansible-vault edit' that tried to check
for --encrypt-vault-id in that mode. But '--encrypt-vault-id'
is not intended for 'edit' since the 'edit' should always
reuse the vault secret that was used to decrypt the text.
Change cli to not check for --encrypt-vault-id on 'edit'.
VaultLib.decrypt_and_get_vault_id() was change to return
the vault secret used to decrypt (in addition to vault_id
and the plaintext).
VaultEditor.edit_file() will now use 'vault_secret_used'
as returned from decrypt_and_get_vault_id() so that
an edited file always gets reencrypted with the same
secret, regardless of any vault id configuration or
cli options.
Fixes#35834
* Fix a traceback in ansible-pull on python3 comparing output from
subprocess with a text string.
* Rename variables that hold byte strings so we are clear that those are
not text strings.
* Use to_text() to transform variable that's being displayed as it's
less fragile than str().
Fixes#36962
ansible-console was calling setup_vault_secrets
twice. Once directly and once via _play_prereqs()
The direct invocation was not setting auto_prompt=False.
However, the direct invocation isn't need at all so
this removes it so only _play_reqs() is used.
That fixes the unrequested vault password
prompting.
Fixes#33027
* add export option
* added 'export mode' to ansible-inventory
this optimizes the output for exporting inventory vs representing the 'ansible view'
fixes#30877
* added group priority when needed
Enforce module deprecation.
After module has reached the end of it's deprecation cycle we will replace it with a docs stub.
* Replace deprecated modules with docs-only sub
* Use of deprecated past deprecation cycle gives meaningful message (see examples below)
* Enforce documentation.deprecation dict via `schema.py`
* Update `ansible-doc` and web docs to display documentation.deprecation
* Document that structure in `dev_guide`
* Ensure that all modules starting with `_` have a `deprecation:` block
* Ensure `deprecation:` block is only used on modules that start with `_`
* `removed_in` A string which represents when this module needs **deleting**
* CHANGELOG.md and porting_guide_2.5.rst list removed modules as well as alternatives
* CHANGELOG.md links to porting guide index
To ensure that meaningful messages are given to the user if they try to use a module at the end of it's deprecation cycle we enforce the module to contain:
```python
if __name__ == '__main__':
removed_module()
```
Enforce that there can be only one --new-vault-id or
--new-vault-password-file and use this instead of
--encrypt-vault-id
* Add a config option for default vault encrypt id
* allow shells to have per host options, remote_tmp
added language to shell
removed module lang setting from general as plugins have it now
use get to avoid bad powershell plugin
more resilient tmp discovery, fall back to `pwd`
add shell to docs
fixed options for when frags are only options
added shell set ops in t_e and fixed option frags
normalize tmp dir usag4e
- pass tmpdir/tmp/temp options as env var to commands, making it default for tempfile
- adjusted ansiballz tmpdir
- default local tempfile usage to the configured local tmp
- set env temp in action
add options to powershell
shift temporary to internal envvar/params
ensure tempdir is set if we pass var
ensure basic and url use expected tempdir
ensure localhost uses local tmp
give /var/tmp priority, less perms issues
more consistent tempfile mgmt for ansiballz
made async_dir configurable
better action handling, allow for finally rm tmp
fixed tmp issue and no more tempdir in ballz
hostvarize world readable and admin users
always set shell tempdir
added comment to discourage use of exception/flow control
* Mostly revert expand_user as it's not quite working.
This was an additional feature anyhow.
Kept the use of pwd as a fallback but moved it to a second ssh
connection. This is not optimal but getting that to work in a single
ssh connection was part of the problem holding this up.
(cherry picked from commit 395b714120522f15e4c90a346f5e8e8d79213aca)
* fixed script and other action plugins
ensure tmpdir deletion
allow for connections that don't support new options (legacy, 3rd party)
fixed tests
`ansible-vault` is the only cli command which knows how to handle the
rekey options `--new-vault-id` and `--new-vault-password-file`. No
point in exposing those rekey options to any of the other ansible
commands.
On a practical level I think this matters most in ensuring that
`--help` doesn't produce any false/unhelpful output.
Without this additional code snippet `ansible-pull` will still accept
the `--vault-id` option. It just won't pass the option along when
invoking `ansible-pull`.
* jsonify inventory
* smarter import, dont pass kwargs where not needed
* added datetime
* Eventual plan for json utilities to migrate to common/json_utils when we split
basic.py no need to move jsonify to another file now as we'll do that later.
* json_dict_bytes_to_unicode and json_dict_unicode_to_bytes will also
change names and move to common/text.py at that time (not to json).
Their purpose is to recursively change the elements of a container
(dict, list, set, tuple) into text or bytes, not to json encode or
decode (they could be a generic precursor to that but are not limited
to that.)
* Reimplement the private _SetEncoder which changes sets and datetimes
into objects that are json serializable into a private function
instead. Functions are more flexible, less overhead, and simpler than
an object.
* Remove code that handled simplejson-1.5.x and earlier. Raise an error
if that's the case instead.
* We require python-2.6 or better which has the json module builtin to
the stdlib. So this is only an issue if the stdlib json has been
overridden by a third party module and the simplejson on the system
is 1.5.x or less. (1.5 was released on 2007-01-18)
* included inventory and callback in new config
allow inventory to be configurable
updated connection options settings
also updated winrm to work with new configs
removed now obsolete set_host_overrides
added notes for future bcoca, current one is just punting, it's future's problem
updated docs per feedback
added remove group/host methods to inv data
moved fact cache from data to constructed
cleaner/better options
fix when vars are added
extended ignore list to config dicts
updated paramiko connection docs
removed options from base that paramiko already handles
left the look option as it is used by other plugin types
resolve delegation
updated cache doc options
fixed test_script
better fragment merge for options
fixed proxy command
restore ini for proxy
normalized options
moved pipelining to class
updates for host_key_checking
restructured mixins
* fix typo
* Fix vault --ask-vault-pass with no tty
2.4.0 added a check for isatty() that would skip setting up interactive
vault password prompts if not running on a tty.
But... getpass.getpass() will fallback to reading from stdin if
it gets that far without a tty. Since 2.4.0 skipped the interactive
prompts / getpass.getpass() in that case, it would never get a chance
to fall back to stdin.
So if 'echo $VAULT_PASSWORD| ansible-playbook --ask-vault-pass site.yml'
was ran without a tty (ie, from a jenkins job or via the vagrant
ansible provisioner) the 2.4 behavior was different than 2.3. 2.4
would never read the password from stdin, resulting in a vault password
error like:
ERROR! Attempting to decrypt but no vault secrets found
Fix is just to always call the interactive password prompts based
on getpass.getpass() on --ask-vault-pass or --vault-id @prompt and
let getpass sort it out.
* up test_prompt_no_tty to expect prompt with no tty
We do call the PromptSecret class if there is no tty, but
we are back to expecting it to read from stdin in that case.
* Fix logic for when to auto-prompt vault pass
If --ask-vault-pass is used, then pretty much always
prompt.
If it is not used, then prompt if there are no other
vault ids provided and 'auto_prompt==True'.
Fixes vagrant bug https://github.com/hashicorp/vagrant/issues/9033Fixes#30993
* Automatically loads and executes an inventory plugin specified by a standard YAML inventory config file containing a `plugin` key at its root.
* Moved inventory PluginLoader to a shared global instance.
So we fixed everything that was not a module to be PEP8 compliant, and
in the meantime these 5 new files were additionally disabled from PEP8
testing.
This fixes it.
Also update Copyright/License statements.
This makes adhoc mirror playbook callback functionality by running a
callback before and after all tasks have run. Adhoc commands now call:
- v2_playbook_on_start
- v2_playbook_on_stats
NOTE: When v2_playbook_on_start is called, a dummy playbook is provided
that says its _file_name is __adhoc_playbook__. All callback plugins
that provide v2_playbook_on_start access the _file_name attribute, so
this should maintain backward compatibility when those plugins are
called with adhoc commands even though they would not have been called
previously. The adhoc play is also added to _entries for any private
callback plugins that might be using this callback.
This reverts commit 07acc579db.
On closer examination of this code, the conditional that had force in it
was not a parent of this one. So handling of force is needed i both
branches.
See the recent comments on #23391
* Avoid default inventory proccessing for pull
- now pull's own special inventory processing should work correctly
- also removed ineffective set_defaults
fixes#31449
* use class property instead
* only do localhost for adhoc
(cherry picked from commit aad5d1432583c4aa4105b774f38c80498e85de59)
This adds a new type of vault-password script (a 'client') that takes advantage of and enhances the
multiple vault password support.
If a vault password script basename ends with the name '-client', consider it a vault password script client.
A vault password script 'client' just means that the script will take a '--vault-id' command line arg.
The previous vault password script (as invoked by --vault-password-file pointing to an executable) takes
no args and returns the password on stdout. But it doesnt know anything about --vault-id or multiple vault
passwords.
The new 'protocol' of the vault password script takes a cli arg ('--vault-id') so that it can lookup that specific
vault-id and return it's password.
Since existing vault password scripts don't know the new 'protocol', a way to distinguish password scripts
that do understand the protocol was needed. The convention now is to consider password scripts that are
named like 'something-client.py' (and executable) to be vault password client scripts.
The new client scripts get invoked with the '--vault-id' they were requested for. An example:
ansible-playbook --vault-id my_vault_id@contrib/vault/vault-keyring-client.py some_playbook.yml
That will cause the 'contrib/vault/vault-keyring-client.py' script to be invoked as:
contrib/vault/vault-keyring-client.py --vault-id my_vault_id
The previous vault-keyring.py password script was extended to become vault-keyring-client.py. It uses
the python 'keyring' module to request secrets from various backends. The plain 'vault-keyring.py' script
would determine which key id and keyring name to use based on values that had to be set in ansible.cfg.
So it was also limited to one keyring name.
The new vault-keyring-client.py will request the secret for the vault id provided via the '--vault-id' option.
The script can be used without config and can be used for multiple keyring ids (and keyrings).
On success, a vault password client script will print the password to stdout and exit with a return code of 0.
If the 'client' script can't find a secret for the --vault-id, the script will exit with return code of 2 and print an error to stderr.
In cli.CLI.unfrack_path callback, special case if the
value of '--output' is '-', and avoid expanding
it to a full path.
vault cli already has special cases for '-', so it
just needs to get the original value to work.
Fixes#30550
* fixed ansible/git invocation options
now falls back to using localhost as 'all' does not include implicit accidentally anymore
fixes#30636
* better fix
* qfq9
This is to catch vault secrets from config and
cli. Previously vault_password_file in config was
missed since it was added by setup_vault_secrets,
so check after setup_vault_secrets.
* Don't ask for password confirm on 'ansible-vault edit'
This is to match the 2.3 behavior on:
ansible-vault edit encrypted_file.yml
Previously, the above command would consider that a 'new password'
scenario and prompt accordingly, ie:
$ ansible-vault edit encrypted_file.yml
New Password:
Confirm New Password:
The bug was cause by 'create_new_password' being used for
'edit' action. This also causes the previous implicit 'auto prompt'
to get triggered and prompt the user.
Fix is to make auto prompt explicit in the calling code to handle
the 'edit' case where we want to auto prompt but we do not want
to request a password confirm.
Fixes#30491
* finalize lookup documentation
* minor fixes to ansible-doc
- actually show which file caused error on when listing plugins
- removed redundant display of type and name
* smart quote fixes from toshio
* module and vault fixes
- fix module_path cli option and usage, which fixes#29653
- move --output to be in subset of vault cli, no need for all vault enabled cli to use it
- added debug to loader to see directories added
* Change ansible-doc usage to show -a is for internal use
ansible-doc -a is for testing that documentation is sane. It should not
be used by normal users in production. The main reason for this is that
it is designed to fail if there are any undocumented modules or plugins.
This is good for testing that all plugins we ship are documented. It is
not good for end users who may have undocumented third-party plugins.
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
* let generate_man also gen rst pages for cli tools
* make template-file, output-dir, output format cli options for generate_man
* update main Makefile to use generate_man.py for docs (man pages and rst)
* update vault docs that use :option:
* Edits based on
6e34ea6242 and
a3afc78535
* add a optparse 'desc' to lib/ansible/cli/config.py
The man page needs a short desc for the 'NAME' field
which it gets from the option parse 'desc' value.
Fixes building ansible-config man page.
* add trim_docstring from pep257 to generate_man
use pep258 docstring trim function to fix up any indention
weirdness inherit to doc strings (ie, lines other than
first line being indented.
* Add refs to cli command actions
To reference ansible-vaults --vault-id option, use:
:option:`The link text here <ansible-vault --vault-id>`
or:
:option:`--vault-id <ansible-vault --vault-id>`
To reference ansible-vault's 'encrypt' action, use:
:ref:`The link text here <ansible_vault_encrypt>`
or most of the time:
:ref:`ansible-vault encrypt <ansible_vault_encrypt>`
This allows to use a pathlist in the ansible.cfg:
[default]
inventory = path/inventory:other_path/inventory
Since ansible allows to use --inventory on CLI more then once, we should also support a pathlist in the config.
Toshio Kuratomi
Brian Coca
Matt Clay
Daniel Nägele
Matt Martz
Sam Doran
joren485
Eric Rinish
Brian Coca
Zhikang Zhang
Luke Short
Abhijeet Kasurde
Matt Davis
Ben Thomasson
Matt Davis
Lars Kellogg-Stedman
Adrian Likins
Victor da Costa
Dylan Murray
Leigh Jenkin
Chris Houseknecht
John R Barker
Pilou
Tristan de Cacqueray
Will Thames
Adam Miller
Andreas Olsson
Valentin Krasontovitsch
Sloane Hertel
Dag Wieers
jctanner
Jacob Floyd
Jerry Chong
Ilya Margolin
Rene Moser
Brock R