* WIP update asa to use network_cli connection plugin
* add asa.py to cliconf plugins
* update asa.py terminal plugin to support regexp and events
* update constants to map asa modules to asa action handler
* update asa action handler to implement persistent connections
* update asa shared module to use persistent connections
* update asa_command module to use new connection
* fixed pep8 issues
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
* shorten warning on reservd fact collision
also remove ansible_ from namespaced facts for vars manager
handle str conversion errors
use tuple to avoid iterator errors
version added added
* only modify final one
* removed ansible_ removal
* add first, last and next usable
* add usable ip filters
* add size usable, range usable and wildcard
* add ip prefix and netmask filter
* add network formatting and check if ip in subnet
* clean up order, add comments
* fix pep8
* update format by index
* clean up and updates from jmcgill298
* add connection plugin for buildah
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
* create a method to invoke buildah
* mount container filesystem persistently so we can access it
during put and fetch
* use copyfile function for copying files
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* revert tests
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* progress
(this will be squashed into a single commit before merge)
* add docs for the conn plugin
* fix issue invoking the integration tests
* add a way to invoke commands inside the container as a different user
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fix shellcheck warning
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* On python3, stdin goes through a buffer that translates from raw bytes
to text. this interferes with pause as it (1) performs universal
newline conversion and therefore '\r' is turned into '\n' and (2) the
buffering prevents us from getting the typed characters immediately
(possibly a python3 bug?) Using the raw byte stream that's behind the
text decoder fixes these problems.
Unrelated cleanups:
* Use to_text instead of str for conversion into strings to avoid possible tracebacks
* Use either \r or \n as the end of a line.
Fixes#26278Resolves#26446
By default, the vendor neutral modules will just go on if no
implementation module is found.
If user specifies the task argument fail_on_missing_module and
sets it to True, then we bail out the play early and report that
to the user.
restored 'rc' inspection but only when failed is not specified
removed redundant changed from basic.py as task_executor already adds
removed redundant filters, they are tests
added aliases to tests removed from filters
fixed test to new rc handling
* fixes become_method: runas for unprivileged users
* sets permissions on tempdir appropriately
* allows automatic system environment generation for new token (old Process.Start way prevents this)
* add basic become runas tests
* correct, cleanup & simplify dwim stack
latlh chIS logh HeS qar wej chel laD
better errors
update find_file to new exception
* addressed latest comments
* test should not use realpath as it follows symlink
this fails when on OS X as /var is now a symlink to /private/var
but first_found was not supposed to follow symlinks
The user variable stores whether we need to set user@ in our connection
string. It's now being used at the toplevel of the run() method so the
default needs to be calculated further up the stack
Fixes#24910
include_vars will now also return a key 'ansible_included_var_files'
which contains the list of files that were successfully loaded.
This is useful information and, amongst other things, a way for users
to know exactly what files were included when debugging their
playbooks.
This also allows us to improve the integration tests around
include_vars.
ansible_host can be pulled from inventory and not match inventory_hostname,
this can "loose" vars to a new host named by ansible_host vs the delegated host
fixes#25770
* Add net_interface declartive module
* Add net_interface module
* Add junos_interface implementation module
* Other minor changes
* Add integration test
* Integration test for net_interface
* Integration test for junos_interface
* Fix CI failures
* Documentation changes
* Start of ansible config project
moved configuration definitions to external yaml file vs hardcoded
* updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
* prints green for default/unchanged and yellow for those that have been overriden
* added list action to show all configurable settings and their associated ini and env var names
* allows specifying config file to see what result would look like
* TBD update, edit and view options
removed test for functions that have been removed
env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling
updated as per feedback
* pep8
According to the redis-py docs, zrank will return the 0 based index for
the value in the sorted set. So the logic here wasn't right to begin
with (It just means that a value at the 0-th position would never show
up as cached). Need to compare against None to know if the value
exists in the cache.
https://redis-py.readthedocs.io/en/latest/#redis.StrictRedis.zrankFixes#25590
* Allow template files to be vaulted
* Make sure to import exceptions we need
* get_real_file can't take bytes, since it looks specifically for string_types
* Now that we aren't using open() we don't need b_source
* Expand playbooks_vault docs to include modules that support vaulted src files
* Add vaulted template test
Initial commit to split includes into static imports/dynamic includes
This implements the new include/import syntax for Ansible 2.4:
* include_{tasks,role,variables} = dynamic
* import_{playbook,tasks,role} = static
The old bare `include` will be considered deprecated, as will any use of the `static: {yes|no}` option.
This also adds docs for import/include and reorganizing the "Playbook Reuse" section of the documentation.
* Add net_user platform agnostic module
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Integration test for net_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* modify eos_user module to support name param as alias to username
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Test collection of users
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Platform agnostic net_system module
Also refactor the action network plugins for better code re-use
Still more refactoring to do once the connection plugin work is complete
* Replace importlib for imp
importlib is not available on 2.6, so we need to stick to imp
* Load action plugin via module metadata
* Better error message if no implementation is found
Now the plugin will show the module name and the network OS in the
error message
* Fix typo on documentation author line
* Fix pep8 issues
* Add missing options key on doc string and stringify version
* Return None in case module has no metadata
* Read module metadata only if it's a python module
Check for module suffix, if it's .py then read metadata.
Otherwise this fails on non-python modules, like Windows PS for example.
* Read metadata variable only if it's a python module
Fix referencing a variable before assignment
* Add action_handler to validate_modules metadata schema
* Pull metadata with plugin_docs get_docstring
Using load_source from PluginLoader is troublesome, it is not guaranteed
a module may be importable at the controller, e.g. if a module depends
on module_utils functions it won't work, because module_utils is not
in the sys path.
Rather than putting that module dependencies introspection, just
use plain parsing like plugin_docs get_docstring does as we only care
about reading ANSIBLE_METADATA.
* Add platform agnostic group of groups for integration tests
This will be the target for platform agnostic integration tests.
* Add integration tests for net_system
* Switch to action plugin inheritance from metadata driven action handler
As the metadata action driven action handler work is being worked on
on its standalone proposal+PR, let's just go back to have one
action handler per platform agnostic module.
Those action plugins will inherit from net_base.
* Add blank line to fix pep8
* Add aliases file to net_system integration test
This will avoid CI failure
* Fix integration tests for net_system
* Give more precedence to task network_os over inventory network_os
* moved the logging statement
moved the logging statement before the actual action
* added status code check
In the existing implementation when the ssh command fails the command
result is silently discarded. It hides the fact that the disconnection
did not go as expected. Effectively the intended action was not
successful, but the play continues.
* Revert "added status code check"
This reverts commit fe2eb2ae4aeb4812fa2f59ccdfabc9efc677e657.
* added command status code check
In the existing implementation the command is checked for the success.
As a result failed execution is silently discarded. The change tests for
return code and fails if it did not work.
This is required for modules that may return a non-zero `rc` value for a
successful run, similar to #24865 for Windows fixing **win_chocolatey**.
We also disable the dependency on `rc` value only, even if `failed` was
set.
Adapted unit and integration tests to the new scheme.
Updated raw, shell, script, expect to take `rc` into account.
* eos python3 changes
* changes to convert response from byte to text
* Add dellos6 python3 changes
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix py3 prompt issue for invalid show command
* Fix review comments
* Add generic fix for error prompt in py3
* Fix CI issue
* Fix network_cli unit test failure
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
* Add nxos changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix CI issues
* Fix review comment
Replace surrogate_or_strict with
surrogate_then_replace as per review
comment os PR #24601
When only looking at the failed state of the TaskResult, certain failures
cause the linear strategy to fail out sooner than it should and not execute
the always portion of blocks.
Fixes#24301
* Add ios changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes
* Fix CI issues
* Fix review comment
* Add sros changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
Code cleanup
pep8 fixes
* Fix CI issue
* Add vyos changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes
* Fix CI issues
In _process_pending_results (strategy/__init__.py), we were using the delegate_to
field directly from the original task, which was not being templated correctly.
As an alternate to #23599, this patch instead pulls the host name out of the delegated
vars passed back in the task result dictionary to avoid having to re-template things.
Fixes#23599Fixes#20508
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.
If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.
The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2. The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type. However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes. If strings are improperly byte strings, they may cause
tracebacks on the receiving end
Fix for persistent connection plugin on Python3. Note that fixes are also needed to each terminal plugin. This PR only fixes the ios terminal (as proof that this approach is workable.) Future PRs can address the other terminal types.
* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin. A pty can't have control characters. So we have to send ascii only. That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent. So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat. Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals
Fixes#24355
Template can take a directory as the destination. When that's the case,
we need to diff between the source and the file inside of the directory.
That happened when the directory was specified with a trailing slash but
not when it was specified on its own. This change fixes that.
Fixes#24413
For playbook base objects, when dumping attributes via dump_attrs() an
attribute like loop_control is a class. Using the default serialization
for these is slow and consumes a lot of memory. Since LoopControl is also
based on the Base class, we can use serialize() instead and save a lot of
resources.
This also adds a from_attrs() complimentary method to nicely turn the
dumped attrs back into proper field attributes.
Fixes#23579
* Create netconf_config.py
To support new attribute 'src' for the netconf_config module.
Cross-check with #24323
* Update netconf_config.py
* Update netconf_config.py
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load. vars/__init__.py needed template/__init__.py
which caused issues. Loading unsafe_proxy from another location fixes
that.
Added an integration test for fetch module idempotence. (Testing
that validate_checksum is doing what it's supposed to is harder as we'd
have to create a race condition with the downloaded data to trigger it.
Probably need to make that a unittest eventually).
Also give a deprecation message to the validate_md5 parameter so that we
can eventually get rid of it.
We have a list of specific messages that we scree-scrape and flag
them as legit errors.
However, we also have a catch-all regex that matches everything
starting with %.
That can cause issues on commands that return lines with that
character, like for example the 'crypto key generate'.
Fixes#23770
This addresses a problem where the action plugin would ignore the
remote_addr value for the host. In this case, only the inventory values
for the hostname would be considered and populate the remote host
remote_addr value for the connection plugin.
* Check for provider values inside check_args of respective network code
* Partial revert of b9ee5aa
The no_log change is okay, but take out the action result munging
When retrieving file contents for diffing we need to get the contents as
binary. Otherwise python3 will try to convert the file to text and fail
with non-decodable contents.
Fixes#23171
Copy module was walking over files in subdirectories repeatedly (a
directory tree a few levels deep could bring the time spent into the
tens of minutes)
This was traced to the fix for this bug report: https://github.com/ansible/ansible/issues/13013Fixed#13013 a different way and added an integration test to check for
regressions of #13013 as we optimize this code.
Fixes#21513
Ansible will now automatically retry a connection if SSH returns an error:
mux_client_hello_exchange: write packet: Broken pipe
This is probably a bug in SSH, but because it's safe to retry this
connection there is no need for Ansible to fail because of it.
We were hard-coding the protocol, port and validate_certs on
eos EAPI via the action plugin.
Put defaults on the eos_argument_spec and pull those values from it.
* 'unable to open shell' -> direct to web help
The "unable to open shell" error is returned for a number of different,
direct people to online docs (we we can update out of band of releases)
to guide them though the various solutions.
* fix pep8 errors
The password_hash filter will generate a salt value if none is supplied.
The character set used by Ansible
(upper & lowercase letters, digits)
did not match that used by libc crypt
(upper & lowercase letters, digits, full stop, forward slash).
This resulted in a slightly smaller key space, and hence hashes would be
slightly easier to attack (e.g. by dictionary, brute force).
* Ansible 2.3 feature support for dellos6.
- With the new Ansible 2.3 infra changes, the dellos modules doesn't work
(the new infra changes are not backward compatible), so added the below
changes support it.
- Added the new terminal plugin for DellOS6
- Added the new action plugin for DellOS6
- Modified the modules to work with the new infra.
- with that it adds support for DellOS6 Persistent Connection support.
* Remove pep8 confirming files from dellos6.py and dellos6_config legacy-files
* fix 'sequence' lookup shortcut syntax and documentation
* Update playbooks_loops.rst
Minor edits for grammar and clarity.
* Update playbooks_loops.rst
Another tweak for clarity.
The fix for leading junk in sudo output: fee6e29 causes problems with
ssh + sudo. On the initial connection using ControlPersist, the output
that we scan for the prompt contains both the command we're sending to
configure the prompt and the prompt itself. The code in fee6e29 ends up
sending the password when it sees the line configuring the prompt which
is too early.
Switch to a version that splits on lines and then checks whether the
first or last line starts with the prompt to decide if it's time to send
the password.
Fixes#23054
References #20858
The pass prompt expects an answer and compares a `str` to a binary buffer, thus crashing.
It's an obvious fix to help transitioning towards Python3 and hopes it does not need a specific test.
* Fix a couple issues in synchronize with docker
* Make the rsync_opts parse as a list using the same criteria as
module_utils argumentspec parsing
* Do not quote arguments in the action plugin. The module will quote as
it knows whether it will invoke rsync with a shell or via exec.
Fixes#23046
* Ansible 2.3 feature support for dellos9 and dellos10
* Use Persistent Connection Manager
* Fix CI issue, revert the doc and metadata changes
* Reverted the meta_info (supported_by) to community from core
* Fixed the CI issues, use module_utisl.six and updated legacy-files
David Moreau-Simard
Matt Clay
Brian Coca
Peter Sprygada
Toshio Kuratomi
Brian Coca
Pilou
Ricardo Carrillo Cruz
Ken Celenza
Michael Scherer
Tomas Tomecek
Manuvaldi
Karim BEN YOUSSEF
Adrian Likins
James Mighion
Jean-Frédéric
Nathaniel Case
Ganesh Nalawade
Matt Davis
Kassian Sun
Abhijeet Kasurde
René Moser
Matt Martz
Tennis Smith
James Cammarata
Trishna Guha
Dag Wieers
piotrsmolinski
QijunPan
Benjamin Schwarze
John R Barker
Joseph Callen
Michael
Jan-Piet Mens
wiso
Nick Piper
Erwin Lang
KeepZero
Antoine Cotten
Senthil Kumar Ganesan
Chris Alfonso
Strahinja Kustudić
Will Thames
Alex Willmer
Konstantin Suvorov
Victor Perron