The ssh connection plugin is overzealous in thinking that error code 255
can only come from ssh. Python can return 255 in some circumstances and
error from php does as well.
* Update eos cliconf plugin methods
* Refactor eos cliconf plugin
* Changes in eos module_utils as per cliconf plugin refactor
* Fix unit test and sanity failures
* Fix review comment
Fixes#37262Fixes#36284
* Updates options in netconf connection to enable
bastion/jump host setting using configuration/enviornment
varaibles.
* Update troubleshooting docs from using bastion host with netconf
connection
* Create base class for network-style connections
* clean up some differences
* Move NetworkConnectionBase
* Tweak netconf for tests
* Tweak when network_os is checked to avoid failing tests
* Pull back exec_command
python2-lxc module needs bytes, on the other hand python3-lxc requires text.
To solve such incompatibility, use to_native other than to_bytes.
This fixes#41060.
* Refactor ios cliconf plugin and ios_config module
* Refactor ios cliconf plugin to support generic network_config module
* Refactor ios_config module to work with cliconf api's
* Enable command and response logging in cliconf pulgin
* cliconf api documentation
* Fix unit test and other minor changes
* Doc update
* Fix CI failure
* Add default flag related changes
* Minor changes
* redact input command logging by default
* Docs - add shared snippet note about password prompts for ssh keys
Signed-off-by: Adam Miller <admiller@redhat.com>
* add note to ssh connection plugin, fix markup, fix typo
Signed-off-by: Adam Miller <admiller@redhat.com>
* winrm: add better exception handling for krb5 auth with pexpect
* Added changelog fragment
* Added exception handler in case kinit path isn't valid, added test cases
* fixed for Python 2 compatibility
* I seem to have forgotten the back half of tests
* Set http timeout from persistent_command_timeout
* Tweak URL generation and provide URL on error
* Push var_options to connection process
* Don't wait forever if coming from persistent
* Don't send the entire contents of variables to ansible-connection
* HTTPAPI connection
* Punt run_commands to cliconf or httpapi
* Fake enable_mode on eapi
* Pull changes to nxos
* Move load_config to edit_config for future-preparedness
* Don't fail on lldp disabled
* Re-enable check_rc on nxos' run_commands
* Reorganize nxos httpapi plugin for compatibility
* draft docs for connection: httpapi
* restores docs for connection:local for eapi
* Add _remote_is_local to httpapi
* Don't rewrite remote paths when remote is secretly local
* Remote overriding is configurable in connection
* Use c.DEFAULT_LOCAL_TMP instead of remote_tmp
* Remove remote_is_local from ConnectionBase
* remote_is_local->_remote_is_local
* Add warning signs to _remote_is_local and explanatory comments to its use
* WIP Pull persistent connection parameters via get_option
* Fix pep8
* Add use_persistent_connection setting to paramiko_ssh plugin
* Add vars section to persistent_command_timeout setting and prevail provider values over config manager
* Use persistent_command_timeout on network_cli instead of timeout
* Fix unit tests
If we don't call loader to get network_cli, then _load_name is never
set and we get KeyError.
* Pull persistent_command_timeout via config manager for ios connection local
* Pull persistent_command_timeout via config manager on connection local
avoids some repetitive loading
- read config file only once
- now cache the ini parser per file
- optimize shell plugin loading
tried to 'optimize' vars_plugins loading but it creates issues with precedence,
probalby due to iterator not being reset, will look into it in subsequent fix/PR
* fixes issue when netconf would report ios is not supported
This change now will map ansible_network_os=ios to the correct netconf
plugin implementation. This will resolve an error where the netconf
connection plugin will report that ios is unsupported.
* Handle multiple sub prompts for network_cli connection
Fixes#35349
* Check if the same prompt is repeated in consecutive window
if it is repeated it indicates there is problem with answer
provided
* In that case report error to user
* Fix CI failure
* Fixes#35349
* Add prompt_retry count to control max number of times
to expect the same prompt before it error's out
* Make required changes in ios and eos terminal plugin to handle
wrong enable password correctly and return proper error
message to user.
* Check if the same prompt is repeated in consecutive window
if it is repeated it indicates there is the problem with an answer
provided
* In that case report error to user
* Fix debug logs failing with persistent connection
Fixes#33047
* As debug logs are written on stdout, it interrupts
the communication between ansible-connection(background)
process and main process. To avoid this add a string similar
to exactly identify the response string.
* Remove unwanted code in ansible-connection
* Fix review comments
* Fix spurious log emitted on ansible-connection stdout issue
* ansible-connection which runs as a background process sends a
json string (contains response received from remote device)
to foreground ansible-playbook process over stdout.
* If in case debug flag is enabled the connection_loader api
invoked from ansible-connection `ssh = connection_loader.get('ssh', class_only=True)`
results in emitting debug logs on stdout. This spurious log
interfere with the actual response and results in failure while
reading json string in ansible-playbook process
* To avoid this save stdout of ansible-connection and redirect it string
buffer to accumulate all the logs emitted by core API's
* Add these logs in `result['messages']` which is send a json string after reinstating saved stdout
* Remove unwanted code in ansible-connection
* Fix review comment
* adds support for using connection=netconf
This change updates the module to provide support for using
connection=netconf instead of connection=local. If connection=netconf
is used, then the various connection arguments will be silently ignored.
* adds netconf plugin default
This adds a default implementation for netconf plugins if the network_os
is not specified. The default plugin will implement only the standard
netconf rpcs
* fix up pep8 issues
* Look for password prompts at the beginning of the line only
* Adds secondary checks for the supplied sub-prompts
Handles the case for enable prompts where enable_pass may be incorrect
and the deivce is still prompting for the password
* allow shells to have per host options, remote_tmp
added language to shell
removed module lang setting from general as plugins have it now
use get to avoid bad powershell plugin
more resilient tmp discovery, fall back to `pwd`
add shell to docs
fixed options for when frags are only options
added shell set ops in t_e and fixed option frags
normalize tmp dir usag4e
- pass tmpdir/tmp/temp options as env var to commands, making it default for tempfile
- adjusted ansiballz tmpdir
- default local tempfile usage to the configured local tmp
- set env temp in action
add options to powershell
shift temporary to internal envvar/params
ensure tempdir is set if we pass var
ensure basic and url use expected tempdir
ensure localhost uses local tmp
give /var/tmp priority, less perms issues
more consistent tempfile mgmt for ansiballz
made async_dir configurable
better action handling, allow for finally rm tmp
fixed tmp issue and no more tempdir in ballz
hostvarize world readable and admin users
always set shell tempdir
added comment to discourage use of exception/flow control
* Mostly revert expand_user as it's not quite working.
This was an additional feature anyhow.
Kept the use of pwd as a fallback but moved it to a second ssh
connection. This is not optimal but getting that to work in a single
ssh connection was part of the problem holding this up.
(cherry picked from commit 395b714120522f15e4c90a346f5e8e8d79213aca)
* fixed script and other action plugins
ensure tmpdir deletion
allow for connections that don't support new options (legacy, 3rd party)
fixed tests
* update DOCUMENTATION for network_cli and netconf
This updates the DOCUMENTATION string for both the netconf and
network_cli connection plugin.
* add additional options to the connection documentation
* update documentation based on review and feedback
* adds persistent connection options to documentation string
* winrm: attempting to get kerb auth to work on MacOS
* moved to use pexpect if possible as it is simpler
* Made the pexpect event more lenient around different localisations
* Add parent pid to persistent connection socket path hash
Fixes#33192
* Add parent pid in persistent connection socket path hash
to avoid using same socket path for multiple simultaneous
connection to same remote host.
* Ensure unique persistent socket path for each ansible-playbook run
* Fix CI failures
* fixes guess_os for netconf connections
This change fixes invalid calls to play_context when the network_os is
not set and the connection attempts to guess the network_os. The method
will now check the correct values for ssh key file and allow agent
instead of returning errors.
* fix up pep8 issues
Fixes#33406Fixes#33405
* Fix typo in network_cli for sendonly
* Send `abort` to remote device in case configuration fails
* Fix indentation issue in eos_static_route integration test
* Revert network_cli change
* Fix junos integration test fixes as per connection refactor (#33050)
Refactor netconf connection plugin to work with netconf plugin
* Fix junos integration test fixes as per connection refactor (#33050)
Refactor netconf connection plugin to work with netconf plugin
Fix CI failure
Fix unit test failure
Fix review comments
* Allow the user to circumvent adding -tt on ssh commands to help aid in
debugging ssh related problems.
* Move config to the plugin
* Set version_added
* Change yaml section to "connection"
* Fix ssh unit tests
This change will now track any created persistent connection and shut it
down at the end of the play run. This change also includes an update to
properly honor the reset_connection meta handler.
* win_reboot: change to sample system uptime instead of checking port status
* added connection timeout back in as now we can manually set it per connection.
* some pep8 fixes
* fix up error message on timeout in case an exception wasn't fired
* Changed doc to English (US) and simplified uptime check
* moved conn timeout over to new config connection options
* included inventory and callback in new config
allow inventory to be configurable
updated connection options settings
also updated winrm to work with new configs
removed now obsolete set_host_overrides
added notes for future bcoca, current one is just punting, it's future's problem
updated docs per feedback
added remove group/host methods to inv data
moved fact cache from data to constructed
cleaner/better options
fix when vars are added
extended ignore list to config dicts
updated paramiko connection docs
removed options from base that paramiko already handles
left the look option as it is used by other plugin types
resolve delegation
updated cache doc options
fixed test_script
better fragment merge for options
fixed proxy command
restore ini for proxy
normalized options
moved pipelining to class
updates for host_key_checking
restructured mixins
* fix typo
* Update connection play_context when socket exists
* Don't fail on connections other than network_cli
* Fix enable prompt detection on ios & eos
* Check against "Module not found" error code, defined in modules/jsonrpc.py
* fixes asa action plugin for connection=local
This change fixes asa modules when using connection=local to load the
provider values.
* fix up pep8 issues
* implements jsonrpc message passing for ansible-connection
* implements more generic mechanism for persistent connections
* starts persistent connection in task_executor if enabled and supported
* supports using network_cli as top level connection plugin
* enhances logging for persistent connection to stdout
* Update action plugins
* Fix Python3 RPC
* Fix Junos bytes<-->str issues
* supports using netconf as top level connection plugin
* Error message when running netconf on an unsupported platform
* Update tests
* Fix `authorize: yes` for `connection: local`
* Handle potentially JSON data in terminal
* Add clarifying detail if possible on ConnectionError
* Fix wrong prompt issue for network moodules
Fixes#31161Fixes#32416
* Store the device prompt in case of error
from remote device
* Check for prompt value in ios action plugin
* Add integration test
* Using docstrings conflicts with the standard use of docstrings
* PYTHON_OPTIMIZE=2 will omit docstrings. Using docstrings makes future
changes to the plugin and module code subject to the requirement that we
ensure it won't be run with optimization.
* only complain about ini deprecation if value is set
* set plugin config for stdout and other types
* updated plugin docs, moved several plugins to new config
* finished ssh docs
* fixed some issues seen in plugins while modifying docs
* placeholder for 'required'
* callbacks must use _plugin_options as _options already in use
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
So we are removing the transport_test for the listed connection types,
because they fail to take into account bastion or proxy servers for
testing the transport.
The result of removing this, is that modules using this facility will do
a complete round-trip attempt, running a module, which is a bit heavier but correct.
This fixes#23774
* Create persistent socket path using port and connection type
* Use remote address, port, connection type and remote user
to create a socket path.
* Fix review comment
* Ansible Config part2
- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code
skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution
* killed extra space with extreeme prejudice
cause its an affront against all that is holy that 2 spaces touch each other!
shippable timing out on some images, but merging as it passes most
1fe67f9 introduced retries to the ssh connection put file and fetch
file. Unfortunately, that change broke the smart transport because it
started raising exceptions instead of returning from _run(). This
breakage is documented in #23711.
An attempt to fix it was made at #23717 but the first attempt was
objected to as needing to touch too much code. The second attmept was
objected to as smart was forced to encapsulate retries (thus retrying
a sftp "rety" times before trying scp "retry" times and then finally
moving onto piped). This third attempt has retries encapsulate smart.
So each sub-transport is tried once and if all three fail, another retry
attempt is made which tries each of the three again.
Fixes#23711Fixes#23717
Currently socket path is send from `ansible-connection` (running as background
process) over stdout. This can conflict with debug logs that are also send on
stdout resulting in incorrect socket path received by the main process.
To avoid this add a socket path delimiter string which is recevied by
main process and socket path is retrieved based on delimiter string.
This implementation will change in future when ansible-connection
framework is made more robust.
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
* add connection plugin for buildah
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
* create a method to invoke buildah
* mount container filesystem persistently so we can access it
during put and fetch
* use copyfile function for copying files
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* revert tests
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* progress
(this will be squashed into a single commit before merge)
* add docs for the conn plugin
* fix issue invoking the integration tests
* add a way to invoke commands inside the container as a different user
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fix shellcheck warning
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* moved the logging statement
moved the logging statement before the actual action
* added status code check
In the existing implementation when the ssh command fails the command
result is silently discarded. It hides the fact that the disconnection
did not go as expected. Effectively the intended action was not
successful, but the play continues.
* Revert "added status code check"
This reverts commit fe2eb2ae4aeb4812fa2f59ccdfabc9efc677e657.
* added command status code check
In the existing implementation the command is checked for the success.
As a result failed execution is silently discarded. The change tests for
return code and fails if it did not work.
* eos python3 changes
* changes to convert response from byte to text
* Add dellos6 python3 changes
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix py3 prompt issue for invalid show command
* Fix review comments
* Add generic fix for error prompt in py3
* Fix CI issue
* Fix network_cli unit test failure
Fix for persistent connection plugin on Python3. Note that fixes are also needed to each terminal plugin. This PR only fixes the ios terminal (as proof that this approach is workable.) Future PRs can address the other terminal types.
* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin. A pty can't have control characters. So we have to send ascii only. That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent. So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat. Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals
Fixes#24355
Ansible will now automatically retry a connection if SSH returns an error:
mux_client_hello_exchange: write packet: Broken pipe
This is probably a bug in SSH, but because it's safe to retry this
connection there is no need for Ansible to fail because of it.
Toshio Kuratomi
Ganesh Nalawade
Nathaniel Case
Jordan Borean
Abhijeet Kasurde
Biao Liu
toshi_pp
jainnikhil30
Brian Coca
james-jra
wiso
Adam Miller
Ricardo Carrillo Cruz
Stephan Lohse
jctanner
Brian Coca
John R Barker
Peter Sprygada
Dag Wieers
bdowling
Luca Berruti
Kedar Kekan
bdowling
Chris Houseknecht
Matt Martz
xuxinkun
Matt Clay
bdowling
Alex Ratner
Alexandre Garnier
Tomas Tomecek
Pierre Guinoiseau
Gaige B Paulsen
Matt Davis
Gabor Lekeny
Michael Scherer
piotrsmolinski
Benjamin Schwarze
Strahinja Kustudić