- support config operations for EXOS-based platforms
- add regex to detect command failure responses
- add exos action plugin for "backup" operation
- add unit tests for exos_command (currently 94% coverage of
exos_config.py)
_parsed_return_data should only be used with the return from a module.
This location was invoking a remote shell command rather than a module
so we don't want it here.
* Remove use of simplejson throughout code base. Fixes#42761
* Address failing tests
* Remove simplejson from contrib and other outlying files
* Add changelog fragment for simplejson removal
Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.
* Change AnsiballZ wrapper to use import to invoke the module
We need the module to think of itself as a script because it could be
coded as:
main()
or as:
if __name__ == '__main__':
main()
Or even as:
if __name__ == '__main__':
random_function_name()
A script will invoke all of those. Prior to this change, we invoked
a second Python interpreter on the module so that it really was
a script. However, this means that we have to run python twice (once
for the AnsiballZ wrapper and once for the module). This change makes
the module think that it is a script (because __name__ in the module ==
'__main__') but it's actually being invoked by us importing the module
code.
There's three ways we've come up to do this.
* The most elegant is to use zipimporter and tell the import mechanism
that the module being loaded is __main__:
* 5959f11c9d/lib/ansible/executor/module_common.py (L175)
* zipimporter is nice because we do not have to extract the module from
the zip file and save it to the disk when we do that. The import
machinery does it all for us.
* The drawback is that modules do not have a __file__ which points
to a real file when they do this. Modules could be using __file__
to for a variety of reasons, most of those probably have
replacements (the most common one is to find a writable directory
for temporary files. AnsibleModule.tmpdir should be used instead)
We can monkeypatch __file__ in fom AnsibleModule initialization
but that's kind of gross. There's no way I can see to do this
from the wrapper.
* Next, there's imp.load_module():
* https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
* imp has the nice property of allowing us to set __name__ to
__main__ without changing the name of the file itself
* We also don't have to do anything special to set __file__ for
backwards compatibility (although the reason for that is the
drawback):
* Its drawback is that it requires the file to exist on disk so we
have to explicitly extract it from the zipfile and save it to
a temporary file
* The last choice is to use exec to execute the module:
* https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
* The code we would have to maintain for this looks pretty clean.
In the wrapper we create a ModuleType, set __file__ on it, read
the module's contents in from the zip file and then exec it.
* Drawbacks: We still have to explicitly extract the file's contents
from the zip archive instead of letting python's import mechanism
handle it.
* Exec also has hidden performance issues and breaks certain
assumptions that modules could be making about their own code:
http://lucumr.pocoo.org/2011/2/1/exec-in-python/
Our plan is to use imp.load_module() for now, deprecate the use of
__file__ in modules, and switch to zipimport once the deprecation
period for __file__ is over (without monkeypatching a fake __file__ in
via AnsibleModule).
* Rename the name of the AnsiBallZ wrapped module
This makes it obvious that the wrapped module isn't the module file that
we distribute. It's part of trying to mitigate the fact that the module
is now named __main)).py in tracebacks.
* Shield all wrapper symbols inside of a function
With the new import code, all symbols in the wrapper become visible in
the module. To mitigate the chance of collisions, move most symbols
into a toplevel function. The only symbols left in the global namespace
are now _ANSIBALLZ_WRAPPER and _ansiballz_main.
revised porting guide entry
Integrate code coverage collection into AnsiballZ.
ci_coverage
ci_complete
One of the earlier implementation of unified temp for 2.4 passed the
temp diretory to the remote side using this environment variable. We
later changed it to be passed via a module parameter but forgot to
remove the environment variable.
Allow specifying the source and destination files' encodings in the template module
* Added output_encoding to the template module, default to utf-8
* Added documentation for the new variables
* Leveraged the encoding argument on to_text() and to_bytes() to keep the implementation as simple as possible
* Added integration tests with files in utf-8 and windows-1252 encodings, testing all combinations
* fix bad smell test by excluding windows-1252 files from the utf8 checks
* fix bad smell test by excluding valid files from the smart quote test
* Support setting persistent command timeout per task basis
Fixes#42200
* Add variable `ansible_command_timeout` to `persistent_command_timeout`
option for `network_cli` and `netconf` connection plugin so that the
command_timeout can be set per task basis while using `connection=network_cli`
or `connection=netconf`
eg:
```
- name: run copy command
ios_command:
commands:
- show version
vars:
ansible_command_timeout: 40
```
* Modify `ansible-connection` to read command_timeout value from
connection plugin options.
* Add `ansible_command_timeout` to `persistent_command_timeout`
option in `persistent` to support `connection=local` so that
it is backward compatibilty
* To support `connection=local` pass the timeout value as variables
from persistent connection to `ansible-connection` instead of sending
it in playcontext
* Fix CI failure
* Fix review comment
This change adds a new argument to the telnet action plugin that will
cause the module to send a newline character before trying to login.
This is convienent when using telnet over a console connection that
needs an initial newline character to start the login process.
This change also will cause the sent command and command response to be
displayed on stdout when running in verbose mode (5 v's).
* win_reboot: fix 2.6 issues and better handle post reboot reboot
* changed winrm _reset to reset
* Add handler to reset calls when .reset() throws an AnsibleError on older hosts
* Moving back to _reset to get the issue fixed
* Fix tmpdir on non root become
- also avoid exception if tmpdir and remote_tmp are None
- give 'None' on deescalation so tempfile will fallback to it's default behaviour
and use system dirs
- fix issue with bad tempdir (not existing/not createable/not writeable)
i.e nobody and ~/.ansible/tmp
- added tests for blockfile case
* Revert "Temporarily revert c119d54"
This reverts commit 5c614a59a6.
* changes based on PR feedback and changelog fragment
* changes based on the review
* Fix tmpdir when makedirs failed so we just use the system tmp
* Let missing remote_tmp fail
If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side. It's better to
be alerted in this case than to silently ignore it.
jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side. empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
* Fix fetching old style facts in junos_facts module
Fixes#42298
* To fetch old style facts from junos device it requires
login credentials in provider, hence while using
connection=nectonf do not delete provider values
as it might not be ignored
* Add `ofacts` as one of the options in `gather_subset`
* Minor update in documentation
* Fix CI failure
* ensure copy action plugin returns an invocation in result
Fixes#18232
Previously the action plugin for copy, which runs operations on the
control host to orchestrate executing both local actions on the
control host and remote actions on the remote host, is not returning
invocation dict in the result dict, this happens here where the
return from _copy_file() is None
When force is True, the _execute_module() method is called, which
returns the dict containing the invocation key. This patch ensures
there is always an invocation key.
Signed-off-by: Adam Miller <admiller@redhat.com>
* fix conditional, handle content no_log
Signed-off-by: Adam Miller <admiller@redhat.com>
Removes the SSH enforcement because it breaks situations where, for example,
an admin maintains Ansible Tower and another team needs to use Ansible
with the BIG-IP modules, but the Tower admin does not want to install
the F5-SDK on the Ansible Tower instance.
In this situation, the user needs to delegate the execution of the BIG-IP
module to another linux host that does have the dependencies installed.
Usually this is done with delegation, often over SSH. Limiting the SSH
connection here prevents this from working.
* First pass at a src parameter that can be used in place of body. Supports binary files
* Add test for uri src body
* Bump version_added to 2.6
* Close the open file handle
* Add uri action plugin that handles src/remote_src
* Document remote_src
* Remove duplicate info about remote_src
* Bump version_added to 2.7
* All instances of local connection should use _remote_is_local=True. Fixes#40551
* Switch to instance attribute for synchronize
* Add test that shows that synchronize _remote_is_local addresses tmpdir building
* Add nxos_install_os integration tests
* Update call to check timers
* Update check_ansible_timer method
* Modify network_cli integration tests
* Add timer check for nxos_install_os
* Add comments for clear_persistent_sockets
* Update connection info for tests
* More updates
* Restructure files for provider and non-provider testing
* Update env var name and add check for ISSU switchover
* Added interpreter parameter to the script module
* Let required and default parameters get documented implicitly for binary parameter
* Renamed interpreter parameter to executable
* win_updates: add scheduled tasks back in for older hosts
* Fixed up typo in category name error message
* Fixed up some minor issues after merge
* added changelog fragment
* Default to become but add override to use scheduled tasks
* Added basic unit tests for win_updates
* fix minor typos
* Fix all cases with pause and ctrl+c
- naked:
- pause:
- with prompt
- pause: prompt=hi
- time wait
- pause: seconds=60
- time wait with prompt
- pause: seconds=60 prompt=hi
Fixes#35372
* Use curses to control stdout
* Use curses to clear lines on interactive input
* Validate input for echo parameter and fail nicely if invalid
* Add integration tests for pause module using pexpect
* Use try except when trying to determine erase sequence to account for lack of TTY in containers in tests
* Improve output validation for regular paus test
* Accept two digit precision for pause length in test
* Check for seconds when seconds is specificed, minutes when minutes is specified
* Add test for no TTY mode
Co-authored by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored by: Brian Coca <brian.coca+git@gmail.com>
* HTTPAPI connection
* Punt run_commands to cliconf or httpapi
* Fake enable_mode on eapi
* Pull changes to nxos
* Move load_config to edit_config for future-preparedness
* Don't fail on lldp disabled
* Re-enable check_rc on nxos' run_commands
* Reorganize nxos httpapi plugin for compatibility
* draft docs for connection: httpapi
* restores docs for connection:local for eapi
* Add _remote_is_local to httpapi
* Don't rewrite remote paths when remote is secretly local
* Remote overriding is configurable in connection
* Use c.DEFAULT_LOCAL_TMP instead of remote_tmp
* Remove remote_is_local from ConnectionBase
* remote_is_local->_remote_is_local
* Add warning signs to _remote_is_local and explanatory comments to its use
* Modifying cnos-facts, cnos_command and cnos-config in line with the design followed in Ansible. Adding unit test cases for these modules. Added plugins to support them.
* Removing doc fragment conflicts with other modules
* Replacing show with display
Fixes several bugs exposed in #34893
* Fixes relative path handling in copy so that it splits directories and
reconstructs the correct file path
* Return failed in the proper circumstances
* Changing Lenovo Inc to Lenovo and update License file to be consistent.
* Editing cli commands to expanded form. Changing username and password fields as per Ansible standards.
* WIP Pull persistent connection parameters via get_option
* Fix pep8
* Add use_persistent_connection setting to paramiko_ssh plugin
* Add vars section to persistent_command_timeout setting and prevail provider values over config manager
* Use persistent_command_timeout on network_cli instead of timeout
* Fix unit tests
If we don't call loader to get network_cli, then _load_name is never
set and we get KeyError.
* Pull persistent_command_timeout via config manager for ios connection local
* Pull persistent_command_timeout via config manager on connection local
* Initial commit
* Socket Timeout and dest file handler
* sftp handling
* module name change as per review
* multiple thread tmp file overwite problem
* Integration test suite for network_put
* add additional testcase for dest argument
* fix pylint/pep8/modules warnings
* add socket timeout for get_file
* network_get module
* pep8 issue on network_get
* Review comments
* create module tmpdir based on remote_tmp
* Source remote_tmp from controller if possible
* Fixed sanity test and not use lambda
* Added expansion of env vars to the remote tmp
* Fixed sanity issues
* Added note around shell remote_tmp option
* Changed fallback tmp dir to ~/.ansible/tmp to make shell defaults
* configurable list of facts modules
- allow for args dict for specific modules
- add way to pass parameters
- avoid facts poluting test
- move to 'facts gathered' flag
- add 'gathering' setting tests
* Remove use of six.b as Python-2.6+ have byte literals.
* Make AnsibleModule a global object so we'll have access to it in all
the functions we're going to break this up into.
* Rework the parameters so things that are in file_common_args are used
from file_common_args or the reason for deviation is documented.
* Remove validate as a parameter: this should be taken care of by
removing it from params before the copy and template action plugin
invoke file.
* Rename diff_peek to _diff_peek as it is an internal parameter.
* add module_name execute_module call to assemble so that it is more greppable
Under a non-utf-8 locale (for instance, LC_ALL=C), passing a non-ascii
filename to many APIs will traceback. Fix that by explicitly converting
to byte strings before passing to external APIs.
May fix#27262
* Fixes for mode=preserve
* Document mode=preserve for template and copy
* Make mode=preserve work with remote_src for copy
* Make mode=preserve work for template
* Integration tests for copy & template mode=preserve
Fixes#39279
* Changed mode option in win_copy to hidden option as it doesn't reflect copy mode
For connection=local check only if the transport value in
provider is cli and the respective module support cli
transport. If not report back appropriate error message.
* Add error handling for junos in case wrong connection type
Fixes#37990
If a junos module doesn't support given connection/transport type
return appropriate error message.
* Fix CI issues
* Fix review comment
* module_common: set required parameter templar
Fix the following error (related to b455901):
$ ./hacking/test-module -m ./lib/ansible/modules/system/ping.py -I ansible_python_interpreter=/usr/bin/python
Traceback (most recent call last):
File "./hacking/test-module", line 268, in <module>
main()
File "./hacking/test-module", line 249, in main
(modfile, modname, module_style) = boilerplate_module(options.module_path, options.module_args, interpreters, options.check, options.filename)
File "./hacking/test-module", line 152, in boilerplate_module
task_vars=task_vars
File "ansible/lib/ansible/executor/module_common.py", line 910, in modify_module
environment=environment)
File "ansible/lib/ansible/executor/module_common.py", line 736, in _find_module_utils
shebang, interpreter = _get_shebang(u'/usr/bin/python', task_vars, templar)
File "ansible/lib/ansible/executor/module_common.py", line 452, in _get_shebang
interpreter = templar.template(task_vars[interpreter_config].strip())
AttributeError: 'NoneType' object has no attribute 'template'
* module_common.modify_module: templar is required
* template: Add integration tests for `lstrip_blocks'
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Fix passing `trim_blocks' inline
Fix passing `trim_blocks' option to the template module as inline
argument. Previously passing the `trim_blocks' option inline instead of
using the YAML dictionary format resulted in it always being set to
`True', even if `trim_blocks=False' was used.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Add option to `lstrip_blocks'
Add option to set `lstrip_blocks' when using the template module to
render Jinja templates. The Jinja documentation suggests that
`trim_blocks' and `lstrip_blocks' is a great combination and the
template module already provides an option for `trim_blocks'.
Note that although `trim_blocks' in Ansible is enabled by default since
version 2.4, in order to avoid breaking things keep `lstrip_blocks'
disabled by default. Maybe in a future version it could be enabled by
default.
This seems to address issue #10725 in a more appropriate way than the
suggested.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Add integration tests for `trim_blocks'
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Check Jinja2 support for `lstrip_blocks'
Since the `lstrip_blocks' option was added in Jinja2 version 2.7, raise
an exception when `lstrip_blocks' is set but Jinja2 does not support it.
Check support for `lstrip_blocks' option by checking `jinja2.defaults'
for `LSTRIP_BLOCKS' and do not use `jinja2.__version__' because the
latter is set to `unknown' in some cases, perhaps due to bug in
`pkg_resources' in Python 2.6.6.
Also update option description to state that Jinja2 version >=2.7 is
required.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
Unified tmp accidentally removed the containing tmpdir from the list of
files to fix the permissions on when we're becoming a different
unprivileged user. This resulted in a visible bug for script but not
for patch. This is because patch also uploads the module to the same
temporary directory and the uploaded module also ends up calling
fixup_perms2() which includes the temporary directory. So by the time
patch needs to access the temporary patch file, the directory is
appropriately set.
script's breakage was visible because script does not upload a module
(it's akin to raw in this way). Therefore, we only call fixup_perms2()
once in script and so leaving out the tmpdir in script means that the
containing directory never has its permissions set appropriately.
Fixing both because it does not cause an extra round trip for patch so
any speedup would be minimal and it's better to fix the perms as close
as possible to where we know we need it. Otherwise, changes to
seemingly unrelated code later could end up breaking it.
Fixes#36398
Lance Richardson
Trishna Guha
Toshio Kuratomi
Matt Martz
Nathaniel Case
Nicolas Finné
Brian Coca
Lindsay Hill
Abhijeet Kasurde
Julien Champseix
Deepak Agrawal
Ganesh Nalawade
Peter Sprygada
Jordan Borean
Jerry Chong
Sam Doran
HD650
Adam Miller
Andrew Gaffney
John Imison
Biao Liu
Sloane Hertel
Tim Rupp
Anil Kumar Muraleedharan
Mike Wiebe
tranceitionalMynd
Carl Niger
Ricardo Carrillo Cruz
Martin Krizek
Abhijit Menon-Sen
Matt Clay
Samer Deeb
Pilou
Alex Tsitsimpis