* Mutually reference Windows and non-Windows modules
To make it easier for Windows or non-Windows users to find the relevant
module information, we are mutually referencing both variants in their
documentation.
We are also adding a special note if a module works on both Windows and
non-Windows targets.
* Mutually reference Windows and non-Windows modules
To make it easier for Windows or non-Windows users to find the relevant
module information, we are mutually referencing both variants in their
documentation.
We are also adding a special note if a module works on both Windows and
non-Windows targets.
* Replace 'look at' with 'use', as requested
ci_complete
Recent Python3 versions require open() to specify binary mode if the data is anything other than text.
Python3: Use int() instead of long() in unarchive
Changes long() to int() for CRC values in the unarchive module. Affects unarchiving of zip files. Since CRC values in zipfile are 32 bits the behaviour should be unchanged even in Python 2.
Changes to the metadata format were approved here:
https://github.com/ansible/proposals/issues/54
* Update documentation to the new metadata format
* Changes to metadata-tool to account for new metadata
* Add GPL license header
* Add upgrade subcommand to upgrade metadata version
* Change default metadata to the new format
* Fix exclusion of non-modules from the metadata report
* Fix ansible-doc for new module metadata
* Exclude metadata version from ansible-doc output
* Fix website docs generation for the new metadata
* Update metadata schema in valiate-modules test
* Update the metadata in all modules to the new version
* Avoid having module documentation links to itself
A lot of modules use M(own_module) in their documentation causing a link
in the documentation to itself.
* Make note more clear now
Since handler.files_in_archive is a list of files coming from
various executables output, that's a bytes list, and we use it
with dest who is a str. So we need to convert that to native
type.
Fixes#4063.
Tar does not use this parameter on extraction (-x) or diff (-d)(the
only two cases where it is passed in unarchive). It only uses it on
creation:
https://www.gnu.org/software/tar/manual/html_section/tar_33.html
Providing `unarchive` with a file mode of `0755` (octal) makes it pass
the argument `--mode 493` (493 = 0755 in decimal) to `tar`, which then
fails while verifying it (because it contains an invalid octal char
'9'). Not passing the parameter to tar solves the issue.
This means we will have to unarchive the complete archive if a single change is found.
Unfortunately we cannot fix this for `unzip`, the only hope is a pure-python reimplementation.
This fixes problems reported in the comments of #3810
* Ensure unicode characters in zip-compressed filenames work correctly
Another corner-case we are fixing hoping it doesn't break anything else.
This fixes:
- The correct encoding of unicode paths internally (so the filenames we scrape from the output and is returned by zipfile match)
- Disable LANG=C for the unzip command (because it breaks the unicode output, unlike on gtar)
* Fix for python3 and other suggestions from @abadger
* Improve the correct handling of gtar and unzip options
Add the option --show-transformed-names when extra_opts is being used
Ignore bogus warnings related to empty filenames
Properly quote _and_ escape filenames for unzip command
Rewrite gtar options and provide run_command with array, not string
This fixes#2480 and #4109.
* Make check-mode work for zip-files
Check-mode was disabled for zip-files since gtar did not support it.
This change enables check-mode support for zip-files, but does skip the task when used with gtar.
(Best of both worlds)
Also remove unused compress_mode variable.
This replaces PR #4401, the changes overlap somewhat so I merged them
When you try to remote unarchive files with the option copy=no the code always fail, as evidenced in issue #4202. That happens because the conditional to check "if remote_src=no or copy=yes" will always be true since the default value of them is remote_src=no and copy=yes.
My modification is only to change the condition from or to and, that way only if both the vars stay with the default value will be true, otherwise you can unarchive remote files.
A capital "S" appears when the the setuid or setgid bit are set but have no effect. Likewise, a capital "T" appears when the sticky bit is set but it has no effect.
* Revert PR #3575 since it causes problems related to exclude patterns
By using a different method for getting archive filelists, and extracting we introduced new problems related to excluding based on gtar patterns.
As a result files that would be excluded by gtar, would still be in the filelist. Implementing our own gtar compatible pattern exclusion mechanism is near to impossible (believe me, we looked at it...). The best way is to look at the original problem and deal with that, and ensure that extraction and filelists are done with the exact same tool and exact same options.
The solution is to decode the octal unicode representation in gtar's output back to unicode. Since gtar has no problem extracting these files in LANG=C, we simply has to compensate for it.
This reverts #3575 and fixes#11348.
* Implement codecs.escape_decode() instead of decode("string_escape") for python3
* A few more sanity checks for detecting unzip output that's not a file entry
Also note that there's a rounding error somewhere in the mtime
comparison code.
* Fix reference to sub-array
* Improve the unzip output scraping
Ensure we capture the complete file (also when it includes spaces).
Drop lines that do not conform (in length) to what we expect (e.g. header/footer).
This fixes#3813
* Fix how split() works
Fallback to unzip if zipfile fails and hope that unzip can deal with it
(sites have an easier time upgrading the unzip utility than all of
python).
https://bugs.python.org/issue3997Fixes#3560
* Detection of handler depends on the wrong handler failing to list the contents of the tarfile.
Use explicit compression types with the python tarfile library to
achieve that.
* bytearray isn't available in python2.4
* unarchive: use Python's tarfile module for tar listing
fixes https://github.com/ansible/ansible/issues/11348
Depending on the current active locale, `tar`'s file listing can end up
spitting backslash-escaped characters. Unfortunately, when that happens,
we end up with double-escaped backslashes, giving us a wrong path,
making our action fail.
We could try un-double-escaping our paths, but that would be complicated
and, I think, error-prone. The easiest way forward seemed to simply use
the `tarfile` module.
Why use it only for listing? Because the `unarchive` option also
supports the `extra_opts` option, and that supporting this would require
us to mimick `tar`'s interface.
For listing files, however, I don't think that the loss of `extra_opts`
support causes problems (well, I hope so).
* unarchive: re-add xz decompression support
Following previous change to use Python's `tarfile` module for tar file
listing, we lost `xz` decompression support. This commits re-add it by
adding a special case in `TarXzArchive` that pre-decompresses the source
file.
* WIP: Making unarchive idempotent
Currently unarchive is not idempotent and has many rough edges and bugs.
The current release is a workable improvement on many fronts:
- zip support is now idempotent (but gtar lacks check-mode)
- New option `exclude` to exclude specific paths/files
- New option `keep_newer` to exclude newer files on target
- New option `extra_opts` to influence unzip/gtar (like synchronize module)
The following items are still ongoing:
- Implement CRC32 support for .zip files
- Re-implement the zip support using native zipfile module
- Re-implement the gtar support using native tarfile/gzip/bz2 modules (lzma external)
- Implement check-mode (works in gzip, but fails using gtar)
- Implement diff-mode (discuss an appropriate output model, like synchronize module)
The re-implementation of unzip/gtar support using native python modules will not only simplify the codebase, additional functionality can be implemented correctly and identically, which is currently not possible. (Other archives could be implemented using native modules equally, incl. options)
* Assorted fixes to zip support (during quality checks)
- Support both rw---- and rwx--- permstr
- Better file type support (more qa needed)
- Symlink support
- Include fix from #3229
* Implement zip diff-mode (itemized change) and avoid changes permissions every time (!)
This commit implements:
- rsync-compatible itemized-change output in diff-mode (using zip)
- no longer changing permissions unconditionally (when idempotent)
* Small fixes to itemized change output
* Fixes to user/group ownership changes
- The implementation of user/group ownership is a bit more complex for idempotency
- We report when a ZIP file incorrectly tags a directory as a file/link
- We only offer diff output when there is a change
* Fix the handling of includes and excludes for unzip
* Remove test output from output (confuses easily)
* Logic and performance improvements to ownership handling, and umask fix
* Handle special files (type '?')
* Make exceptions compatible with python 2.4
* Implement CRC32 support
* Revert some unintended/unknown changes ?
* Taking over maintenance as offered by current maintainer
* Fix support for white-spaces in filenames
* Remove/rename incorrect regex
* Ensure that fat executables end up with execute permission
* Remove check_result from output when unchanged
* When unarchiving as a user, or when owner/group/mode is supplied --diff is insufficient
Only way to be sure is to check request with what is on disk (as we do for zip).
Leave this up to set_fs_attributes_if_different() instead of inducing a (false) change
* By default, don't send confusing check_results in verbose output
This fixes#74.
Without this change, a download failure may bail out with the message:
"Failure downloading http://foo/bar, 'NoneType' object has no attribute 'read'"
whereas with this fix, you'd get a proper error like:
"Failure downloading http://foo/bar, Request failed: <urlopen error [Errno 113] No route to host>"
or one of the many other possible download errors that can occur.
Dag Wieers
Abhijeet Kasurde
Matt Martz
Karl Bergström
Paul Montero
Brian Coca
Toshio Kuratomi
Tim Rupp
Matt Clay
sirkubax
Brian Coca
Fabio Alessandro Locati
Sam Doran
Michael Scherer
JesseEmond
Lucas Costa Beyeler
Ryan S. Brown
James Cammarata
neo
Jan Fader
Michael Scherer
Veaceslav Mindru
Virgil Dupras
Toshio Kuratomi