The new create option with the default value 'no' changes the
behavior from the previous Ansible releases. Change the default to
'yes' to create missing ini files by default.
Fixes: #5488
This fixes the behavior that the dest is directory,
when we set the "force: no" argument.
To be join the dest and the src's basename,
before checking the "force" argument.
In the description of the find module return value, the sample dict
has its key=value strings converted to key=value: None in the
web documentation. This commit updates the sample output to a 'real'
dict.
Minor additional edit in the description: "return list *of* files".
make format function 'format only'
added platform dependant info, when it is available
avoid rechecking same info
added comments to each info gathering section
(cherry picked from commit a79acf73d7eb79b76d808ff8a1d6c505dfd9ec82)
Since the module use re and os, we need to import them.
And rather than importing '*', we should limit to the
only object/function needed, so we can more easily refactor
later.
Since handler.files_in_archive is a list of files coming from
various executables output, that's a bytes list, and we use it
with dest who is a str. So we need to convert that to native
type.
Fixes#4063.
Tar does not use this parameter on extraction (-x) or diff (-d)(the
only two cases where it is passed in unarchive). It only uses it on
creation:
https://www.gnu.org/software/tar/manual/html_section/tar_33.html
Providing `unarchive` with a file mode of `0755` (octal) makes it pass
the argument `--mode 493` (493 = 0755 in decimal) to `tar`, which then
fails while verifying it (because it contains an invalid octal char
'9'). Not passing the parameter to tar solves the issue.
This means we will have to unarchive the complete archive if a single change is found.
Unfortunately we cannot fix this for `unzip`, the only hope is a pure-python reimplementation.
This fixes problems reported in the comments of #3810
* Ensure unicode characters in zip-compressed filenames work correctly
Another corner-case we are fixing hoping it doesn't break anything else.
This fixes:
- The correct encoding of unicode paths internally (so the filenames we scrape from the output and is returned by zipfile match)
- Disable LANG=C for the unzip command (because it breaks the unicode output, unlike on gtar)
* Fix for python3 and other suggestions from @abadger
* Python3 fixes to copy, file, and stat so that the connection integration tests can be run
* Forgot to audit the helper functions as well.
* Fix dest to refledt b_dest (found by @mattclay)
On python3, we want to use the surrogateescape error handler if
available for filesystem paths and the like. On python2, have to use
strict in these circumstances. Use the new error strategy for to_text,
to_bytes, and to_native that allows this.
Lineinfile deals heavily with Unic text files. Makes some sense to deal
with it all as byte strings. So there is a lot of work done here to
show that we're dealing with byte strings throughout.
* Improve the correct handling of gtar and unzip options
Add the option --show-transformed-names when extra_opts is being used
Ignore bogus warnings related to empty filenames
Properly quote _and_ escape filenames for unzip command
Rewrite gtar options and provide run_command with array, not string
This fixes#2480 and #4109.
* Make check-mode work for zip-files
Check-mode was disabled for zip-files since gtar did not support it.
This change enables check-mode support for zip-files, but does skip the task when used with gtar.
(Best of both worlds)
Also remove unused compress_mode variable.
This replaces PR #4401, the changes overlap somewhat so I merged them
* FreeBSD do not support --omit-header and --absolute-names
* The option for following symlink wth getfacl is different on FreeBSD
* ZFS on Freebsd use nfsv4 acls, who use a slightly different syntax
* FreeBSD do not have a --test flag, so always return 'True'
* FreeBSD do not have the --omit-headers options, so we have to filter by ourself
* Mark Freebsd as working for the acl module
When you try to remote unarchive files with the option copy=no the code always fail, as evidenced in issue #4202. That happens because the conditional to check "if remote_src=no or copy=yes" will always be true since the default value of them is remote_src=no and copy=yes.
My modification is only to change the condition from or to and, that way only if both the vars stay with the default value will be true, otherwise you can unarchive remote files.
A capital "S" appears when the the setuid or setgid bit are set but have no effect. Likewise, a capital "T" appears when the sticky bit is set but it has no effect.
* Revert PR #3575 since it causes problems related to exclude patterns
By using a different method for getting archive filelists, and extracting we introduced new problems related to excluding based on gtar patterns.
As a result files that would be excluded by gtar, would still be in the filelist. Implementing our own gtar compatible pattern exclusion mechanism is near to impossible (believe me, we looked at it...). The best way is to look at the original problem and deal with that, and ensure that extraction and filelists are done with the exact same tool and exact same options.
The solution is to decode the octal unicode representation in gtar's output back to unicode. Since gtar has no problem extracting these files in LANG=C, we simply has to compensate for it.
This reverts #3575 and fixes#11348.
* Implement codecs.escape_decode() instead of decode("string_escape") for python3
* A few more sanity checks for detecting unzip output that's not a file entry
Also note that there's a rounding error somewhere in the mtime
comparison code.
* Fix reference to sub-array
The example for delete=yes does not specify recursive although it is
required. In addition, the wording for the delete option is confusing
about from where files are really deleted. This should clarify that.
* Improve the unzip output scraping
Ensure we capture the complete file (also when it includes spaces).
Drop lines that do not conform (in length) to what we expect (e.g. header/footer).
This fixes#3813
* Fix how split() works
Fallback to unzip if zipfile fails and hope that unzip can deal with it
(sites have an easier time upgrading the unzip utility than all of
python).
https://bugs.python.org/issue3997Fixes#3560
* Detection of handler depends on the wrong handler failing to list the contents of the tarfile.
Use explicit compression types with the python tarfile library to
achieve that.
* bytearray isn't available in python2.4
* unarchive: use Python's tarfile module for tar listing
fixes https://github.com/ansible/ansible/issues/11348
Depending on the current active locale, `tar`'s file listing can end up
spitting backslash-escaped characters. Unfortunately, when that happens,
we end up with double-escaped backslashes, giving us a wrong path,
making our action fail.
We could try un-double-escaping our paths, but that would be complicated
and, I think, error-prone. The easiest way forward seemed to simply use
the `tarfile` module.
Why use it only for listing? Because the `unarchive` option also
supports the `extra_opts` option, and that supporting this would require
us to mimick `tar`'s interface.
For listing files, however, I don't think that the loss of `extra_opts`
support causes problems (well, I hope so).
* unarchive: re-add xz decompression support
Following previous change to use Python's `tarfile` module for tar file
listing, we lost `xz` decompression support. This commits re-add it by
adding a special case in `TarXzArchive` that pre-decompresses the source
file.
* WIP: Making unarchive idempotent
Currently unarchive is not idempotent and has many rough edges and bugs.
The current release is a workable improvement on many fronts:
- zip support is now idempotent (but gtar lacks check-mode)
- New option `exclude` to exclude specific paths/files
- New option `keep_newer` to exclude newer files on target
- New option `extra_opts` to influence unzip/gtar (like synchronize module)
The following items are still ongoing:
- Implement CRC32 support for .zip files
- Re-implement the zip support using native zipfile module
- Re-implement the gtar support using native tarfile/gzip/bz2 modules (lzma external)
- Implement check-mode (works in gzip, but fails using gtar)
- Implement diff-mode (discuss an appropriate output model, like synchronize module)
The re-implementation of unzip/gtar support using native python modules will not only simplify the codebase, additional functionality can be implemented correctly and identically, which is currently not possible. (Other archives could be implemented using native modules equally, incl. options)
* Assorted fixes to zip support (during quality checks)
- Support both rw---- and rwx--- permstr
- Better file type support (more qa needed)
- Symlink support
- Include fix from #3229
* Implement zip diff-mode (itemized change) and avoid changes permissions every time (!)
This commit implements:
- rsync-compatible itemized-change output in diff-mode (using zip)
- no longer changing permissions unconditionally (when idempotent)
* Small fixes to itemized change output
* Fixes to user/group ownership changes
- The implementation of user/group ownership is a bit more complex for idempotency
- We report when a ZIP file incorrectly tags a directory as a file/link
- We only offer diff output when there is a change
* Fix the handling of includes and excludes for unzip
* Remove test output from output (confuses easily)
* Logic and performance improvements to ownership handling, and umask fix
* Handle special files (type '?')
* Make exceptions compatible with python 2.4
* Implement CRC32 support
* Revert some unintended/unknown changes ?
* Taking over maintenance as offered by current maintainer
* Fix support for white-spaces in filenames
* Remove/rename incorrect regex
* Ensure that fat executables end up with execute permission
* Remove check_result from output when unchanged
* When unarchiving as a user, or when owner/group/mode is supplied --diff is insufficient
Only way to be sure is to check request with what is on disk (as we do for zip).
Leave this up to set_fs_attributes_if_different() instead of inducing a (false) change
* By default, don't send confusing check_results in verbose output
This fixes#74.
now uses atomic move to avoid data corruption
correclty cleans up temp files in every case
returns backup_file info if needed
validate validate before temp file gets created
backup AFTER validate
Without this change, a download failure may bail out with the message:
"Failure downloading http://foo/bar, 'NoneType' object has no attribute 'read'"
whereas with this fix, you'd get a proper error like:
"Failure downloading http://foo/bar, Request failed: <urlopen error [Errno 113] No route to host>"
or one of the many other possible download errors that can occur.
Using the difflist feature added in ansible/ansible@c337293 we can add
two diffs to the `diff` dict returned as JSON: A `before` and `after` pair of
changed file contents and the diff of the file attributes.
n.b.: the difflist handling from the above commit is logically broken.
PR will follow.
Example output:
TASK [change line and mode] ************************************************************
changed: [localhost]
--- before: /tmp/sshd_config (content)
+++ after: /tmp/sshd_config (content)
@@ -65,21 +65,21 @@
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no
#MaxStartups 10:30:60
#Banner /etc/issue.net
# Allow client to pass locale environment variables
-AcceptEnv LANG LC_*
+AcceptEnv LANG LC_* GF_ENV_*
Subsystem sftp /usr/lib/openssh/sftp-server
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
--- before: /tmp/sshd_config (file attributes)
+++ after: /tmp/sshd_config (file attributes)
@@ -1,3 +1,3 @@
{
- "mode": "0700"
+ "mode": "0644"
}
The parameters 'diff_peek' and 'validate' are not expected to be used
by users. They are internal. To make it clear, this change adds the
comments 'Internal use only' to each of those definitions to make
it clear that they are actually used, just not by end-users.
The 'diff_peek' option isn't documented at all, and provides a
rudimentary check that the content isn't binary. Documentation is
added to explain the option.
The 'validate' option has a declaration, but isn't implemented.
Therefore it may as well be removed from the module.
Specifically, the stat module now has a checksum_algorithm parameter.
This lets the module utilize one of the hash algorithms available on the host
to return the checksum of the file.
This change is backwards compatible. The checksum_algorithm defaults to
sha1 and still returns its result to the stat.checksum property.
An attempt to make clear how privilege escalation works with respect to the src/source host and dest/destination host. One existing note was incorporated into three new ones, iterating each.
It is not documented in [the Ansible doc page][1] nor
[the BSD setfacl man entry][2] (which means it might not be compatible
with BSD) so removing it does not break the API.
On the other hand, it does not conform with POSIX 1003.1e DRAFT
STANDARD 17 according to the [Linux setfacl man entry][3] so safer to
remove.
Finally, the most important reason: in non POSIX 1003.e mode, only ACL
entries without the permissions field are accepted, so having an
optional field here is very much error-prone.
[1]: http://docs.ansible.com/ansible/acl_module.html
[2]: http://www.freebsd.org/cgi/man.cgi?format=html&query=setfacl(1)
[3]: http://linuxcommand.org/man_pages/setfacl1.html
- Make build_entry compatible with Python 2.4
- Re-add missing warning/comment that was forgotten while refactoring
- Replace `all()` with a good ol' for-loop Python 2.4 compatibility
- Make a condition check more explicit (when `state` is `query`)
- Make sure this module can only be run with on a Linux distribution
- Add a note about Linux-only support in the documentation
- Set the version in which recursive support was added, 2.0
If we try to make a directory, but someone else creates the directory
at the same time as us, we don't need to raise that error to the user.
They asked for the directory to exist, and now it does. This fixes
the race condition which was causing that error to be raised, and
closes#1648.
When using YAML multi-line strings, e.g.:
- lineinfile:
dest: /tmp/foo
line: >
foo
bar
the line already ends with a newline. If an extra newline is appended unconditionally it will lead to inserting an extra newline on each run.
Sam Doran
Brian Coca
Gyorgy Szombathelyi
René Moser
Michael Herold
zuiurs
bencomp
Evan Kaufman
Timothy Appnel
Luca Berruti
Brian Coca
Charles Zaffery
Michael Scherer
John R Barker
Toshio Kuratomi
Matt Clay
Andrew Gaffney
JesseEmond
nichivo
Brian Maddy
Dag Wieers
James Pharaoh
Gennady Trafimenkov
Michael Scherer
Chris Short
Lucas Costa Beyeler
peter.jang
David Chwalisz
Ryan S. Brown
tobixx
James Cammarata
Lukas Grossar
Keisuke Kamada
neo
Matt Fischer
Brendan Jurd
Matthias Frey
Jan Fader
Marcos Diez
Veaceslav Mindru
Virgil Dupras
yarick123
Mengdi Gao
Bence Nagy
Matt Martz
Bert Van Vreckem
Toshio Kuratomi
Jose Moreira
Guruprasad
Tobias Wolf
Adam Fields
quoing
Ales Nosek
krdlab
Dylan Martin
Charles Ferguson
Charles Paul
Maarten Claes
Ryan Sydnor
varnav
skylerbunny
Jérémie Astori
Matthias Frey
Marius Gedminas
Markus Juenemann
Mike Putnam
Jeff Widman
Isaac Simmons
Patrik Lundin
Mitchell Ludwig
Matthew Gilliard
verm666
Vladimir Martsul
gimoh
Juan Picca
Jonathan Mainguy
Ritesh Khadgaray
Greg DeKoenigsberg
Tristan Fisher
jaypei
Greg Taylor
Jason Paige
Ryan Hartkopf