Fallback to unzip if zipfile fails and hope that unzip can deal with it
(sites have an easier time upgrading the unzip utility than all of
python).
https://bugs.python.org/issue3997Fixes#3560
* Detection of handler depends on the wrong handler failing to list the contents of the tarfile.
Use explicit compression types with the python tarfile library to
achieve that.
* bytearray isn't available in python2.4
* unarchive: use Python's tarfile module for tar listing
fixes https://github.com/ansible/ansible/issues/11348
Depending on the current active locale, `tar`'s file listing can end up
spitting backslash-escaped characters. Unfortunately, when that happens,
we end up with double-escaped backslashes, giving us a wrong path,
making our action fail.
We could try un-double-escaping our paths, but that would be complicated
and, I think, error-prone. The easiest way forward seemed to simply use
the `tarfile` module.
Why use it only for listing? Because the `unarchive` option also
supports the `extra_opts` option, and that supporting this would require
us to mimick `tar`'s interface.
For listing files, however, I don't think that the loss of `extra_opts`
support causes problems (well, I hope so).
* unarchive: re-add xz decompression support
Following previous change to use Python's `tarfile` module for tar file
listing, we lost `xz` decompression support. This commits re-add it by
adding a special case in `TarXzArchive` that pre-decompresses the source
file.
* WIP: Making unarchive idempotent
Currently unarchive is not idempotent and has many rough edges and bugs.
The current release is a workable improvement on many fronts:
- zip support is now idempotent (but gtar lacks check-mode)
- New option `exclude` to exclude specific paths/files
- New option `keep_newer` to exclude newer files on target
- New option `extra_opts` to influence unzip/gtar (like synchronize module)
The following items are still ongoing:
- Implement CRC32 support for .zip files
- Re-implement the zip support using native zipfile module
- Re-implement the gtar support using native tarfile/gzip/bz2 modules (lzma external)
- Implement check-mode (works in gzip, but fails using gtar)
- Implement diff-mode (discuss an appropriate output model, like synchronize module)
The re-implementation of unzip/gtar support using native python modules will not only simplify the codebase, additional functionality can be implemented correctly and identically, which is currently not possible. (Other archives could be implemented using native modules equally, incl. options)
* Assorted fixes to zip support (during quality checks)
- Support both rw---- and rwx--- permstr
- Better file type support (more qa needed)
- Symlink support
- Include fix from #3229
* Implement zip diff-mode (itemized change) and avoid changes permissions every time (!)
This commit implements:
- rsync-compatible itemized-change output in diff-mode (using zip)
- no longer changing permissions unconditionally (when idempotent)
* Small fixes to itemized change output
* Fixes to user/group ownership changes
- The implementation of user/group ownership is a bit more complex for idempotency
- We report when a ZIP file incorrectly tags a directory as a file/link
- We only offer diff output when there is a change
* Fix the handling of includes and excludes for unzip
* Remove test output from output (confuses easily)
* Logic and performance improvements to ownership handling, and umask fix
* Handle special files (type '?')
* Make exceptions compatible with python 2.4
* Implement CRC32 support
* Revert some unintended/unknown changes ?
* Taking over maintenance as offered by current maintainer
* Fix support for white-spaces in filenames
* Remove/rename incorrect regex
* Ensure that fat executables end up with execute permission
* Remove check_result from output when unchanged
* When unarchiving as a user, or when owner/group/mode is supplied --diff is insufficient
Only way to be sure is to check request with what is on disk (as we do for zip).
Leave this up to set_fs_attributes_if_different() instead of inducing a (false) change
* By default, don't send confusing check_results in verbose output
This fixes#74.
now uses atomic move to avoid data corruption
correclty cleans up temp files in every case
returns backup_file info if needed
validate validate before temp file gets created
backup AFTER validate
Without this change, a download failure may bail out with the message:
"Failure downloading http://foo/bar, 'NoneType' object has no attribute 'read'"
whereas with this fix, you'd get a proper error like:
"Failure downloading http://foo/bar, Request failed: <urlopen error [Errno 113] No route to host>"
or one of the many other possible download errors that can occur.
Using the difflist feature added in ansible/ansible@c337293 we can add
two diffs to the `diff` dict returned as JSON: A `before` and `after` pair of
changed file contents and the diff of the file attributes.
n.b.: the difflist handling from the above commit is logically broken.
PR will follow.
Example output:
TASK [change line and mode] ************************************************************
changed: [localhost]
--- before: /tmp/sshd_config (content)
+++ after: /tmp/sshd_config (content)
@@ -65,21 +65,21 @@
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no
#MaxStartups 10:30:60
#Banner /etc/issue.net
# Allow client to pass locale environment variables
-AcceptEnv LANG LC_*
+AcceptEnv LANG LC_* GF_ENV_*
Subsystem sftp /usr/lib/openssh/sftp-server
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
--- before: /tmp/sshd_config (file attributes)
+++ after: /tmp/sshd_config (file attributes)
@@ -1,3 +1,3 @@
{
- "mode": "0700"
+ "mode": "0644"
}
The parameters 'diff_peek' and 'validate' are not expected to be used
by users. They are internal. To make it clear, this change adds the
comments 'Internal use only' to each of those definitions to make
it clear that they are actually used, just not by end-users.
The 'diff_peek' option isn't documented at all, and provides a
rudimentary check that the content isn't binary. Documentation is
added to explain the option.
The 'validate' option has a declaration, but isn't implemented.
Therefore it may as well be removed from the module.
Specifically, the stat module now has a checksum_algorithm parameter.
This lets the module utilize one of the hash algorithms available on the host
to return the checksum of the file.
This change is backwards compatible. The checksum_algorithm defaults to
sha1 and still returns its result to the stat.checksum property.
An attempt to make clear how privilege escalation works with respect to the src/source host and dest/destination host. One existing note was incorporated into three new ones, iterating each.
It is not documented in [the Ansible doc page][1] nor
[the BSD setfacl man entry][2] (which means it might not be compatible
with BSD) so removing it does not break the API.
On the other hand, it does not conform with POSIX 1003.1e DRAFT
STANDARD 17 according to the [Linux setfacl man entry][3] so safer to
remove.
Finally, the most important reason: in non POSIX 1003.e mode, only ACL
entries without the permissions field are accepted, so having an
optional field here is very much error-prone.
[1]: http://docs.ansible.com/ansible/acl_module.html
[2]: http://www.freebsd.org/cgi/man.cgi?format=html&query=setfacl(1)
[3]: http://linuxcommand.org/man_pages/setfacl1.html
- Make build_entry compatible with Python 2.4
- Re-add missing warning/comment that was forgotten while refactoring
- Replace `all()` with a good ol' for-loop Python 2.4 compatibility
- Make a condition check more explicit (when `state` is `query`)
- Make sure this module can only be run with on a Linux distribution
- Add a note about Linux-only support in the documentation
- Set the version in which recursive support was added, 2.0
If we try to make a directory, but someone else creates the directory
at the same time as us, we don't need to raise that error to the user.
They asked for the directory to exist, and now it does. This fixes
the race condition which was causing that error to be raised, and
closes#1648.
When using YAML multi-line strings, e.g.:
- lineinfile:
dest: /tmp/foo
line: >
foo
bar
the line already ends with a newline. If an extra newline is appended unconditionally it will lead to inserting an extra newline on each run.
If insertbefore/insertafter didn't match anything, lineinfile module was doing nothing, instead of adding the line at end of fille as it's supposed to.
Having read the doc for this module several times and completely missing that it can be used for existing remote archives, I propose this update to the wording to make clear from the top the two ways in which this module can be used.
ansible-doc expects the value of the description field to be a list,
otherwise the output is not correct. This patch updates the flat
description to be a list.
This is a further fix for: https://github.com/ansible/ansible/issues/9092
when the relative path contains a subdirectory. Like:
ansible localhost -m copy -a 'src=/etc/group dest=foo/bar/'
This small change corrects behavior when one uses an .rsync-filter file to exclude some paths from both being transferred and being deleted, so that these excluded paths can be handled separately with different tasks (e.g. in order to deploy the excluded paths independently from the rest paths and notify handlers appropriately). The problem with the double -FF option is that it excludes the .rsync-filter file from being transferred to the receiver. However, deletions are done on the side of the receiver, so it is absolutely necessary the .rsync-filter file to be transferred to the receiver, so that the receiver knows what files to delete and what not to delete.
If you try to set rwX permissions, ACL fails to set them at all.
Expected:
$ sudo setfacl -m 'group::rwX' www
...
drwxrwxr-x 2 root root 4096 Nov 10 17:09 www
With Ansible:
acl: name=/var/www permissions=rwX etype=group state=present
...
drwxrw-r-x 2 root root 4096 Nov 10 17:30 www
x for group is erased. =/
In particular, if `rsync` is not installed on the remote machine the following error message will be encountered:
"rsync error: remote command not found"
This can be tested with this command :
ansible -c local -m copy -a 'src=/etc/group dest=foo/' all
This is a corner case of the algorithm used to find how we should
copy recursively a folder, and this commit detect it and avoid it.
Check https://github.com/ansible/ansible/issues/9092 for the story
Encouraging users to use this Ansible module to enable SELinux seems
like a better idea. It also warms Dan Walsh's heart.
Signed-off-by: Major Hayden <major@mhtx.net>
Matthias Frey
Jan Fader
Dag Wieers
Toshio Kuratomi
Marcos Diez
Michael Scherer
Michael Scherer
Veaceslav Mindru
Virgil Dupras
yarick123
Mengdi Gao
peter.jang
Bence Nagy
Brian Coca
Matt Martz
Bert Van Vreckem
Toshio Kuratomi
Jose Moreira
Guruprasad
Tobias Wolf
Adam Fields
quoing
Ales Nosek
krdlab
Dylan Martin
Charles Ferguson
Charles Paul
Maarten Claes
Timothy Appnel
Ryan Sydnor
varnav
James Cammarata
skylerbunny
Jérémie Astori
Matthias Frey
Marius Gedminas
Markus Juenemann
Mike Putnam
Jeff Widman
Isaac Simmons
Patrik Lundin
Mitchell Ludwig
Matthew Gilliard
verm666
Vladimir Martsul
gimoh
Juan Picca
Jonathan Mainguy
Ritesh Khadgaray
Greg DeKoenigsberg
Tristan Fisher
jaypei
Greg Taylor
Jason Paige
Ryan Hartkopf
James Bowes
Piotr Gosławski
blxd
mrsheepuk
Gauvain Pocentek
Rahul Mehrotra
Petros Moisiadis
sysadmin75
Jesse Buchanan
anatoly techtonik
Jeff Bradberry
follower
Michael Scherer
Eric
Jeff Geerling
Ted Timmons
Jon Hadfield
Tadej Janež
Joe Adams
Tongliang Liu
Eric Hanchrow
Major Hayden
Michael DeHaan