Commit Graph

109 Commits (02893520a93f8c2c49577453411e8ee444ff8aa9)

Author SHA1 Message Date
Victor Volle c9d2a67ec7 16888: comment module_utils/urls.py (#16987)
* 16888: comment module_utils/urls.py

* improved documentation of module (mentioning ‘requests’)

* remove mentioning of implementation details (urllib)

* improved parameter and return value documentation
2016-08-10 08:39:48 -07:00
Toshio Kuratomi 5a3493be5f Port urls.py to python3 and other byte vs text fixes (#16124)
* Port urls.py to python3

Fixes (largely normalizing byte vs text strings) for python3

* Rework what we do with attributes that aren't set already.

* Comments
2016-06-04 16:19:57 -07:00
Matt Martz 878b0dca68 Use .code instead of .getcode() as py24 does not have .getcode(). Fixes https://github.com/ansible/ansible-modules-core/issues/3608 2016-05-13 09:44:00 -05:00
Toshio Kuratomi 2583f2774d Some beginning python3 porting for urls.py 2016-04-27 07:15:51 -07:00
Toshio Kuratomi a0f397eb6c Fix traceback in fetch_urls when status code is in info as well as the error. 2016-04-27 06:49:12 -07:00
Olivier GROSJEANNE 4647e8b74e HTTPError can also function as a non-exceptional file-like return value (#14915)
* HTTPError can also function as a non-exceptional file-like return value (the same thing that urlopen() returns)

* HTTPError - adding response to info dictionnary

* HTTPError - adding response to info dictionnary

* HTTPError - adding body response to info dictionnary
2016-04-25 13:21:45 -04:00
James Cammarata ed35e8bce2 Don't override the http status code when successful in urls.py 2016-04-18 13:20:24 -04:00
Matt Martz a985bf6a31 Don't pass context to urlopen, instead add it to the handlers. Fixes https://github.com/ansible/ansible-modules-core/issues/3437 2016-04-15 16:10:54 -05:00
Matt Martz 398218b6ea More intelligent building of the SSLValidationError message based on capabilities 2016-04-06 11:04:04 -05:00
Matt Martz 6e9c09d7f7 Utilize urllib3.contrib.pyopenssl functionality for SNI capability in python versions lacking SNI support
Also add SNI tests, move test_uri to destructive since we are messing with packages for SNI testing
2016-04-06 10:14:13 -05:00
Toshio Kuratomi 4b0aa1214c Ziploader
* Ziploader proof of concept (jimi-c)

* Cleanups to proof of concept ziploader branch:

* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
  systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
  we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
  is kept so that tracebacks show the real filenames that the errors
  appear in.

* Make sure we import modules that are used into the module_utils files that they are used in.

* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer

* Make it possible to set the module compression as an inventory var

This may be necessary on systems where python has been compiled without
zlib compression.

* Refactoring of module_common code:

* module replacer only replaces values that make sense for that type of
  file (example: don't attempt to replace python imports if we're in
  a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
  via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
  numbering)

* Don't repeat ourselves about detecting REPLACER

* Add an easy way to debug

* Port test-module to the ziploader-aware modify_module()

* strip comments and blank lines from the wrapper so we send less over the wire.

* Comments cleanup

* Remember to output write the module line itself in powershell modules

* for line in lines strips the newlines so we have to add them back in
2016-04-05 11:06:17 -07:00
Brian Coca c09e085480 Merge pull request #14735 from jsok/open_url-netrc
Lookup netrc for credentials in open_url
2016-03-30 07:41:56 -07:00
Daniel Kempkens 696b68f07a Check return value of get_distribution()
On none-Linux systems `get_distribution()` returns `None`, which fails in `fetch_url`, because the return value of `get_distribution()` is not checked before calling `lower()` on the result.
2016-03-16 10:49:21 +01:00
Matt Martz 098333b2ec Fix redirects for get_url
* fetch_url shouldn't both accept follow_redirects and support follow_redircts via module.params
* Default follow_redirects for open_url should be 'urllib2'
* Add redirect test for get_url
2016-03-07 15:44:31 -06:00
Jonathan Sokolowski 5ec7adc64d Lookup netrc for credentials in open_url 2016-03-07 09:38:22 +11:00
Matt Martz 6ff0b079b4 Support SSL validation with redirect control for python versions without ssl context 2016-03-02 16:30:16 -06:00
Matt Martz 0e57c577f4 Add RedirectHandler class and factory function for controlling redirects in urllib2 2016-02-05 12:12:04 -06:00
Toshio Kuratomi 33863eb653 Conditionally create the CustomHTTPSConnection class only if we have the required baseclasses.
Fixes #11918
2015-12-16 07:47:09 -08:00
Toshio Kuratomi 72a0654b81 Fixes for proxy on RHEL5 2015-12-16 07:47:07 -08:00
Toshio Kuratomi 73ead4fbba First attempt to fix https certificate errors through a proxy with python-2.7.9+
Fixes #12549
2015-12-16 07:46:54 -08:00
Eric Johnson dfad7c64fb Allow PATCH to list of valid HTTP methods 2015-11-03 17:38:52 +00:00
gduke 744679601f Remove mutable default arguments. 2015-10-20 22:13:23 -07:00
Toshio Kuratomi 28c933ce5e properly fail_json when we don't have ssl on a non-redhat distro 2015-10-16 08:06:32 -07:00
Toshio Kuratomi d2346fd2e2 Python2.4 compat fix 2015-07-27 15:34:51 -07:00
Toshio Kuratomi 8ee3b7384d Guard the PROTOCOL setting so that we work on older pythons 2015-07-24 15:07:02 -07:00
Toshio Kuratomi 3d3e1c82a2 Have openssl autonegotiate tls protocol on python < 2.7.9
This allows usage of tls-1.1 and tls-1.2 if the underlying openssl
library supports it.  Unfortunately it also allows sslv2 and sslv3 if
the server is only configured to support those.  In this day and age,
that's probably something that the server administrator should fix
anyhow.
2015-07-15 13:17:00 -07:00
Toshio Kuratomi 327b1676a8 Add support for SNI and TLS-1.1 and TLS-1.2 to the fetch_url() helper
Fixes #1716
Fixes #1695
2015-07-14 12:48:35 -07:00
Iiro Uusitalo 4e7542af37 Merge upstream changes 2015-07-10 08:44:20 +03:00
Iiro Uusitalo 403f4881ee Enables 'basic auth force' -feature globally 2015-07-09 23:11:52 +03:00
Toshio Kuratomi 9911a947ed Vendorize match_hostname code so that ansible can push it out to clients along with the code that uses it. 2015-06-25 08:17:58 -07:00
Toshio Kuratomi a1a7d6c462 Fix forwarding the user-given params from fetch_url() to open_url() 2015-06-23 15:17:26 -07:00
Toshio Kuratomi 4161d78a94 Split the fetch_url() function into fetch_url and open_url().
open_url() is suitable for use outside of a module environment.  Will
let us use open_url to do SSL cert verification in other, non-module
code.
2015-06-12 12:54:56 -07:00
Toshio Kuratomi afc19894e1 Make fetch_url check the server's certificate on https connections 2015-05-28 13:20:40 -07:00
Simon Dick 6e65ccabc3 Allow the use of HTTP on custom ports in the fetch_url function 2015-05-01 13:52:29 +01:00
Brian Coca 78e1a7ed93 Revert "Fix: Add support for SSL protocol version configuration option" 2015-01-19 08:36:17 -05:00
Brian Coca 9ccabbb95e Merge pull request #9808 from swimlappy/sslconfig
Fix: Add support for SSL protocol version configuration option
2015-01-16 10:25:48 -05:00
Jason Holland eedc51f213 Add support for SSL protocol version configuration option. Also fix 2 places where the SSL version was not being set properly. 2014-12-13 21:20:33 -06:00
Jason Holland 38dbce1527 Allow Ansible to honor the "no_proxy" environment varaible. 2014-12-13 21:12:23 -06:00
Jure Triglav 2f869a6309 Add the default Homebrew path for OpenSSL certs on OS X 2014-10-29 14:16:01 +01:00
James Cammarata e54178f904 Catch additional errors in fetch_url
Fixes #8971
2014-09-11 09:47:28 -05:00
James Cammarata 8bafc646cb Disable custom https handler for fetch_url on older pythons
Fixes #8898
2014-09-05 13:48:45 -05:00
James Cammarata cd99821f8a Fix py26 difference in ssl socket connect call from 99ba9d6 2014-09-03 10:04:02 -05:00
Dan Buch 290f2759e1 Supporting SSL cert location on SunOS (SmartOS, really) 2014-08-29 10:33:14 -04:00
James Cammarata 99ba9d6e24 Default fetch_url to use TLSv1 instead of SSLv2/3 2014-08-28 08:22:24 -05:00
James Cammarata d44ed533b3 Default use_proxy to True for fetch_url()
Also added some error handling to the fetch_url() call in the
apt_repository module, so that failures to look up the PPA info
are properly handled.

Fixes #7322
2014-05-19 23:04:13 -05:00
James Cammarata 08406c0ee2 Adding the capability to proxy the ssl cert check
The ssl cert check will now respect the http and https proxy
environment settings. The url may also have the username/password
embedded, in which case basic auth will be used to connect to the
proxy server.

Fixes #7413
2014-05-19 16:00:32 -05:00
Joost Cassee 978e6d2cd6 Make concatenating certs robust in urls.py
Add a newline after each certificate file explicitly to avoid problems
with files that do not end with a newline themselves.
2014-04-30 21:46:37 +02:00
James Cammarata 117952cf6c Fixing a bug in the new fetch_url username/password logic 2014-04-24 00:44:39 -05:00
James Cammarata 89fa9b7305 Add parameters to get_url for the url username/password
Fixes #6928
2014-04-24 00:26:50 -05:00
Wim 6ed4ca97a8 hint to install python-ssl on redhat systems 2014-04-18 00:07:07 +02:00
James Cammarata d240d073eb Changing SSL cert detection method to allow for auto-negotiation of SSL protocols
Fixes #6904
2014-04-15 13:45:21 -05:00
Matt Martz 1d3d73a0b6 Only write the DUMMY_CA_CERT on OS X 2014-03-19 09:01:13 -05:00
Matt Martz 3b5aa8bd30 Provide a dummy ca to allow OS X to do it's OpenSSL keychain magic 2014-03-18 17:16:44 -05:00
Greg Dallavalle 77229553a3 fetch_url: Avoid credential stripping for FTP-scheme URLs 2014-03-16 20:41:03 -05:00
James Cammarata 2c7d58abe0 Compile ca certs into a temp file to reduce number of attempts
For those who may have a large number of certs found, this can reduce
the number of ssl connections attempted.
2014-03-12 13:45:16 -05:00
James Cammarata a9017af2bb Adding validate_certs to all modules that use fetch_url 2014-03-12 10:19:54 -05:00
James Cammarata 7f38cff989 Remove unused code from get_ca_certs() function 2014-03-12 09:33:19 -05:00
James Cammarata 804e4166c8 Rewriting ssl validation to try multiple certs found in paths
Previously, the function checked only for a single CA root cert, however
some distributions may have multiple certs in a directory. This will now
try any .crt or .pem file contained within several common paths for
each platform.

Fixes #6412
2014-03-12 09:21:19 -05:00
James Cammarata 9730157525 Validate SSL certs accessed through urllib*
* Adds another module utility file which generalizes the
  access of urls via the urllib* libraries.
* Adds a new spec generator for common arguments.
* Makes the user-agent string configurable.

Fixes #6211
2014-03-10 16:06:52 -05:00