Commit Graph

64 Commits (1609afbd12abb68d1434ee5ce56fe09e12a53fdb)

Author SHA1 Message Date
Matt Clay 10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
2017-01-29 07:28:53 +00:00
Matt Clay e2c0b375d3 PEP 8 cleanup. (#20790)
* PEP 8 E115 cleanup.
* PEP 8 E131 cleanup.
2017-01-28 01:39:40 -08:00
Matt Clay d0d1158c5e PEP 8 cleanup. (#20789)
* PEP 8 E703 cleanup.
* PEP 8 E701 cleanup.
* PEP 8 E711 cleanup.
* PEP 8 W191 and E101 cleanup.
2017-01-28 00:12:11 -08:00
James Cammarata 8afa090417 Resolving differences in core modules post-merge 2016-12-08 11:25:36 -05:00
Toshio Kuratomi 7319104552 Refreshed metadata for core modules 2016-12-08 11:25:35 -05:00
Fabio Alessandro Locati c8fd24ee04 Call main in conditional way - cloud/amazon 2016-12-08 11:25:35 -05:00
Ted Timmons 367f675420 use 'six' for urlparse compatability (#5777)
* use 'six' for urlparse compatability
2016-12-08 11:25:34 -05:00
Sam Doran 38c0769abb Examples syntax batch5 (#5622)
* Change example syntax on supervisorctl module

* Change example syntax or _ec2_ami_search module

* Change example syntax on cloudformation module

* Change example syntax on ec2 module

* Change example syntax on ec2_facts module

* Change example syntax on ec2_eip module

* Change example syntax on rds module

* Change example syntax on route53 module

* Change example syntax on s3 module

* Change example syntax on digital_ocean module

* Change example syntax on docker_service module

* Change example syntax on cloudformation module

* Change example syntax on gc_storage module

* Change example syntax on gce module

* Change example syntax on gce_mig module

* Change example syntax on _glance_image module

* Change example syntax on _keystone_user module

* Change example syntax on _nova_keypair module

* Change example syntax on _quantum_floating module

* Change example syntax on _quantum_floating_ip_associate module

* Change example syntax on _quantum_network module

* Change example syntax on _quantum_router module

* Change example syntax on _quantum_router_gateway module

* Change example syntax on _quantum_router_interface module

* Change example syntax on _quantum_subnet module

* SQUASH _quantum_subnet

* Add missing quotes
2016-12-08 11:25:31 -05:00
Javier M. Mellid 49f9404cd2 Add s3_url requirement in doc when rgw support is enabled in s3.py
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
2016-12-08 11:24:38 -05:00
Javier M. Mellid 6346cad9d0 Add proper version_added for rgw option in s3.py
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
2016-12-08 11:24:38 -05:00
Javier M. Mellid 0b156aae16 Add Ceph RGW S3 compatibility
Ceph Object Gateway (Ceph RGW) is an object storage interface built on top of
librados to provide applications with a RESTful gateway to Ceph Storage
Clusters:

http://docs.ceph.com/docs/master/radosgw/

This patch adds the required bits to use the RGW S3 RESTful API properly.

Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
2016-12-08 11:24:38 -05:00
@ 3fa745eef9 Fixing error exception handling for python3. Does not need to be compa… (#3840)
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.

* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.

* Fixing compile time errors IRT error exception handling for Python 3.5.
This does not need to be compatible with Python2.4 b/c Boto is Python 2.6 and above.
2016-12-08 11:24:34 -05:00
Doug Luce 9167abdd10 Fix S3 unavailable region error
This is to address this error:

  fatal: [site]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to connect to S3: Region  does not seem to be available for awsmodule boto.s3. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path"}

Commit 0dd58e9 changed the logic so an exception is thrown (by
`connect_to_aws`) before the `s3 is None` check is performed. This
changes the `None` check to a catch so the old logic can compensate.
2016-12-08 11:24:30 -05:00
Brian Coca fdcc1fe763 clarified docs on overwrite 2016-12-08 11:24:25 -05:00
Ted Timmons 8c6741b2bb fix typo in documentation
let's -> lets
2016-12-08 11:24:13 -05:00
Brian Coca b61d531dfc corrected typo on aws connection arguments var 2016-12-08 11:24:10 -05:00
Will Thames 52accd7d1f Use connect_to_aws where possible
`connect_to_aws` fixes a bug with security tokens in AWS.
Modules should use that rather than calling
`boto.x.connect_to_region`
2016-12-08 11:24:09 -05:00
Matt Martz aaf762573e Various simple linting type cleanups on modules 2016-12-08 11:24:06 -05:00
Matt Ferrante 2dd53a8d91 updated s3 module documentation 2016-12-08 11:23:59 -05:00
Brian Coca f50fbe9dd2 updated s3 docs to reflect mode choices
fixes #2318
2016-12-08 11:23:58 -05:00
Greg DeKoenigsberg 248ea2deea Remove @ralph-tice from maintainership per his request 2016-12-08 11:23:45 -05:00
Val Komarov 0b294d69c5 Remove duplicate logic 2016-12-08 11:23:41 -05:00
Frank van Tol db6c6ec5c9 Update s3.py
We are copying from S3,  the bucket is the source, not the target.
2016-12-08 11:23:36 -05:00
Taneli Lepp fb317c72b6 Added option to set multiple ACLs for S3 objects. Also verifies the selected permissions against Boto's canned ACL strings list. 2016-12-08 11:23:35 -05:00
Till Backhaus e005ef0c36 Delete dead and broken code 2016-12-08 11:23:33 -05:00
Jason Cowley 2c511a9470 Add support for S3 canned permissions. resolves #1939 2016-12-08 11:23:32 -05:00
James Cammarata 99075976a8 Fix missing params to download_s3file in s3 2016-12-08 11:23:27 -05:00
James Cammarata 689f13548f Fixing s3 failures when bucket names contain dots 2016-12-08 11:23:27 -05:00
Brian Coca cd072e3090 added version added to headers in s3 2016-12-08 11:23:26 -05:00
Feanil Patel e4898a4348 If overwrite is set, download eagerly.
If the we are going to overwrite the file anyway,
there is no need to do any checksums locally or
grab the remote etag.
2016-12-08 11:23:26 -05:00
Ash Caire 682bf34a43 add headers param to s3 uploads 2016-12-08 11:23:25 -05:00
khassen 8b60dd349d Use the common/shared MD5 function. 2016-12-08 11:23:25 -05:00
ayush f46daacf67 Updated doc strings so each character isn't considered a line 2016-12-08 11:23:23 -05:00
Apoorva Kulkarni 1f35e578b4 s3 module: Add missing version tag to "encrypt" parameter 2016-12-08 11:23:20 -05:00
Patrick Roby e638913c97 iterate through all keys in a more pythonic manner 2016-12-08 11:23:15 -05:00
Patrick Roby bc5d79fc30 update documentation, adding new params 2016-12-08 11:23:15 -05:00
Patrick Roby b318bc8b43 Add support for listing keys in a specific S3 bucket
Includes support for specifying a prefix, marker, and/or max_keys. Returns a list of key names (as strings).
2016-12-08 11:23:11 -05:00
Dan Abel a51a8e1fd6 use aws connect calls that allow boto profile use 2016-12-08 11:23:10 -05:00
Greg DeKoenigsberg 18ca7aee07 Updating cloud modules with proper github author information 2016-12-08 11:23:07 -05:00
Tyler Cross 9babe9b07d Get specific object version with S3 module.
- allow specifying version for mode=get and mode=getstr
- when version specified doesn't exist give an error message that indicates so
2016-12-08 11:23:07 -05:00
Ching Yi, Chan be9cddbddf Refactoring for easier to read 2016-12-08 11:23:06 -05:00
Ching Yi, Chan 7cafbb5f05 Prevent memory-error from a large file 2016-12-08 11:23:06 -05:00
Bruce Pennypacker 4cb7df2631 Added delobj command to delete an object within a bucket 2016-12-08 11:23:06 -05:00
tedder dd6d5f0cb5 feature pull request: catch and retry recoverable errors
boto can throw SSLError when timeouts occur (among other SSL errors). Catch these so proper JSON can be returned, and also add the ability to retry the operation.

There's an open issue in boto for this: https://github.com/boto/boto/issues/2409

Here's a sample stacktrace that inspired me to work on this. I'm on 1.7, but there's no meaningful differences in the 1.8 release that would affect this. I've added line breaks to the trace for readability.

    failed to parse: Traceback (most recent call last):
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 2031, in <module> main()
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 353, in main download_s3file(module, s3, bucket, obj, dest)
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 234, in download_s3file key.get_contents_to_filename(dest)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1665, in get_contents_to_filename response_headers=response_headers)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1603, in get_contents_to_file response_headers=response_headers)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1435, in get_file query_args=None)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1488, in _get_file_internal for bytes in self:
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 368, in next data = self.resp.read(self.BufferSize)
      File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 416, in read return httplib.HTTPResponse.read(self, amt)
      File "/usr/lib/python2.7/httplib.py", line 567, in read s = self.fp.read(amt)
      File "/usr/lib/python2.7/socket.py", line 380, in read data = self._sock.recv(left)
      File "/usr/lib/python2.7/ssl.py", line 341, in recv return self.read(buflen)
      File "/usr/lib/python2.7/ssl.py", line 260, in read return self._sslobj.read(len) ssl.SSLError: The read operation timed out
2016-12-08 11:23:04 -05:00
Benno Joy 70188fe02a Fixes issue 1197 for s3 module, where the file gets downloaded even if checksums match 2016-12-08 11:23:03 -05:00
billwanjohi a6fd78af37 s3 put: add support for server-side encryption
- added 'encrypt' boolean option
- reordered module options alphanumerically
2016-12-08 11:23:02 -05:00
Toshio Kuratomi b6a312e489 Minor fixups found during review of #582 2016-12-08 11:23:02 -05:00
Brian Coca 953ed0aecc avoid exception when boto is not available 2016-12-08 11:22:56 -05:00
steenzout 67f769d9a6 issue #994: use HAS_BOTO to determine if import was successful: - removed import of sys module. - HAS_BOTO constant to check if import was successful. - trigger a failure when import fails. - removed unnecessary imports. 2016-12-08 11:22:53 -05:00
Mick Bass 5de4c053b6 clean rebase of https://github.com/ansible/ansible-modules-core/pull/1024 on a new branch 2016-12-08 11:22:51 -05:00