Commit Graph

880 Commits (bfbdb3f3f0fecf880cae6ae7ed8ff3e4a8daaa60)

Author SHA1 Message Date
Sloane Hertel 467a1f54a3 s3_bucket: fix python3 sorting incompatibility (#27502)
* s3_bucket: fix policy sorting for python3 so strings are evaluated as less than tuples.

Add tests to ensure this behavior is maintained.

* Fix s3_bucket comparison function to work on both Python 3.5 and 3.6

* s3_bucket: document that cmp_to_key is used for python 2.7.

Add another test for s3_bucket to compare policies of different sizes.

* fix pep8

* Work around code-smell grepping by not using the word 'cmp'.
2017-08-03 15:41:26 -04:00
Sloane Hertel dbbad16385 [cloud] New module: Add module for managing AWS Datapipelines (cloud/amazon/data_pipeline) (#22878)
* New module for managing AWS Datapipelines

* Supports create/activate/deactivate and deletion
* Handles idempotent creation by embeding the version in the
uniqueId field
* Waits for requested state to be reached, as Botocore doesn't
have waiters yet for datapipelines

* rename module, fix imports, add tags option, improve exit_json results, fix a couple bugs, add a TODO so I don't forget

fix pep8

allow timeout to be used for pipeline creation

make .format syntax uniform

fix pep8

fix exception handling

allow pipeline to be modified, refactor, add some comments, remove unnecessary imports

pipeline activation may not be in the activated state long

remove datapipeline version option

change a loop to a list comprehension

create idempotence by hashing the options given to the module minus the objects (which can be modified)

small bugfix

* data_pipeline unittests

make unittests pep8

fix bug in unittests

* remove exception handling that serves no purpose

* Fix python3 incompatibilities in datapipeline tests and add placebo fixture maybe_sleep for faster tests

Fix python3 incompatibilities in data_pipeline build_unique_id()

Don't delete a pipeline in diff_pipeline() because it's unexpected

Don't use time.time() because it causes an issue with placebo testing

re-recorded tests

fix pep8 in data_pipeline

Remove disable_rollback from tests

Make sure unique identifier is a string

re-record tests

* improve documentation and add another example

* use a placebo fixture instead of redundant code in tests

fix tests for PLACEBO_RECORD=false

* Fix data_pipeline docs

use isinstance instead of type()

fix documentation

* fix documentation

* Remove use of undefined variable from data_pipeline module and fix license

* fix copyright header
2017-08-03 15:04:10 -04:00
Gregory Danenberg 1e42e1e04c [cloud] cloudformation: fix changeset_name bug fix #27487 (#27518)
Closes #27487
2017-08-03 08:40:28 -04:00
Sloane Hertel b0efbc5418 [cloud] ec2_vpc_igw: fix check mode - fixes #27490 (#27637)
Fix check mode for adding/removing tags; boto's DryRun is not equivalent to check mode.

Fixes #27490
2017-08-02 13:47:13 -04:00
Sloane Hertel c02deef454 [cloud] filter by vpc and subnet if provided before terminating instances (#27394) 2017-08-02 13:43:24 -04:00
piotrkarbowski-ciklum a260d99f74 ec2_ami_copy.py: Fix WaiterError handling. (#27416)
* ec2_ami_copy.py: Fix WaiterError handling.

Change suggested by Adam Johnson at https://github.com/ansible/ansible/issues/26971

* ec2_ami_copy.py: WaiterError: fail_json: add exception parameter.

* ec2_ami_copy.py: import traceback
2017-08-02 13:05:08 -04:00
Brandon Davidson 3d20a2ea82 Fix typo in cloud/amazon/ec2_vpc_nacl_facts (#27534) 2017-08-02 12:26:28 -04:00
Abhijeet Kasurde be58fa979d Fix broken import in cloudformation
* Fix imports
* Add new GPLv3 Licence

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-08-02 09:05:25 -07:00
Jordan Bach 24e393aef1 allow use of jinja2 variables for ec2_group from_port/to_port params (#27145) 2017-08-02 10:26:38 +10:00
Sloane Hertel d8eccdda3a [cloud] make ec2 paging() more functional and add paging to ec2_lc_facts (#27355) 2017-08-01 15:06:46 -04:00
Sloane Hertel ad2c1e1838 [cloud] ec2_customer_gateway: add routing option to allow bgp_asn to be optional (#27276)
fixes #27157

 add routing option to allow bgp_asn to be optional
2017-08-01 14:54:34 -04:00
Sloane Hertel 61ebb89fe3 improve cloudformation failures (#27427) 2017-08-01 12:23:32 -06:00
Brandon Davidson 4ec8744b8b Ignore main route table association when determining which associations to remove. (#23781)
They can't be removed anyway, and will just throw an error if you try to disassociate them.
2017-08-01 12:10:16 -06:00
Jens Carl cc72dead45 New module: Add module to collect facts for Amazon Redshift Cluster (cloud/amazon/redshift_facts) (#22615)
* New module redshift_facts

Gather facts about aws Redshift cluster(s).

* Fix she-bang, PEP8 violations, and ansible metadata.

* Bump version_added to "2.4"

* Fix some typos and add supporting check mode

* Fix type of RETURN fields

Fix type of RETURN fields number_of_nodes and
automated_snapshot_retention_period from integer to int.

* Add traceback information to fail_json()
2017-08-01 12:09:55 -06:00
hyperized 00ef894f6c Add module to list S3 buckets (#27322)
* Add module to list S3 buckets

* Move check for boto3 before boto3 is attempted to be used
2017-08-01 09:25:54 -04:00
Will Thames f972994662 [cloud] fix VPC behavior for ec2_group module, improve integration tests (#27038)
* Add tests for group in a VPC

* Improve ec2_group output and documentation

Update ec2_group to provide full security group information
Add RETURN documentation to match

* Fix ec2_group creation within a VPC

Ensure VPC ID gets passed when creating security group

* Add test for auto creating SG

* Fix ec2_group auto group creation

* Add backoff to describe_security_groups

Getting LimitExceeded from describe_security_groups is definitely
possible (source: me) so add backoff to increase likelihood of
success.

To ensure that all `describe_security_group` calls are backed off,
remove implicit ones that use `ec2.SecurityGroup`. From there,
the decision to remove the `ec2` boto3 resource and rely on the client
alone makes good sense.

* Tidy up auto created security group

Add resource_prefix to auto created security group and delete
it in the `always` section.
Use YAML argument form for all module parameters
2017-08-01 06:53:43 -04:00
Toshio Kuratomi 9a55487dff Remove get_exception usage from amazon modules 2017-07-29 07:02:12 -07:00
Sloane Hertel 91781487ab aws_kms: don't append to unicode - fixes #25786 (#27352)
* AWS key management service fix; statement may not have a principal, and if there is only one AWS principal it will not be a list as expected. Fixes 25786.

* remove len(), only catch exception in function for json.dumps() failure

* use a defined variable and make formatting python 2.6 compatible
2017-07-28 19:36:37 +10:00
Nathaniel McAuliffe c94c2831da extracting list from CustomerGateways to match create (#24897)
* Limiting CustomerGateway to dictionary within list of CustomerGateways

* Copying CustomerGateways[0] to CustomerGateway only if it exsits
2017-07-27 09:12:20 -04:00
mdawid92 bfdf85e002 [cloud] IAM module returns created keys (#21237)
* return new key, porting https://github.com/ansible/ansible-modules-core/pull/3385/ by defionscode

* fix python3 compatibility

* fixed indentation

* added user_meta field
2017-07-26 16:24:29 -04:00
Toshio Kuratomi 225fa5d092 Fix undefined variables, basestring usage, and some associated python3 issues 2017-07-25 15:58:23 -07:00
Toshio Kuratomi 9f7b0dfc30 Remove automatic use of system six
* Enable the pylint no-name-in-module check.  Checks that identifiers in
  imports actually exist.  When we do this, we also have to ignore
  _MovedItems used in our bundled six.  This means pylint won't check
  for bad imports below ansible.module_utils.six.moves but that's
  something that pylint punts on with a system copy of six so this is
  still an improvement.
* Remove automatic use of system six.  The exec in the six code which
  tried to use a system library if available destroyed pylint's ability
  to check for imports of identifiers which did not exist (the
  no-name-in-module check).  That test is important enough that we
  should sacrifice the bundling detection in favour of the test.
  Distributions that want to unbundle six can replace the bundled six in
  ansible/module_utils/six/__init__.py to unbundle.  however, be aware
  that six is tricky to unbundle.  They may want to base their efforts
  off the code we were using:

  2fff690caa/lib/ansible/module_utils/six/__init__.py

* Update tests for new location of bundled six Several code-smell tests
  whitelist the bundled six library.  Update the path to the library so
  that they work.

* Also check for basestring in modules as the enabled pylint tests will
  also point out basestring usage for us.
2017-07-25 15:58:23 -07:00
Abhijeet Kasurde dfb97b2728 Remove excessive imports from cloudformation_facts (#27209)
Fix removes excessive imports from cloudformation_facts module.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-07-25 10:53:17 -04:00
Sloane Hertel 495f91158c fix RETURN docs 2017-07-25 05:21:58 -04:00
Constantin Bugneac aa341edf4c rds.py - added more attributes to the returned output 2017-07-25 05:21:58 -04:00
Toshio Kuratomi 035be94271 Fix cloudformation module for import of non-module_utils file 2017-07-24 18:19:32 -07:00
Toshio Kuratomi 84285a741c Various fixes to ec2_vpc_dhcp_options module
* Stringify exception with a nicer error and include traceback
* Give a nice message if boto is not installed
* Fix undefined vars.  Cleaning up all undefined variables so that we
  can run static analysis on new commits.

  References #27193
2017-07-24 14:59:19 -07:00
bfloyd89 1f637f06a5 Make lambda_event.py python 2.6 compatible (#27241)
"Changed in version 2.7: The positional argument specifiers can be omitted, so '{} {}' is equivalent to '{0} {1}'."
2017-07-24 17:01:19 -04:00
Rob 3401a4b330 Rename classic elb modules to match new names applied to application LB modules (#25745) 2017-07-24 16:28:02 -04:00
Sloane Hertel 79e54f4006 [cloud] s3_bucket: policy comparison - fixes #25428 (#25723)
Creates a new way to compare bucket policies by making their elements hashable & comparing that way after normalizing syntax like single-item lists.
2017-07-19 07:01:33 -04:00
Sloane Hertel d971629062 [cloud] Fix ASG updates when LaunchConfigurationName does not exist for ASG instances
when replacing launch configurations the previous launch config is removed from any instances (#26966)
2017-07-18 16:21:11 -04:00
Vinay Dandekar 5b109506c4 [cloud] Rename ec2_facts to ec2_metadata_facts (#26293)
- Update CHANGELOG with ec2_facts deprecation notice
2017-07-18 16:11:09 -04:00
Ted Timmons 65da6a0789 [cloud][py3] Use to_bytes in ec2_key module in two more places (#26936)
PR #23051 fixes the first failure of this, but it doesn't fix the successive calls.
2017-07-18 15:35:31 -04:00
Sloane Hertel cdd1a147f1 [cloud] fix exception examples in AWS guidelines to be compatible with python3 (#26948) 2017-07-18 09:35:19 -04:00
Sloane Hertel 66f5a094bc [cloud] iam_cert: allow paths to be specified so slurp is not necessary for remote host… (#26097)
* allow paths to be specified so slurp is not necessary for remote hosts to use

* Make requested changes

remove unused parameters

* remove module used out of scope

check the potential filepath to be true before checking isfile

remove required: false

* rephrase documentation

* remove 'lookups' from example
2017-07-17 15:00:54 -04:00
Toshio Kuratomi d64e291274 Fix modules' use of BOOLEANS*
* The canonical location of BOOLEANS has moved.  Switch imports to use that.
* clean up argument_spec use of booleans.
* Clean up imports to not use wildcards
* Remove usage of get_exception
2017-07-17 11:48:05 -07:00
Rob efbc65bff0 New module: Add elb_application_lb_facts (cloud/amazon/elb_application_lb_facts) (#24584)
* New module - elb_application_lb_facts

* Make sure tags dont get snake cased

* Add listeners and listener rules to application facts

* python3 compat fix
2017-07-17 12:00:58 -04:00
Rob d0d2beafba have elb_application_lb use modify_listeners to avoid removing/recreating them (#25650)
* Rework how listeners and rules and handled. Fixes #25270

* Tidy up, documentation and add rules to returned output

* Remove required=False from argument_spec

* Remove unused functions. Add or [] in case of no elb

* Handle when listners is None in ensure_listeners_default_action_has_arn
2017-07-17 14:33:04 +10:00
sramakr b980a5c02a Use Boto3 for ec2_group Fixes #23507 (#25340)
* Use Boto3 for ec2_group

Currently boto doesn't support ipv6. To support ipv6 in ec2_group, we need boto3.
boto3 has significant API changes, which caused more re-factoring for ec2_group module.
Added additional integration test to test_ec2_group role.

* Follow the standard for boto3 ansible

Fixed imports. Use boto3 ansible exception with camel_dict_to_snake_dict.
Refactored the call to authorize/revoke  ingress and egress.

* Removed dependancy with module ipaddress

Added new parameter called cidr_ipv6 for specifying
ipv6 addresses inline with how boto3 handles ipv6 addresses.

* Updated integration test

* Added ipv6 integration test for ec2_group

* Set purge_rules to false for integration test

* Fixed import statements

Added example for ipv6.
Removed defining HAS_BOTO3 variable and import HAS_BOTO3 from ec2.
Cleaned up import statements.

* Fixed exception handling

* Add IAM permissions for ec2_group tests

Missing AuthorizeSecurityGroupEgress necessary for latest tests

* Wrapped botocore import in try/except block

Import just botocore to be more similar to other modules
2017-07-17 12:03:31 +10:00
Will Thames df321e8579 [cloud] change iam_managed_policy return to snake_case (#26219)
* Change iam_managed_policy to return snake case

Import module_utils.ec2 methods explicitly

* iam_managed_policy: tidy argument spec

Remove unnecessary specifications (`default=None`, `required=False`)
Use `required_if` to check for `policy` when `state` is `present`

* Provide exception handling in iam_managed_policy

Pretty much all AWS API calls can go wrong, and we should handle them.

Update line wrapping to improve readability of method calls

* Improve error handling when policy version limit exceeded

Better document policy version limit exceeded, and check
the error code to see that that's actually what happened
(rather than e.g. no permissions to add a new version)

* iam_managed_policy: better handle pagination and retry
2017-07-13 21:30:41 -04:00
Takuya Sato 179e2882f7 ecs_taskdefinition module: containers.environment convert to string (fix #23297) (#23681)
It is to fix the failed comparison of containers.
2017-07-13 20:10:31 -04:00
Sloane Hertel ed51380c2c [cloud] ec2_vol module snapshot and volume_size are not mutually exclusive (#26764) 2017-07-13 15:08:03 -04:00
Hagai Kariti a2d4e083f1 ec2_vpc_route_table doc fix: 'subnets' isn't required (#26656) 2017-07-12 10:39:11 +10:00
Ryan Brown aa3975e084 [cloud] Stop CloudFormation module from always making changesets (#26619)
In Python a function is always truthy, and the name of the
`create_changeset` function was being accidentally used instead of
`module.params['changeset']`.
2017-07-10 16:57:37 -04:00
Will Thames f8d522de69 Support check mode for efs_facts (#26273)
Facts modules support check mode by default

Fix pep8 compliance
2017-07-05 15:06:08 +02:00
Todd Hertel e9447e9481 ec2_group fix TypeError bug #26291 2017-07-03 16:59:43 -04:00
Denis Afonso c9b00d2f02 Fixed the exception handling logic for the delete_group function. (#26109)
* Fixed he exception handling logic for the delete_group function.

fixes issue #26100

* Removed the unnecessary del_meta variables and made some other adjustments to the delete_user function
2017-06-30 14:12:50 -04:00
Will Thames 7ae4027c58 Improve ec2_vpc_subnet check mode (#23108)
check_mode should behave pretty similarly to non-check mode -
just don't actually create or delete subnets or change tags.

Using DryRun for check_mode behaves very differently and results
in the following module failure:

```
"msg": "Unable to update tags for subnet-abcd1234,
error: EC2ResponseError: 412 Precondition Failed
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<Response><Errors><Error><Code>DryRunOperation</Code>
<Message>Request would have succeeded, but DryRun flag
is set.</Message></Error></Errors>
<RequestID>12345678-abcd-1234-abcd-abcd1234abcd</RequestID></Response>"
```
2017-06-30 08:04:48 -04:00
Vinay Dandekar c884d4ab7f Add support for EC2 dynamic data in ec2_facts (#21532)
* Add support for EC2 dynamic data in ec2_facts

- Flattens out JSON in the instance identity document and IAM info/credentials for easy access to facts
- This changes region fact from ‘ansible_ec2_placement_region’ to ’ansible_ec2_instance_identity_document_region’

* Maintain backwards compatibility by putting the region into the old key

* Improve JSON parsing logic and split security group IDs

* Add documentation, backwards compatibility, fix bug and formatting

- Update documentation for ec2_facts with return values
- Preserve JSON value from the metadata service for backwards compatibility
- Fix bug in fix_invalid_varnames
  - The keys in the dict were being modified in place; new dict now created to hold the sanitized keys
  - Consolidate two replace calls with a regex substitution
- Move imports for ec2_facts to the top

* Add support for parsing the IAM instance profile role
2017-06-30 16:27:49 +10:00
Rob d1652aecf0 Upgrade ec2_remote_facts to use boto3 (#22937)
* Deprecate ec2_remote_facts in favor of ec2_instance_facts which uses boto3

* Update legacy-files.txt with deprecated file name change
2017-06-30 14:19:08 +10:00