Commit Graph

1225 Commits (e0945900df3789895415e97772eea1c79821ee67)

Author SHA1 Message Date
Will Thames aa1e491244 [cloud aws_s3] Enforce required parameters for the various aws_s3 modes (#36767)
[cloud aws_s3] Most modes require `object` parameter, and this is easy to
get wrong (e.g. through confusion with the `dest` parameter). As
it's as easy to enforce, let's do that.
2018-03-02 16:33:04 -05:00
Will Thames 836d7a42a4 Remove old boto documentation and promote AnsibleAWSModule (#36763)
There should no longer be a need for boto documentation - it
just adds significant extra clutter to the guidelines.

AnsibleAWSModule greatly improves the codebase to such an extent
that we should mandate it for new code unless there is a
documented good reason.
2018-03-02 14:03:12 -05:00
Ryan Brown 2f72d082c0 Use Tower v2 API when available, falling back to v1 where necessary (#36663)
* Use Tower v2 API when available, falling back to v1 where necessary
2018-03-02 11:31:53 -05:00
Kevin Breit a3390b63fe Reversed orser of icmp_type and icmp_code per bug 36378 (#36380) 2018-03-02 16:09:32 +01:00
Ed Costello 07bc98f019 Additions to AWS Module Guidlines from Pull Request Feedback (#36834)
* Update doc of AnsibleAWSModule to remove incorrect connect example.

Current example uses aws_connect method which no longer exists. Replace
this with the client and resource methods that do exist.

Also remove try/except block in connect example as guidelines imply that
there aren't connection errors thrown on connection, just when later
sing the connection.

* Update AWS module guidelines to include the AnsibleAWSModule connection methods.

* Add information on integration testing to the AWS module guidelines.

* Add information on common documentation fragments to AWS module
Guidelines.

* Changes as requested on PR.

* Restructure connection section to start with current best practice
* Explain the use of the CI build groups
* Use YAML Anchors for aws credentials example

* Replace guidance on use of test groups with link to aliases file.

This achieves the goal of explaining why this file is necessary without
introducing overhead of keeping documentation up to date when test
groups change.
2018-03-02 11:40:01 +10:00
Sloane Hertel 9deef5626f Only create tags if tags are provided (#36921) 2018-03-01 15:30:08 -05:00
mikedlr 3f19ef680a aws_ssm_parameter_store module - value parameter should be no_log since it's often a secret (#36843) 2018-02-28 14:40:58 +00:00
Ed Costello 645952c139 Add aws_caller_facts module and use it in setup-iam.yml (#36683)
* Add aws_caller_facts module and use it in setup-iam.yml

This removes the dependency on having the command line AWS tools
installed.
2018-02-28 16:30:34 +10:00
Julien Vey 57f01c63e9 Add missing import keyword into snippet in Guidelines
PR #36779
2018-02-27 15:22:00 +01:00
Julien Vey d212bcf1e7 ec2_group:documentation explicit a behaviour for sg rules (#36586) 2018-02-23 09:24:25 +10:00
Julien Vey 3abaa43b1c efs: improve documentation about the update of an efs (#36473) 2018-02-22 10:30:50 +10:00
Julien Vey a77b89e7b5 [cloud] ec2_group: fix check mode when using tags (#36503)
Fixes #36412
2018-02-21 14:48:51 -05:00
Ed Costello ca59a4ede4 Better handling of absent AWS SES identity notification information. (#36354)
* Better handling of absent AWS SES identity notification information.

Fixes #36065

aws_ses_identity module now handles the cases where information about
the notification setup for the identity isn't returned by the AWS api.

This seems to happen in an edge case, believed to be eventual
consistency on registering new identities. So this case is treated
as if has been no notification setup for the identity yet.

Also fix 2 flake8 warnings in the module, a missing newline and unused
import.

* Increase the Boto Retries on SES APIs to deal with throttling.

This should address the unstable integration test failing due to
parallel runs in shippable hitting AWS throttling.

* Add retries loading SES details for inclusion in successful response.

There seems to be an eventual consistency behaviour with identity
registration. It's possible to still get no identity back after
registration.

This can cause failures in the shippable builds. This should fix that by
creating a retry of retrieving the identity information after
registration.

A similar retry loop has been added to notification attributes to ensure
this doesn't suffer from the same failure.

* Add missing sleep in get_notification_attributes to avoid busy loop.
2018-02-21 09:33:33 -05:00
Sloane Hertel 710db04ebd [ec2_vpc_net_facts] Fix UnsupportedOperation for regions other than u… (#35302)
* [ec2_vpc_net_facts] Fix UnsupportedOperation for regions other than us-east-1

* Make fix more Pythonic

* Fix the exception handling for the module
2018-02-21 09:11:36 -05:00
Sloane Hertel f7d79d4789 [cloud] Retry WAF actions on WAFStaleDataException (#36405)
Add a util to run functions with AWSRetry to retry on WAFStaleDataExceptions and update ChangeToken for each attempt
2018-02-21 08:14:17 -05:00
Ryan Brown 66c38dd6c3 Change to longer interval jittered backoffs for efs facts (#36266)
* Change to longer interval jittered backoffs for security group fact gathering

* Add the ECS throttling error code to the list of things to retry on
2018-02-20 10:17:31 -05:00
Ryan Brown e9c57e732f Add AWSRetry when describing VPCs to help stabilize integration tests (#36264)
* Add AWSRetry when describing VPCs to help stabilize integration tests

* Add retry on create_tags because it is possible to reach this API call before the VPC is finished creating
2018-02-20 08:56:34 -05:00
kbreit 6a3fd92654 Improved language around in_vpc parameter. (#36137) 2018-02-16 13:14:58 +00:00
Ryan Brown 53a314f767 Fix incorrect docs in `ec2_tag` per #35738 (#36268) 2018-02-16 13:01:02 +10:00
Ryan Brown 2678508d42
[cloud] Fix inconsistent case of `Allocation_id` in NAT Gateway outputs (#36058) 2018-02-15 10:51:16 -05:00
Stefan Horning 404f9260d9 [cloud][aws] Add metrics_collection options to ec2_asg module (#35180)
* Added metrics_collection management to ec2_asg module to switch ASG metrics on and off.

* Fixed typo in documentation of ec2_asg module

* Removed extra blank line in ec2_asg.py

* Docs fixes for ec2_asg module

* Added integration test for ec2_asg metrics flag

* Trying different syntax for ec2_asg test
2018-02-12 10:22:18 -05:00
Sloane Hertel fd33dc6cd1 [cloud] Make ec2_vpc_route_table wait for the route to propagate (#35975)
* Stabilize ec2_vpc_route_table

Wait for route table to be present before attempting to use it

Sleep before getting the final state of the route table in case modifications are incomplete

* Conditionally wait if changes were made

* Simplify logic
2018-02-09 16:32:23 -05:00
Ryan Brown f4d896c716 Tower callback fixes (#35927)
* Fix tower_callback script return code

* Add ec2_key parameter to integration tests
2018-02-09 14:28:52 -05:00
Ryan Brown 0036cff493
[cloud] Add key_name parameter to ec2_instance (#35901) 2018-02-08 09:27:02 -05:00
Ryan Brown e71c6d8e17 New module: ec2_instance (#35749)
New module: ec2_instance

Integration tests for ec2_instance
2018-02-08 01:50:41 -05:00
Sloane Hertel e10e1d6ddf [ecs_service_facts] replace map with dict for Python3 compatibility (#35355) 2018-02-07 21:05:51 -05:00
KrdLab 2a6b03459b Fix aws_s3_cors documentation (#35684) 2018-02-07 18:27:47 -05:00
subcon42 920eaafde2 [cloud] New Module ec2_vpc_vpn: Facilitate VPN TunnelOptions (#35210)
AWS (and Boto3) supports VPN TunnelOption keys to be specified for VPNs.  Specifically (at least at the moment), the options for 'TunnelInsideCidr' (str) and 'PreSharedKey' (str) are particularly noteworthy.  One or both may be specified within a 'dict' object (max 2 dicts).  AWS defaults shall apply in absence of said parameters.
2018-02-07 17:39:12 -05:00
Willian Ricardo ee64dbfec5 new module cloudfrontwatchlogs_log_group to create cloudwatch log group (#32272)
* new module to create cloudwatch log group

* Add new module cloudfrontwatchlogs_log_group to the changelog.
2018-02-07 15:52:28 -05:00
Sloane Hertel f20b9dab72 Add cloudwatchlogs_log_group_facts module (#35314)
* Add cloudwatchlogs_log_group_facts module

* pep8

* Add new module cloudwatchlogs_log_group_facts to CHANGELOG
2018-02-07 15:04:36 -05:00
Rob 9451212855 [cloud] New AWS module - elb_target (#26483)
* New module - elb_target

* Work to add deregister functionality

* Add jurajseffer work on waiting for status change

* List the set of statuses as a choices list

* default target_status_timeout is 60

* Add 'unavailable' as a target status choice per API docs

* Add support to remove targets that are in unused state

* add support for availability zone selection

Fix deregistering targets from used target groups

Fix waiting for unused state for targets that have started deregistering
2018-02-07 07:45:35 -05:00
Will Thames fcd09e2f02 New aws_waf_web_acl module (#33212)
* Add aws_waf_web_acl rule

* Add new aws_waf_web_acl to CHANGELOG
2018-02-07 07:06:15 -05:00
mikedlr d31ded47fb Aws ssm multiple fixes (#35569)
* aws ssm parameter lookup test case - fails demonstrating no exception when parameter missing

* aws ssm parameter lookup - fail in case parameter doesn't exist

* aws ssm parameter lookup test case - failing case for nice return from path lookup

* aws ssm parameter lookup - convert incoming taglist to a key-value dictionary

* aws ssm parameter lookup - pep8 / style clean up

* aws_ssm lookup plugin rewrite for more standard interface

* aws_ssm module and lookup - introduce integration test and fix:

* aws_ssm module and lookup - error case integraton test and many PEP8 and other cleanups

* aws ssm parameter lookup - Various fixes in response to review + recursive fix & test

* aws ssm parameter lookup - more in response to review - shertel/abadger

* aws ssm parameter lookup unit test - move to mocker according to abadger

* aws ssm parameter lookup - integrate with new documentation fragment

* aws ssm parameter lookup - accept either aws_profile or boto_profile

* aws ssm parameter lookup - eliminate lookup document fragment until env vars are fixed later
2018-02-06 17:41:46 -05:00
Will Thames 4e30eff651 [cloud][aws] New module: aws_waf_rule module (#33124)
Add a new module for managing AWS WAF rules

Preceded by aws_waf_condition and to be succeeded by aws_waf_web_acl
2018-02-02 17:54:27 -05:00
Sloane Hertel 7917e3c988
Fix syntax error so route table tests always try to clean up (#35663)
* Always try to clean up after route table tests

* Wait for a new VPC a chance to become available before attempting to get its current state
2018-02-02 15:38:47 -05:00
Willem van Ketwich 4faad64579 [cloud][aws] New module: cloudfront_invalidation (#35601)
* Remove redundant default for target_paths

* Make invalidation requests idempotent

* Remove redundant exception handling around boto3_conn

* Add cloudfront_invalidation to the changelog
2018-02-02 15:09:55 -05:00
Will Thames b5a1643e3d Add new aws_waf_condition module (#33110) 2018-02-01 18:16:27 -05:00
Sloane Hertel ee209e5f6f
Fix idempotence for deleting ElasticBeanstalk applications (#35614) 2018-02-01 14:08:59 -05:00
Willem van Ketwich e8d8c5b19c New module: manage Amazon CloudFront origin access identities (cloud/amazon/cloudfront_origin_access_identity) (#35540) 2018-02-01 11:32:33 -05:00
Sloane Hertel 556bb7ab00 [cloud] Log more exception info to fix sts_assume_role integration tests (#35570) 2018-01-31 13:18:09 -05:00
Ryan S. Brown 05f765bda0 New module: aws_batch_job_definition 2018-01-30 17:28:45 -05:00
Jonathan Meran 8578b3c1c5 [cloud] new module aws_batch_job_queue: support for AWS Batch Job Queues (#26701)
1. Removed redundant default values in argument_spec
2. Enclosed non standard python libraries in Try Except ImportError block
3. Added region as a mandatory element in argument_spec
4. Updated the typo in api_params while accessing job queue. Changed it from job_queue to jobQueue
5. Updated the set_api_params() method to use snake_dict_to_camel_dict()
6. Updated the ANSIBLE_METADATA version from 1.0 to 1.1
7. Updated the version_added in Documentation from 2.4 to 2.5
8. Initialized response value to None
9. Updated the RETUN documentation
10. Updated the return from manage_state() method to not embed batch_job_queue_action in ansible_facts
11. Updated the module.exit_json()  in main() to convert any keys back to snake_dict format using camel_dict_to_snake_dict to maintain consistency throughout the module
12. Changed the filename from batch_job_queue.py to aws_batch_job_queue.py
2018-01-30 17:22:06 -05:00
Sloane Hertel fda09a75d3 elasticache: default zone to None - fixes #31779 (#31785)
* elasticache: default zone to None.

* Only add PreferredAvailabilityZone to kwargs if self.zone is not None
2018-01-30 16:12:22 -05:00
Sloane Hertel 2c4f52d404 [cloud] iam_role: allow retention of existing policies (#28513)
* module_iam_role: add_remove_policies_option

* Update 24080 with requested changes.

* Fix whitespace and rename the option remove_unlisted_policies to purge_policies.

Update version_added

* Fix exception handling in iam_role module

* Fix line length
2018-01-30 15:13:45 -05:00
Jonathan Meran 9e68ac9d86 Initial support for AWS Batch Compute Environment (#26699)
* Add AWS Batch Compute Environment module

* Move common code for AWS batch modules in module_utils/aws/batch
2018-01-30 13:24:38 -05:00
John R Barker a23c95023b
Module deprecation: docs, scheme and tests (#34100)
Enforce module deprecation.
After module has reached the end of it's deprecation cycle we will replace it with a docs stub.

* Replace deprecated modules with docs-only sub
* Use of deprecated past deprecation cycle gives meaningful message (see examples below)
* Enforce documentation.deprecation dict via `schema.py`
* Update `ansible-doc` and web docs to display documentation.deprecation
* Document that structure in `dev_guide`
* Ensure that all modules starting with `_` have a `deprecation:` block
* Ensure `deprecation:` block is only used on modules that start with `_`
* `removed_in` A string which represents when this module needs **deleting**
* CHANGELOG.md and porting_guide_2.5.rst list removed modules as well as alternatives
* CHANGELOG.md links to porting guide index

To ensure that meaningful messages are given to the user if they try to use a module at the end of it's deprecation cycle we enforce the module to contain:
```python
if __name__ == '__main__':
    removed_module()
```
2018-01-30 12:23:52 +00:00
Stephen Granger 7fa09390b0 Elasticbeanstalk app (#31079)
* New AWS elasticbeanstalk app module

* Integration tests for elasticbeanstalk_app
2018-01-29 18:38:32 -05:00
Sloane Hertel 6dda001a8c [cloud] new module - AWS Direct Connect virtual interface (#33211)
* Add an AWS Direct Connect virtual interface module

Fix DirectConnectError exception wrapper to be compatible with fail_json_aws()

* Fix documentation and handle BotoCoreErrors as well

* Add the aws and ec2 documentation fragments
2018-01-29 17:23:03 -05:00
Sloane Hertel fed4217fd7 [cloud] ec2_eni: Add a purge option for secondary ip addresses - fixes #26575 (#26596)
* add purge option for secondary ip addresses and fix changed to reflect when addresses are modified

* Improve option documentation and make the secondary_private_ip_addresses option required if purging secondary private ip addresses.

* ec2_eni: update version_added to 2.5 for purge_secondary_private_ip_addresses option.
2018-01-29 16:31:21 -05:00
Ed Costello d16bc1c3f4 New aws_ses_identity module to manage AWS Simple Email Service Identity (#31140)
* Add aws_ses_identity module

* Update CI alias, add BotoCoreError exception handling.

* Add SES and SNS permissions to hacking/aws_config to run aws_ses_identity integration tests
2018-01-29 15:35:49 -05:00