* Improve ec2_ami tests
Ensure that ec2_ami_image_id fact gets set immediately after AMI
creation so that they get torn down even if tests fail
Use YAML anchor to simplify AWS credential passing
Use aws_connection_info to reduce AWS credential boilerplate
Improve exception handling when updating image attributes
Error messages weren't correctly formatted to show image ids.
The docs suggest that `port` parameter is the default port upon
which targets listen. As such, a target need only provide a `Port`
key to override the default.
* Fix the function_name handling logic for lambda_policy
Switch the logic handling function_names that are ARNs
so that ARNs are correctly handled and detected
* Add tests for lambda_policy function_arn
Ensure that function_arn works.
Needs a reasonable ansible_lambda_role.
* Reflect updated names of the aws_batch_job modules
* Remove mention of `lambda_invoke` module which has yet to make it to
Ansible.
* Update broken rst link in win_dsc module
* Add aws_ses_identity_policy module for managing SES sending policies
* Add option to AnsibleAWSModule for applying a retry decorator to all calls.
* Add per-callsite opt in to retry behaviours in AnsibleAWSModule
* Update aws_ses_identity_policy module to opt in to retries at all callsites.
* Add test for aws_ses_identity_policy module with inline policy.
* Remove implicit retrys on boto resources since they're not working yet.
Consolidate waiters to a single file
* Add waiter message with token ID
* Add waiter
Add waiter for WAF change tokens
Working waiter for waf_condition module
Add support for waiters to waf_rule
* WAF data model refactor
* Fix ref to self.client
* Add custom waiters to aws_waf_web_acl
* Allow add/remove rule tasks to operate in parallel, then wait for their change tokens to complete
* Move waiter into run_func_with_change_token_backoff since it is generic to all WAF update operations
* Wait for deletes on waf_web_acl
* Remove always-wait
* Remove waiter retry catch
* stabilize ec2_vpc_subnet module
* Add waiters for ec2_vpc_subnet
Clean up integration tests
* Reenable CI for stabilized ec2_vpc_subnet tests
* rename waiters
* Use module_json_aws where applicable
Handle WaiterError first if waiting failed
* Fix traceback when tagging with keys/values that look like booleans
* Fix check mode with tags
* Add integration tests for tags that look like booleans and check mode
* Add waiter for deleting subnet
* Sleep a few seconds after using aws command line
* Use AnsibleAWSModule to simplify AWS connection
* Add Exception handling, pagination, retries and backoff
* Allow events to be switched off
* Allow details to be obtained without having to specify services
SGs created when a VPC ID was not specified would not necessarily
get the default egress rule, even when no explicit egress rules
were set.
Add some checks for egress rules in results from existing tests
Change assign_public_ip to default to None rather than False so
that we can detect whether the value is being explicitly set or
not, and only warn if it is explicitly set to False for something
with a public_dns_name
Fixes#37985
* Make the module idempotent
* pep8 fixes.
* Made required changes for overwrite_value for idempotency.
* Fix missing error definition
* Add in missing documentation variable.
* Added missing scalable target creation
* Changed if statement
* Added support to results of all actions
* Fixed line lengths, whitespaces and blank lines between functions
* Fixed documentation formatting
* Work in progress, fixed returns from functions, still need to do exception handling
* Work in progress, still need to do exception handling
* Moved to AnsibleAWSModule, Added exception handling
* Added detailed return doc
* Fixed return doc alarms
* fixed return yaml
* Fixed function calls when creating/deleting
* fixed unnecessary blank line
* removed imports and unnecessary checks handled by AnsibleAWSModule
* removed whitespace
* [cloud] ec2_vpc_route_table: ignore routes without DestinationCidrBlock
Add module warnings rather than silently skipping
* Permit warnings for routes tables containing vpc endpoints to be turned off
* Add tests to ensure a VPC endpoint associated with a route table does not result in a traceback
The user_data field is base64 encoded inside of the boto library. In
Python3, base64 must be used with byte strings. So we make sure to
encode the user_data into a byte string before passing it on to the boto
library.
Fixes#34978
* Increase delay and tries for ec2_vpc_net backoff
Wait for DHCP option to be created in ec2_vpc_dhcp_option
Wait for all modifications to the VPC
* Use the vpc_available waiter because is uses Filters
* Missed one
* Optimize retries to only occur if the functionality is available
* Increase max wait time
* Add comments to explain what the waiters are doing
This creates a way for us to use boto3's data-driven waiter support to use custom waiters where Boto3 hasn't implemented them yet.
The only waiter implemented so far is for VPC Route Tables to check that they exist, and this replaces some custom retry code.
Fixes#35993 - Changes to update_size in commit eb4cc31 made it so
the group dict passed into update_size was not modified. As a result,
the 'replace' call does not see an updated min_size like it previously
did and doesn't pause to wait for any new instances to spin up. Instead,
it moves straight into terminating old instances. Fix is to add batch_size
to min_size when calling wait_for_new_inst.
Fixes#28087 - Make replace_all_instances and replace_instances behave
exactly the same by setting replace_instances = current list of instances
when replace_all_instances used. Root cause of issue was that without lc_check
terminate_batch will terminate all instances passed to it and after updating
the asg size we were querying the asg again for the list of instances - so terminate batch
saw the list including new ones just spun up.
When creating new asg with replace_all_instances: yes and lc_check: false
the instances that are initially created are then subsequently replaced.
This change makes it so replace only occurs if the asg already existed.
Add integration tests for #28087 and #35993.
[cloud aws_s3] Most modes require `object` parameter, and this is easy to
get wrong (e.g. through confusion with the `dest` parameter). As
it's as easy to enforce, let's do that.
There should no longer be a need for boto documentation - it
just adds significant extra clutter to the guidelines.
AnsibleAWSModule greatly improves the codebase to such an extent
that we should mandate it for new code unless there is a
documented good reason.
* Update doc of AnsibleAWSModule to remove incorrect connect example.
Current example uses aws_connect method which no longer exists. Replace
this with the client and resource methods that do exist.
Also remove try/except block in connect example as guidelines imply that
there aren't connection errors thrown on connection, just when later
sing the connection.
* Update AWS module guidelines to include the AnsibleAWSModule connection methods.
* Add information on integration testing to the AWS module guidelines.
* Add information on common documentation fragments to AWS module
Guidelines.
* Changes as requested on PR.
* Restructure connection section to start with current best practice
* Explain the use of the CI build groups
* Use YAML Anchors for aws credentials example
* Replace guidance on use of test groups with link to aliases file.
This achieves the goal of explaining why this file is necessary without
introducing overhead of keeping documentation up to date when test
groups change.
* Better handling of absent AWS SES identity notification information.
Fixes#36065
aws_ses_identity module now handles the cases where information about
the notification setup for the identity isn't returned by the AWS api.
This seems to happen in an edge case, believed to be eventual
consistency on registering new identities. So this case is treated
as if has been no notification setup for the identity yet.
Also fix 2 flake8 warnings in the module, a missing newline and unused
import.
* Increase the Boto Retries on SES APIs to deal with throttling.
This should address the unstable integration test failing due to
parallel runs in shippable hitting AWS throttling.
* Add retries loading SES details for inclusion in successful response.
There seems to be an eventual consistency behaviour with identity
registration. It's possible to still get no identity back after
registration.
This can cause failures in the shippable builds. This should fix that by
creating a retry of retrieving the identity information after
registration.
A similar retry loop has been added to notification attributes to ensure
this doesn't suffer from the same failure.
* Add missing sleep in get_notification_attributes to avoid busy loop.
* Add AWSRetry when describing VPCs to help stabilize integration tests
* Add retry on create_tags because it is possible to reach this API call before the VPC is finished creating
* Added metrics_collection management to ec2_asg module to switch ASG metrics on and off.
* Fixed typo in documentation of ec2_asg module
* Removed extra blank line in ec2_asg.py
* Docs fixes for ec2_asg module
* Added integration test for ec2_asg metrics flag
* Trying different syntax for ec2_asg test
* Stabilize ec2_vpc_route_table
Wait for route table to be present before attempting to use it
Sleep before getting the final state of the route table in case modifications are incomplete
* Conditionally wait if changes were made
* Simplify logic
AWS (and Boto3) supports VPN TunnelOption keys to be specified for VPNs. Specifically (at least at the moment), the options for 'TunnelInsideCidr' (str) and 'PreSharedKey' (str) are particularly noteworthy. One or both may be specified within a 'dict' object (max 2 dicts). AWS defaults shall apply in absence of said parameters.
* New module - elb_target
* Work to add deregister functionality
* Add jurajseffer work on waiting for status change
* List the set of statuses as a choices list
* default target_status_timeout is 60
* Add 'unavailable' as a target status choice per API docs
* Add support to remove targets that are in unused state
* add support for availability zone selection
Fix deregistering targets from used target groups
Fix waiting for unused state for targets that have started deregistering
* aws ssm parameter lookup test case - fails demonstrating no exception when parameter missing
* aws ssm parameter lookup - fail in case parameter doesn't exist
* aws ssm parameter lookup test case - failing case for nice return from path lookup
* aws ssm parameter lookup - convert incoming taglist to a key-value dictionary
* aws ssm parameter lookup - pep8 / style clean up
* aws_ssm lookup plugin rewrite for more standard interface
* aws_ssm module and lookup - introduce integration test and fix:
* aws_ssm module and lookup - error case integraton test and many PEP8 and other cleanups
* aws ssm parameter lookup - Various fixes in response to review + recursive fix & test
* aws ssm parameter lookup - more in response to review - shertel/abadger
* aws ssm parameter lookup unit test - move to mocker according to abadger
* aws ssm parameter lookup - integrate with new documentation fragment
* aws ssm parameter lookup - accept either aws_profile or boto_profile
* aws ssm parameter lookup - eliminate lookup document fragment until env vars are fixed later
* Remove redundant default for target_paths
* Make invalidation requests idempotent
* Remove redundant exception handling around boto3_conn
* Add cloudfront_invalidation to the changelog
Ryan S. Brown
Alicia Cozine
John R Barker
Matt Clay
Will Thames
Benjamin Ness
Sloane Hertel
Brad Macpherson
Sébastien Portebois
Vinay Dandekar
Maish
Julien Vey
Julien PRIGENT
Ed Costello
Gennady Aleksandrov
Sean Lok
nathanwebsterdotme
chenl87
Toshio Kuratomi
Alessandro Arnone
Dag Wieers
jjshoe
jeffreykoetsier
TJ Tang
msven
Kevin Breit
mikedlr
Stefan Horning
KrdLab
subcon42
Willian Ricardo
Rob
Willem van Ketwich