* Add functions to retrieve the allowed and required parameters for boto3 client methods
* Add custom waiter for stopping an RDS DB instance
* Add rds_instance module
* Add rds_instance integration tests
* address requested changes from ryansb
* address requested changes from willthames
* address requested changes from dmsimard
* Fix final snapshots
Fix idempotence with already-deleting DB instances
Remove unused import from module_utils/aws/core.py
Consolidate function to get all boto3 client method parameters and the subset of required parameters
* Add some additional rds_instance integration tests
* Add some common functions to module_utils/aws/rds
* Move common code out of rds_instance
* Remove hardcoded engine choices and require the minimum boto3
* Document wait behavior
* Provide a list of valid engines in the error message if it is invalid
Add supported methods to whitelist
Remove AWSRetry around waiter
Wait for a less crazy amount of time
Remove unused variables
* Add a test for an invalid engine option
* pep8
* Missed adding a method to the whitelist
* Use retries
* Fix some little things
* Fix more things
* Improve error message
* Support creating cross-region read replicas
* Remove unused imports
* Add retry when getting RDS instance
* Soft-check required options so module fails properly when options are missing
* Fix mariadb parameter version
* Fix cross-region read_replica creation and tests
* fix modify tests
* Fix a modification test
* Fix typo
* Remove test for option_group_name that exists for this account but may not for others and added as a TODO to do properly
* initial source of the elb_target_facts module, which gathers all
registered ELBv2 target groups for a given instance id
* updating
* initial dump of tests
* updating docs, adding AWSRetry decorators
* updating tests
* More brush up, some refactoring
* updating for sanity tests
* removing extra space
* updating elb_target_facts to use a new name for the return value, not return a fact, and use instance variables for the AWS connections. updating tests to go along with that
* updating classes to be 'new-style' classes
The AWS API and console docs are inconsistent about whether EC2 instances have IAM profiles or roles. Things which follow the API tend to use profile but the console uses “IAM Role”. This adds that term to the help text so it's searchable.
+label: docsite_pr
This PR includes:
- Removal of maintainers that are listed as author in the module
- Removal of entries that do not extend the original author list
- Move ignored-statement to namespace/directory (where useful)
- In some cases, fix the authors-list or add missing github id
We end up with a list of exceptions/additions and a large set of
namespace/directory maintainers or team of maintainers.
Some entries could be further improved by discussing with some
maintainers.
* Removed modules no longer have documentation
Decided this was causing people to think that modules were supported
even after being removed. This change is a new strategy to have the
error message trying to use a removed module point people to the older
documentation.
* Add stubs for modules removed in 2.7
These are freshly removed so we want people who are still using them
when they upgrade Ansible to have a hint as to where to find information
on how to port.
* Finish properly undeprecating include
include was undeprecated earlier but not all of the pieces that marked
it as deprecated were reverted. This change fixes the remaining
pieces
* elb_application_lb: fix dangerous default of deleting an ELB if state is omitted by changing state to default to present to be more like other AWS modules
* Add (preview) diff mode support ec2_group
* Add diff mode to some ec2_group integration tests
* Remove unnecessary arguments and add comment to the module notes
* Add changelog
* Correct the default doc for attached in ec2_eni
Also corrected a typo in the summary
* Address ansible-test sanity error about E324
* Fix and remove the E325 suppression for ec2_eni
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
* added account_alias in the response of module aws_caller_facts
* added comment to explain list_account_aliases
* renamed caller_identity to caller_facts as the content is extended
* created changelog
* security-policy needs the iam:ListAccountAliases for this module to work
* test now checks for the added field account_alias
* gracefully handle missing iam:ListAccountAliases permission
Wow, this does not seem to be an uncommon misspelling. Might be there
are some left that span over two lines. I noticed the one in the git
module and then used `grep -rw 'the the'` to find some more.
* remove leading dot in efs_facts mountpoint
* remove leading dot in efs_facts mountpoint
* remove leading dot in efs.py
* introduce filesystem_address var in efs* modules and revert changes to mount_point
* Use https:// for docs.aws.amazon.com in aws_s3 module
* Support Amazon S3 Dual-Stack Endpoints in aws_module
The default S3 endpoint only resolves to IPv4 addresses. By enabling
the dual-stack endpoint IPv6 too get added as an option.
* Convert to AnsibleAWSModule and support IAM permission boundaries
* Handle adding boundary to existing role that lacks one
* Properly clean up role boundary associations on delete
* Handle case when policy boundary is `""` but does not exist
* Add parameter to keep elb rules
Does not purge elb rules. This is usefull if running the elb_application_lb
role and there is the desire to keep existing rules.
* Change variable name keep_rules to purge_rules
The descriptor purge has been used in the past.
* Changed default for purge_rules
Default is purge_rules. This is how the module has functioned previously. This change maintains
the previous behavior.
* Add integration test for purge_rules flag
* Change wording of test task
* Fix merge conflcit
* Changed default for purge_rules
Default is purge_rules. This is how the module has functioned previously. This change maintains
the previous behavior.
* merge conflcit
* Change wording of test task
* Add purge_rules option to test
* Change test description wording
* Expand purge_rules documentation
* Clarifies documentation for purge_rules option
* ec2.py:
* source_dest_check default value is now None, updated docs
* Refactor restart_instances and startstop_instances -> Two new functions to prevent repetition: check_source_dest_attr and check_termination_protection
Ensure that FieldLevelEncryptionId is properly handled - passing it if
set, and keeping it if returned by GetDistribution
Update cloudfront_distribution tests to remove references to
test_identifier so test suite actually works
Fixes#40724
* Delete all versions and deletemarkers in S3_Bucket when force parameter is passed
* Fix PEP8 style conformance
* Clarify explanation of force parameter
* __file__ won't work if we want to invoke modules via -m or if we
figure out how to keep modules from hitting the disk with pipelining.
* module.tmpdir is the new way to place a file where it will be cleaned
automatically.
Change format string to not depend on __file__:
* cloud/amazon/ec2_elb_lb.py
* cloud/amazon/elb_classic_lb.py
Use module.tempdir:
* packaging/os/apt.py
* files/unarchive.py
* aws_eks_cluster: Improve output documentation
This data is already returned by the module, it just wasn't documented. These
fields are required for accessing the created Kubernetes API with e.g. the
k8s_raw module.
* aws_eks_cluster: Add wait functionality
This enables further cluster configuration once it's created and active.
20 minutes was chosen as an arbitrary default, so that if it takes longer than
the documented "usually less than 10 minutes" it's still likely to succeed.
* Correct security group name in aws_eks tests
* Improve teardown of aws_eks tests
Fix minor teardown issues. The `pause` step is a placeholder until
a waiter for `state: absent`
* Add execution_role_arn parameter
* Change ecs_taskdefinition to use AnsibleAWSmodule
Botocore version checking is becomming more common. Changing the ecs_taskdefinition
to use AnsibleAWSmodule allows more easily for this.
* Change launch type check to use botocore_at_least function
* Remove execution_role_arn param from params dict
* Change check to use parameter
* Fix typo
* Add test for old botocore version
* Add test for execution role parameter
* Remove iam_role_facts task
Task was unecessary. The same information could be gathered by registering
the iam_role task.
* add check description update
* Ensure 'Description' is in the role so KeyError isn't caused
* Fix changed when modifying the description with check mode
[AWS iam_policy] Avoid the _undefined name_ by catching the raised exception into the variable __e__ so it can be reported on the following line.
flake8 testing of https://github.com/ansible/ansible on Python 3.6.3
$ __flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics__
```
./lib/ansible/modules/cloud/amazon/iam_policy.py:305:16: F821 undefined name 'e'
if e.errno == 2:
^
./lib/ansible/modules/cloud/misc/rhevm.py:594:24: F821 undefined name 'e'
setMsg(str(e))
^
./lib/ansible/modules/files/archive.py:391:92: F821 undefined name 'e'
module.fail_json(dest=dest, msg='Error deleting some source files: ' + str(e), files=errors)
^
3 F821 undefined name 'e'
3
```
* Add a module parameter to configure the max fetched AWS CFN stack events
* Add version documentation for new configuration option
* Increase default in order to make sure that enough are fetched by default. This align roughly with the limit of manageable resources in CloudFormation.
Read the Docs moved hosting to readthedocs.io instead of
readthedocs.org. Fix all links in the project.
For additional details, see:
https://blog.readthedocs.com/securing-subdomains/
> Starting today, Read the Docs will start hosting projects from
> subdomains on the domain readthedocs.io, instead of on
> readthedocs.org. This change addresses some security concerns around
> site cookies while hosting user generated data on the same domain as
> our dashboard.
* Add aws/core.py function to check for specific AWS error codes
* Use sys.exc_info to get exception object if it isn't passed in
* Allow catching exceptions with is_boto3_error_code
* Replace from_code with is_boto3_error_code
* Return a type that will never be raised to support stricter type comparisons in Python 3+
* Use is_boto3_error_code in aws_eks_cluster
* Add duplicate-except to ignores when using is_boto3_error_code
* Add is_boto3_error_code to module development guideline docs
* Test case for missing permissions
* Update aws_s3 module to latest standards
* Use AnsibleAWSModule
* Handle BotoCoreErrors properly
* Test for BotoCoreErrors
* Check for XNotImplemented exceptions (#38569)
* Don't prematurely fail if user does not have s3:GetObject permission
* Allow S3 drop-ins to ignore put_object_acl and put_bucket_acl
* aws_eks: New module for managing AWS EKS
aws_eks module is used for creating and removing EKS clusters.
Includes full test suite and updates to IAM policies to enable it.
* Clean up all security groups
* appease shippable
* Rename aws_eks module to aws_eks_cluster
* Allow the use of 'aws:kms' as an encryption method
* Allow the use of a non standard KMS key
* Deduce whether AWS Signature Version 4 is required rather than specifying with a parameter
The compute policy was exceeding maximum size and contained
policies that already exist in ecs-policy.
Look up suitable AMIs rather than hardcode
We don't want to maintain multiple image IDs for multiple regions
so use ec2_ami_facts to set a suitable image ID
Improve exception handling
Fargate instances do not require memory and cpu descriptors. EC2 instances
do require descriptions. https://botocore.readthedocs.io/en/latest/reference/services/ecs.html#ECS.Client.describe_task_definition
Fargate requires that cpu and memory be defined at task definition level.
EC2 launch requires them to be defined at the container level.
Fargate requires the use of awsvpc for the networking_mode. Also updated,
the documentation regarding where and when memory/cpu needs to the assigned.
The task_definition variable for the awspvc configuration colided with
the ecs_service for the bridge network. This would cause the test to fail.
Add testing for fargate
Add examples for fargate and ec2
* fix ec2_ami block_device_mapping size to be int
* fixed cr issues
renamed `type` to `attribute_type`
reused `new_item` instead of creating new variable `value`
* Port aws_ses_identity module to use AnsibleAWSModule
* Support Check Mode in aws_ses_identity
* Add tests for check mode
* Move feedback forwarding parameter check to before any changes are made.
* Adding module for AWS Config service
* adding integration tests
* Split resource types into their own modules
* Properly use resource_prefix and retry on IAM "eventual consistency"
* Add config aggregator module
* AWS config aggregator integration test fixes
* AWS config recorder module
* Config aggregation auth rule
* Use resource_prefix in IAM role name
* Disable config tests
* Refactor ec2_group
Replace nested for loops with list comprehensions
Purge rules before adding new ones in case sg has maximum permitted rules
* Add check mode tests for ec2_group
* add tests
* Remove dead code
* Fix integration test assertions for old boto versions
* Add waiter for security group that is autocreated
* Add support for in-account group rules
* Add common util to get AWS account ID
Fixes#31383
* Fix protocol number and add separate tests for egress rule handling
* Return egress rule treatment to be backwards compatible
* Remove functions that were obsoleted by `Rule` namedtuple
* IP tests
* Move description updates to a function
* Fix string formatting missing index
* Add tests for auto-creation of the same group in quick succession
* Resolve use of brand-new group in a rule without a description
* Clean up duplicated get-security-group function
* Add reverse cleanup in case of dependency issues
* Add crossaccount ELB group support
* Deal with non-STS calls to account API
* Add filtering of owner IDs that match the current account
* New module = AWS Glue connection
* Add a few initial integration tests
* Add alias for CI
* module rename
* finish module rename
* add loop when getting glue connection again so we dont get None
* Limit number of retries to get new glue connection info
* - add tests for s3_lifecycle
- fix a bug comparing transitions with different storage_types
* make s3_lifecycle work with boto3
* add noncurrent version lifecycle rules
* Fix failing aws_ses_identity integration tests
Reduce boilerplate with yaml anchor
* remove unstable test alias
* Update feedback forwarding check to use desired state rather than
repeated API calls.
RDS instance and snapshot facts modules
Move common code into module_utils/rds.py
This work is a precursor to new rds_snapshot
and rds_instance modules
Improve rds_snapshot_facts to latest AWS standards
Improve exception handling, pagination etc.
Version added
Add clustered snapshot facts
Add proper RETURN information for snapshots and cluster_snapshots
pep8
Co-authored-by: Michael De La Rue <jababubic.margoteen@spamgourmet.com>
Create rds_instance_facts module using boto3.
Create common rds.py module_util library to be used by
rds_instance_facts and future RDS modules.
rds_instance_facts: use latest AWS helper methods
* Use `module.client` with `retry` to create AWS connection
* Return tags, using AWSRetry helper to backoff
* Use paginator to describe DB instances
* Further improve exception handling
* Fix RETURN documentation
update version_added
Co-authored-by: Michael De La Rue <jababubic.margoteen@spamgourmet.com>
* New module: ec2_vpc_vpn_facts
* Add integration tests for ec2_vpc_vpn_facts and the IAM permissions
* Add retry to VPC removal
* Use unique name for VGW
* Always clean up after tests and add retries
* Allow idempotent use of ec2_ami_copy
When `tag_equality` is set true, use tags to determine
if AMIs in different accounts are the same, and don't
copy the AMI twice if they are the same.
Use AnsibleAWSModule and make imports more consistent
with other modules
* Update version added
* More code review changes
* Review changes - Recommended way to start EC2 connection
Based on the documentation, 'wait_timeout' is 'Used in conjunction with instance_ids option'. This lead me to believe that I could not use this parameter to try and solve the 'Waited too long for ELB instances to be healthy' error I was experiencing.
* ec2_vpc_route_table: Update matching_count parsing on find_subnets function and tests
* ec2_vpc_route_table: Update matching_count parsing on find_subnets function
* Stabilize ec2_vpc_vgw and ec2_vpc_vpn so tests for ec2_vpc_vpn_facts in PR 35983 can be run in CI
* Add updated placebo recordings
* ensure find_vgw uses the virtual gateway id if available
Add AWSRetry.jittered_backoff to attach_vpn_gateway to deal with errors when attaching a new VPC directly after detaching
Add integrations tests for ec2_vpc_vgw
* Sort VPN Gateways by ID
* Add helpful failure message if target_type=ip is not supported
Create test case for target_type=ip not supported
* Update elb_target_group module to latest standards
Use AnsibleAWSModule
Improve exception handling
Improve connection handling
* Add a module to get storage gateway facts
* Review fixes
* Last review fixes
* Add filtering gathering & some fixes
* doc fix
* API error handling
* Remove ec2_argument_spec import
Use imported BotoCoreError and ClientError rather than botocore.exceptions
Updated documentation to refer to 'network_interface_id' rather than 'interface_id' as the latter results in an error: 'Parameter validation failed: Unknown parameter in input: \"InterfaceId\"'
* Add option to specify reusable delegation set while creating public zones
* Add mutual exclusion argument spec for delegation set and VPC/private zone
* Get zone delegation set ID when updating a public zone
Enable awsvpc network mode for ECS services and tasks and
their underlying task definitions
Improve test suite to thoroughly test the changes
Use runme.sh technique to run old and new versions of botocore to
ensure that the modules work with older botocore and older network modes
and fail gracefully if awsvpc network mode is used with older botocore
* Improve ec2_ami tests
Ensure that ec2_ami_image_id fact gets set immediately after AMI
creation so that they get torn down even if tests fail
Use YAML anchor to simplify AWS credential passing
Use aws_connection_info to reduce AWS credential boilerplate
Improve exception handling when updating image attributes
Error messages weren't correctly formatted to show image ids.
The docs suggest that `port` parameter is the default port upon
which targets listen. As such, a target need only provide a `Port`
key to override the default.
* Fix the function_name handling logic for lambda_policy
Switch the logic handling function_names that are ARNs
so that ARNs are correctly handled and detected
* Add tests for lambda_policy function_arn
Ensure that function_arn works.
Needs a reasonable ansible_lambda_role.
* Reflect updated names of the aws_batch_job modules
* Remove mention of `lambda_invoke` module which has yet to make it to
Ansible.
* Update broken rst link in win_dsc module
* Add aws_ses_identity_policy module for managing SES sending policies
* Add option to AnsibleAWSModule for applying a retry decorator to all calls.
* Add per-callsite opt in to retry behaviours in AnsibleAWSModule
* Update aws_ses_identity_policy module to opt in to retries at all callsites.
* Add test for aws_ses_identity_policy module with inline policy.
* Remove implicit retrys on boto resources since they're not working yet.
Consolidate waiters to a single file
* Add waiter message with token ID
* Add waiter
Add waiter for WAF change tokens
Working waiter for waf_condition module
Add support for waiters to waf_rule
* WAF data model refactor
* Fix ref to self.client
* Add custom waiters to aws_waf_web_acl
* Allow add/remove rule tasks to operate in parallel, then wait for their change tokens to complete
* Move waiter into run_func_with_change_token_backoff since it is generic to all WAF update operations
* Wait for deletes on waf_web_acl
* Remove always-wait
* Remove waiter retry catch
* stabilize ec2_vpc_subnet module
* Add waiters for ec2_vpc_subnet
Clean up integration tests
* Reenable CI for stabilized ec2_vpc_subnet tests
* rename waiters
* Use module_json_aws where applicable
Handle WaiterError first if waiting failed
* Fix traceback when tagging with keys/values that look like booleans
* Fix check mode with tags
* Add integration tests for tags that look like booleans and check mode
* Add waiter for deleting subnet
* Sleep a few seconds after using aws command line
* Use AnsibleAWSModule to simplify AWS connection
* Add Exception handling, pagination, retries and backoff
* Allow events to be switched off
* Allow details to be obtained without having to specify services
SGs created when a VPC ID was not specified would not necessarily
get the default egress rule, even when no explicit egress rules
were set.
Add some checks for egress rules in results from existing tests
Change assign_public_ip to default to None rather than False so
that we can detect whether the value is being explicitly set or
not, and only warn if it is explicitly set to False for something
with a public_dns_name
Fixes#37985
* Make the module idempotent
* pep8 fixes.
* Made required changes for overwrite_value for idempotency.
* Fix missing error definition
* Add in missing documentation variable.
* Added missing scalable target creation
* Changed if statement
* Added support to results of all actions
* Fixed line lengths, whitespaces and blank lines between functions
* Fixed documentation formatting
* Work in progress, fixed returns from functions, still need to do exception handling
* Work in progress, still need to do exception handling
* Moved to AnsibleAWSModule, Added exception handling
* Added detailed return doc
* Fixed return doc alarms
* fixed return yaml
* Fixed function calls when creating/deleting
* fixed unnecessary blank line
* removed imports and unnecessary checks handled by AnsibleAWSModule
* removed whitespace
* [cloud] ec2_vpc_route_table: ignore routes without DestinationCidrBlock
Add module warnings rather than silently skipping
* Permit warnings for routes tables containing vpc endpoints to be turned off
* Add tests to ensure a VPC endpoint associated with a route table does not result in a traceback
The user_data field is base64 encoded inside of the boto library. In
Python3, base64 must be used with byte strings. So we make sure to
encode the user_data into a byte string before passing it on to the boto
library.
Fixes#34978
Sloane Hertel
Adam
yaakov kuperman
Constantin Bugneac
Chris Adams
Dag Wieers
Toshio Kuratomi
Ryan S. Brown
Will Thames
GElkayam
Uwila
/v\atthew L Daniel
Matt Martz
sdubrul
♫ Christian Krause ♫
Wes Mosher
Pedro Magalhães
Ben Irving
Rob St. Marie
Julien PRIGENT
Juan Jimenez
curry9999
Willian Ricardo
Rushan
Joren Vrancken
Andreas Olsson
flowerysong
mjmayer
Roberto Barreda
Vladimir Pavljuchenkov
Guy Templeton
Jared Hocutt
Gennady Aleksandrov
Calvin Wu
Grzegorz Śliwiński
Julien Vey
Deiwin Sarjas
cahlchang
Rob
cclauss
Leif Madsen
David Medberry
Dennis Benkert
cpollard0
Deepakkothandan
Kristian Østergaard Martensen
Jon Dufresne
Christian Groschupp
Brandon W Maister
Abhishek Garg
René Moser
Michael Mayer
Jon Steinich
Tine Jozelj
Ed Costello
ossark
Moritz Grimm
saydamir
Matt Clay
Ben Berry
Madhura-CSI
Ramki Subramanian
znantho
wang shuang
Felix Fontein
Vinay Dandekar
Loïc Blot
vab2048
Michele Zuccala
Dennis Conrad
KrdLab
Alicia Cozine
John R Barker
Benjamin Ness
Brad Macpherson
Sébastien Portebois
Maish
Sean Lok
nathanwebsterdotme
chenl87
Alessandro Arnone