* Update doc of AnsibleAWSModule to remove incorrect connect example.
Current example uses aws_connect method which no longer exists. Replace
this with the client and resource methods that do exist.
Also remove try/except block in connect example as guidelines imply that
there aren't connection errors thrown on connection, just when later
sing the connection.
* Update AWS module guidelines to include the AnsibleAWSModule connection methods.
* Add information on integration testing to the AWS module guidelines.
* Add information on common documentation fragments to AWS module
Guidelines.
* Changes as requested on PR.
* Restructure connection section to start with current best practice
* Explain the use of the CI build groups
* Use YAML Anchors for aws credentials example
* Replace guidance on use of test groups with link to aliases file.
This achieves the goal of explaining why this file is necessary without
introducing overhead of keeping documentation up to date when test
groups change.
* Better handling of absent AWS SES identity notification information.
Fixes#36065
aws_ses_identity module now handles the cases where information about
the notification setup for the identity isn't returned by the AWS api.
This seems to happen in an edge case, believed to be eventual
consistency on registering new identities. So this case is treated
as if has been no notification setup for the identity yet.
Also fix 2 flake8 warnings in the module, a missing newline and unused
import.
* Increase the Boto Retries on SES APIs to deal with throttling.
This should address the unstable integration test failing due to
parallel runs in shippable hitting AWS throttling.
* Add retries loading SES details for inclusion in successful response.
There seems to be an eventual consistency behaviour with identity
registration. It's possible to still get no identity back after
registration.
This can cause failures in the shippable builds. This should fix that by
creating a retry of retrieving the identity information after
registration.
A similar retry loop has been added to notification attributes to ensure
this doesn't suffer from the same failure.
* Add missing sleep in get_notification_attributes to avoid busy loop.
* Add AWSRetry when describing VPCs to help stabilize integration tests
* Add retry on create_tags because it is possible to reach this API call before the VPC is finished creating
* Added metrics_collection management to ec2_asg module to switch ASG metrics on and off.
* Fixed typo in documentation of ec2_asg module
* Removed extra blank line in ec2_asg.py
* Docs fixes for ec2_asg module
* Added integration test for ec2_asg metrics flag
* Trying different syntax for ec2_asg test
* Stabilize ec2_vpc_route_table
Wait for route table to be present before attempting to use it
Sleep before getting the final state of the route table in case modifications are incomplete
* Conditionally wait if changes were made
* Simplify logic
AWS (and Boto3) supports VPN TunnelOption keys to be specified for VPNs. Specifically (at least at the moment), the options for 'TunnelInsideCidr' (str) and 'PreSharedKey' (str) are particularly noteworthy. One or both may be specified within a 'dict' object (max 2 dicts). AWS defaults shall apply in absence of said parameters.
* New module - elb_target
* Work to add deregister functionality
* Add jurajseffer work on waiting for status change
* List the set of statuses as a choices list
* default target_status_timeout is 60
* Add 'unavailable' as a target status choice per API docs
* Add support to remove targets that are in unused state
* add support for availability zone selection
Fix deregistering targets from used target groups
Fix waiting for unused state for targets that have started deregistering
* aws ssm parameter lookup test case - fails demonstrating no exception when parameter missing
* aws ssm parameter lookup - fail in case parameter doesn't exist
* aws ssm parameter lookup test case - failing case for nice return from path lookup
* aws ssm parameter lookup - convert incoming taglist to a key-value dictionary
* aws ssm parameter lookup - pep8 / style clean up
* aws_ssm lookup plugin rewrite for more standard interface
* aws_ssm module and lookup - introduce integration test and fix:
* aws_ssm module and lookup - error case integraton test and many PEP8 and other cleanups
* aws ssm parameter lookup - Various fixes in response to review + recursive fix & test
* aws ssm parameter lookup - more in response to review - shertel/abadger
* aws ssm parameter lookup unit test - move to mocker according to abadger
* aws ssm parameter lookup - integrate with new documentation fragment
* aws ssm parameter lookup - accept either aws_profile or boto_profile
* aws ssm parameter lookup - eliminate lookup document fragment until env vars are fixed later
* Remove redundant default for target_paths
* Make invalidation requests idempotent
* Remove redundant exception handling around boto3_conn
* Add cloudfront_invalidation to the changelog
1. Removed redundant default values in argument_spec
2. Enclosed non standard python libraries in Try Except ImportError block
3. Added region as a mandatory element in argument_spec
4. Updated the typo in api_params while accessing job queue. Changed it from job_queue to jobQueue
5. Updated the set_api_params() method to use snake_dict_to_camel_dict()
6. Updated the ANSIBLE_METADATA version from 1.0 to 1.1
7. Updated the version_added in Documentation from 2.4 to 2.5
8. Initialized response value to None
9. Updated the RETUN documentation
10. Updated the return from manage_state() method to not embed batch_job_queue_action in ansible_facts
11. Updated the module.exit_json() in main() to convert any keys back to snake_dict format using camel_dict_to_snake_dict to maintain consistency throughout the module
12. Changed the filename from batch_job_queue.py to aws_batch_job_queue.py
Enforce module deprecation.
After module has reached the end of it's deprecation cycle we will replace it with a docs stub.
* Replace deprecated modules with docs-only sub
* Use of deprecated past deprecation cycle gives meaningful message (see examples below)
* Enforce documentation.deprecation dict via `schema.py`
* Update `ansible-doc` and web docs to display documentation.deprecation
* Document that structure in `dev_guide`
* Ensure that all modules starting with `_` have a `deprecation:` block
* Ensure `deprecation:` block is only used on modules that start with `_`
* `removed_in` A string which represents when this module needs **deleting**
* CHANGELOG.md and porting_guide_2.5.rst list removed modules as well as alternatives
* CHANGELOG.md links to porting guide index
To ensure that meaningful messages are given to the user if they try to use a module at the end of it's deprecation cycle we enforce the module to contain:
```python
if __name__ == '__main__':
removed_module()
```
* Add an AWS Direct Connect virtual interface module
Fix DirectConnectError exception wrapper to be compatible with fail_json_aws()
* Fix documentation and handle BotoCoreErrors as well
* Add the aws and ec2 documentation fragments
* add purge option for secondary ip addresses and fix changed to reflect when addresses are modified
* Improve option documentation and make the secondary_private_ip_addresses option required if purging secondary private ip addresses.
* ec2_eni: update version_added to 2.5 for purge_secondary_private_ip_addresses option.
* Add aws_ses_identity module
* Update CI alias, add BotoCoreError exception handling.
* Add SES and SNS permissions to hacking/aws_config to run aws_ses_identity integration tests
* permit tagging of lambda functions
Signed-off-by: fernandoalex <jfernandoalex@gmail.com>
* changed iteritems for items and test for identity to is not
* fixed task name in example
* module object was not being passed to the function
Module object was not being passed to the funtions whitch was causing error in the error handling.
* Use compare_aws_tags
Check for attribute and fail if botocore needs to be updated
Update version_added
* Fix examples
* pep8
* Remove unnecessary var
Handle BotoCoreError
ecs_taskdefinition_facts should not populate ansible_facts with
so much information.
Better to just return the contents of the dict directly.
Reduce line lengths, use AnsibleAWSModule, don't reimplement
camel_dict_to_snake_dict.
`wait_for_instances: no` means do not wait for instances to become
InService/Healthy before terminating them. It does not mean don't
wait for the ASG to delete.
Not waiting for the ASG to delete can cause problems when recreating it.
Ensure that waiting for the ASG to delete respects `wait_timeout`.
Add `iam_user` module that supports check mode
Based entirely on `iam_group` with check mode support added.
Does not support management of groups, so as not to interfere with the
`iam_group` module.
* Update the copyright
* Remove e.message from exception handling since it is not available on python 3
* New aws_region_facts module to be able to get facts from AWS regions
* Fix copyright
Handle BotoCoreError too
Remove exception handling around around boto3_conn
Fix docs
Fix version_added
* Update metadata_version
Fix return doc
* New module: ec2_vpc_egress_igw
* minor pep8 fix and doc update
* add test dir and files
* add tests for gateway module
* fix up return documentation per review
* remove HAS_BOTO3 stuff as it is handled in AnsibleAWSModule per review
* fix an error with failure message and add custom handler for non-existent vpc ID
* add additional tests and update tests per review
* ignore errors on clean up tasks
* update module copyright to newer format
* fix exception handling since BotoCoreError doesnt have response attribute
* actually fix exception handling this time so it works with Py3 as well
* ecs_ecr: Remove registry ID from create repository call
[Boto3 documentation][1] specifies 'repositoryName' as the only expected
argument. The `**build_kwargs(registry_id)` part also adds 'registryId' which,
when executed, fails with: 'Unknown parameter in input: “registryId”, must be
one of: repositoryName'.
[AWS API documentation][2] also lists only the 'repositoryName' parameter. I.e.
this is not a problem with the boto3 library.
The default registry ID for the account that's making the request will be used
when creating the rpository. This means that if the `registry_id` specified by
the user is different from the default registry ID, then the policy changes
following the repository creation would fail, because the repository will have
been created in one repository but subsequent calls try to modify it in
another. Added a safeguard against this scenario.
[1]: https://boto3.readthedocs.io/en/latest/reference/services/ecr.html#ECR.Client.create_repository
[2]: https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_CreateRepository.html
* Fix concurrent ECR integration tests
If the `ecr_name` is the same in multiple concurrent test runs, then they can
interfere with one another causing both to fail. The `resource_prefix` is
guaranteed to be unique for different jobs running in CI an so avoids this
issue while also making it easier to identify the test which created the
resource.
It is not possible to modify the load balancer configuration
for ECS Service.
As it is possible to detect this, it's nicer to fail gracefully
than return AWS's less meaningful failure message.
Fix PEP8 compliance
* Moved the encryption to its own action method.
* removed silly default value for encryption type.
* Code formatting issues from pull request ANSIBOT.
* changed version_added to "2.5"
because of new new options available
* changed version_added to "2.5"
because of new new options available
* changed version_added to "2.5"
because of new new options available
* added cloudfont.py, modified cloudfront_facts.py class name and fixed a minor bug
* Improvements to cloudfront_distribution
* Reduce the scope of the cloudfront_distribution module
* Remove presigning
* Remove streaming distribution functionality
* Add full test suite for cloudfront distribution
* Meet Ansible AWS guidelines
* Make requested changes
Fix tests
Use built-in waiter
Update copyright
* Added check to prevent failed empty changesets from being left behind
* Fixing comments from PR 34933, prevent infinte loop and stricter exception catching
* describe_images is very slow if not filtered to owner accounts
*or* if the Owners parameter is passed (unless the Owners parameter
is `self`). Convert Owners parameters to `owner-id` and `owner-alias`
filters where possible. Tests with CLI show that `--owners self` is
fast, `--owners 123456789012` is slow (with or without owner-id filter).
* describe_image_attributes fails against accounts other than your
own. Launch permissions are useful information, but not critical.
* Fix ec2_vpc_net tags
PR #33105 broke the tags returned by ec2_vpc_net - it was returning the raw boto3 list instead of a dict as expected.
* Add a test for tags
* Module DOCUMENTATION should match argspec
Large update of many modules so that DOCUMENTATION option name and
aliases match those defined in the argspec.
Issues identified by https://github.com/ansible/ansible/pull/34809
In addition to many typos and missing aliases, the following notable
changes were made:
* Create `module_docs_fragments/url.py` for `url_argument_spec`
* `dellos*_command` shouldn't have ever had `waitfor` (was incorrectly copied)
* `ce_aaa_server_host.py` `s/raduis_server_type/radius_server_type/g`
* `Junos_lldp` enable should be part of `state`.
* port elb_classic_facts to boto3
update module to use AnsibleAWSModule
* Add RETURN docs for elb_classic_lb_facts
* Remove superfluous exception handling around connection
Fix exit_json call and RETURN docs
* fix broken import
* ansible.module_utils.ec2.HAS_BOTO is already used
* wait_timeout parameter: use 'int' type
* wait is always True there
* doc: use formatting function
* Allow protection of certain keys during camel_to_snake
Create an `ignore_list` parameter that preserves the case
of the contents of certain dictionaries. Most valuable
for `tags` but other uses might arise.
* Port ec2_vpc_route_table to boto3
Update tests to reflect fixes in boto3.
* Add RETURN documentation to ec2_vpc_route_table
* Update DOCUMENTATION to be valid yaml
* Add check mode tests
* Add test for unexpected egress rule in default VPC
When passing rules_egress to ec2_group, the default
egress rule shouldn't be created (if `purge_rules_egress`)
is set. Test this.
* Respect egress rule defintions for default VPC groups
When passing rules_egress and purge_rules_egress, the
default egress rule should not be created
Fixes#34429
* Change AWS credential passing to be YAML anchors
Vastly simplify the AWS tasks by reducing the credentials to a YAML
block
* wrap boto3 connection in try/except and handle exception, add traceback import
* params dont need to be mutually exclusive and support check mode
* add check to set nacl_ids to empty list instead of None for boto3 func
* standard exception handling using traceback
* update current RETURN documentation which is incorrect.
* update logic to check for and return ipv6 cidr instead of ipv4 cidr in a NACL
* ignore default and reserved rule numbers - greater than 32766
* was breaking on non-standard protocol numbers - AWS nacls now support full list -1 to 254
* update port range and icmp type/code handling
* add some more detail to returned nacl entries
* Update exception handling to account for BotoCoreError
Remove exception handling from the boto connection creation since it does nothing
* rules numbers from 32767 to 65535 are reserved for internal AWS use so we ignore here
* Add some integration tests for ec2_asg.
* Remove exception handling from ec2_asg boto3 connection since it is handled in boto3_conn().
* Update test failure assertions
* Use yaml anchor for credentials and remove unnecessary dependencies.
* Move AWS boto3 module credentials tests to separate target
* Remove filters from tests
* Use `AnsibleAWSModule`
* Update exception handling to use `fail_json_aws` and check
for `BotoCoreError` exceptions associated with bad connection
parameters.
* Remove connection creation exception handling as it does nothing.
Remove examples from main section and make sure it is all in the examples. Should helpw ith the HTML breakage (it is too wide for most monitors).
tyop fixes too.
* WIP adds network subnetting functions
* adds functions to convert between netmask and masklen
* adds functions to verify netmask and masklen
* adds function to dtermine network and subnet from address / mask pair
* network_common: add a function to get the first 48 bits in a IPv6 address.
ec2_group: only use network bits of a CIDR.
* Add tests for CIDRs with host bits set.
* ec2_group: add warning if CIDR isn't the networking address.
* Fix pep8.
* Improve wording.
* fix import for network utils
* Update tests to use pytest instead of unittest
* add test for to_ipv6_network()
* Fix PEP8
* Port ec2_vpc_net to boto3 and add support to expand existing VPCs
* Add s-hertel as an author for ec2_vpc_net
* Update ec2_vpc_net test for new error triggered by lack of credentials
Fix backwards compatibility
Document new return value
* Fix pep8 and return documentation
* Remove boto usage from boto3 modules
* Remove region checking
boto3_conn now takes care of region checking and handles NoRegionError
exceptions with a standard message
boto3_conn also takes care of other connection exceptions too.
* Document boto3 as a requirement for ec2_eni_facts
* Return id of ENI in addition to network_interface_id. To be compatible to ec2_eni.
* Added documentation for the return values of the ec2_eni_facts module
* Fix typo in docs for ec2_eni_facts
* Readded vital return parameters to the ec2_asg that have been spared for no obvious reason
* Fix typo in ec2_asg docs
* Fixing another typo in ec2_asg docs.
* Bulk pep8 fixes - hand crafted
Fix by hand the remaining issues that autopep8 couldn't
* Next batch of hand crafted pep8 fixes
* Ignore W503
https://github.com/PyCQA/pycodestyle/pull/499
* Revert more of W503
* Fail when attempting to modify unmodifiable target group parameters
As you can't modify Port, Protocol or VPC id for a target group, fail
when this happens rather than pretending to do it.
One could argue that the target group could be recreated rather than
failing, but this has massive knock on implications to other resources
that depend on the TG (all ASGs would need to be updated, the ELB
listener would need to be updated, etc)
* Use `.get()` instead of direct dictionary access
* Added modules to create, delete, and describe EC2 Placement Groups.
* Remove unnecessary print statement
* Update to use boto3.
* De-linting
* Remove facts from this PR
* Update to newer method of handling Boto3 connections and exceptions.
* Futzing around with imports and HAS_BOTO3
* Fix up exception imports.
* Remove redundant default.
* Handle DryRunOperation errors appropriately.
* Remove redundant BOTO3 check.
* Use shorter licence declaration.
* Remove redundant HAS_BOTO3 import.
* Add AWSRetry decorators to API calls.
* Add new 'strategy' parameter to allow for cluster and spread PGs.
* Added module to describe EC2 Placement Groups.
* Update to newer way of handling Boto3 connections and exceptions.
* Futzing around with imports and HAS_BOTO3
* Fix up exception imports.
* Fix license block
* Correct task indenting in doc
* Added list-filtering example to docs
All the values currently documented as return values are returned inside a 'result' key.
So if you registered the output of the task as 'output', then you would need to do 'output.result.zone_id' instead of 'output.zone_id'.
This commit fixes that so that you can do 'output.zone_id'.
* AWS S3: fix method call for fakes3 S3 backend
Fixes#33083
Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
* Auto append port based on proto
Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
* Add integration test suite for ec2_vpc_subnet
* wrap boto3 connection in try/except
update module documentation and add RETURN docs
add IPv6 support to VPC subnet module
rename ipv6cidr to ipv6_cidr, use required_if for parameter testing, update some failure messages to be more descriptive
DryRun mode was removed from this function a while ago but exception handling was still checking for it, removed
add wait and timeout for subnet creation process
fixup the ipv6 cidr disassociation logic a bit per review
update RETURN values per review
added module parameter check
removed DryRun parameter from boto3 call since it would always be false here
fix subnet wait loop
add a purge_tags parameter, fix the ensure_tags function, update to use compare_aws_tags func
fix tags type error per review
remove **kwargs use in create_subnet function per review
* rebased on #31870, fixed merge conflicts, and updated error messages
* fixes to pass tests
* add test for failure on invalid ipv6 block and update tags test for purge_tags=true function
* fix pylint issue
* fix exception handling error when run with python3
* add ipv6 tests and fix module code
* Add permissions to hacking/aws_config/testing_policies/ec2-policy.json for adding IPv6 cidr blocks to VPC and subnets
* fix type in tests and update assert conditional to check entire returned value
* add AWS_SESSION_TOKEN into environment for aws cli commands to work in CI
* remove key and value options from call to boto3_tag_list_to_ansible_dict
* remove wait loop and use boto3 EC2 waiter
* remove unused register: result vars
* revert az argument default value to original setting default=None
* Allow backoff for describe_subnets
Improve exception handling to latest standards
* Add integration test suite for ec2_vpc_subnet
* Add test for creating subnet without AZ
Fix bug identified by test
Fixes#31905
* Allow ec2_lc module to use volume_type for block devices
Makes ec2_lc consistent with ec2, ec2_ami, ec2_vol etc.
* Add deprecation message for device_type
New module for obtaining facts from elasticache clusters
Fixes#30373
* Removed unnecessary boto3 and exception checking
* AnsibleAWSModule checks for lack of boto3
* boto3_conn handles error checking of AWS connection
Ed Costello
Sloane Hertel
mikedlr
Julien Vey
Ryan Brown
kbreit
Stefan Horning
KrdLab
subcon42
Willian Ricardo
Rob
Will Thames
Willem van Ketwich
Jonathan Meran
John R Barker
Stephen Granger
Fernando Alexandre
Josh Souza
Henrique Rodrigues
Daniel Shepherd
Deiwin Sarjas
René Moser
Marek
Chetan-CSI
Christopher Warner
Prasad Katti
clarkst
Matt Martz
RobReus
Matt Clay
Brandon Davidson
kwerey
Madhura-CSI
Pilou
Vinay Dandekar
Takuya Sato
Mohit Kumar
Abhijeet Kasurde
Will Thames
Igor (Tsigankov) Eyrich
Ted Timmons
Gustavo Maia
Kim Egede Jakobsen
Gobin Sougrakpam
Brian Lamar
Øyvind Saltvik
Matthew Staebler
Mamad Purbo
Justin Menga
Matt Doller
Brad Macpherson
Deepakkothandan
Christopher Bradley
Kaz Cheng
Daniel Temme
Brad Chamberlain
Simon Fullick
Marc-Aurèle Brothier