remove cloudtrail.py from legacy-files as it passes pep8 tests now
update return values to use snake case per coding guidelines
update boto3 exception handling to use method outlined in coding guidelines
update parameter spec and return value code per PR code review
Ensure newly created NAT gateways get converted to snake dict
Remove custom code, and associated test, for generating snake
dict and use `camel_dict_to_snake_dict`
Make use of `required_if` rather than bespoke parameter checks
Remove ec2_vpc_nat_gateway from pep8 legacy files list
If a lambda exists but does not have a VPC configuration,
add the VPC configuration when it's present in the ansible
parameters.
Prior to this change, setting VPC configuration on a lambda
that did not have any VPC configuration would ignore that config.
* switch to boto3 and add support for application ELBs with target groups.
* use py23 compatible dict iterator.
* removing commented out fail_json calls
utilize sets to simplify logic
remove setting a redundant variable
add bounds checking in two places
add AWSRetry decorator - do we want this for other functions too?
change xrange to range so python3 doesn't fail
remove sorting lists of dicts; in python2 this returns None, in python3 this fails
* remove error variable from traceback.format_exc
* Remove boto2-style calls brought in by rebase
Old boto-style calls to `as_group` attributes break in boto3
Also remove module from legacy-PEP8 list
* Add parameter to target_group_arn option
* Fix HAS_BOTO3 check
* use tags.items() instead of iteritems
* import botocore
* Fixed bugs in deleting autoscaling groups
* make changes in deleting autoscaling groups pep8
* more pep8
* fix version
* fix bugs so local integration tests run
* fix launch config check
* reflect changed status for ASG updates
* Fix existing exception handling and use traceback.
Fix imports
* line length
* Fix notification setup
* Fix mutually exclusive arguments
Only one of the AvailabilityZones and VPCZoneIdentifier arguments should be provided to the CreateAutoScalingGroup call.
* Allow desired_capacity, min_size, max_size, launch_config_name to be derived from the existing ASG if not specified
Remove code updating dict after ASG already uses it
* Add new helper function for comparing AWS tag key pair dicts. Also modify boto3_tag_list_to_ansible_dict function to be more generic when looking for key names because AWS sometimes uses 'Key', sometimes 'TagKey' and who knows what the future holds! Fixed modules to work with changes.
* Review changes
* Add some more doc to GUIDELINES for tags and fix var name for snaked values in ec2_group_facts
* allow elb scheme to be updated by restarting the resource
use ensure_gone and require wait option
* fix pep8
* fix yaml
* requested changes
* fix fail_json
- Use `fail_json` for `get_all_load_balancers` unhandled exception.
- Do not mask error message on failure of `enable_zones`.
- Do not mask error message on failure of `disable_zones`.
Also added/removed blank lines to improve PEP 8 compliance.
* fix documentation and correct exception handling
* follow AWS exception guidelines
* fix parameter_group_family req; only needed when creating cache parameter group
make pep8 and remove from legacy files
update module to support more standard state=present/absent syntax
update module to use required_if, required_together, mutually_exclusive functions where possible
per ryansb review: make documentation section more clear, fix some extra quotes, remove FIXME comment
pre willthames review: force private_zone to True if vpc_id is set and fix word wrap
* Support check mode in ec2_vpc_dhcp_options_facts
As a facts module, ec2_vpc_dhcp_options_facts supports check mode
by default
* ec2_vpc_dhcp_options_facts tidy up
Use named method imports, move imports to top of code
Use shared code to handle filters and tags
Use snake case for parameter names while retaining backward compatibility
This fixes issue when list from module contains more than one element.
Ansible and/or boto may put same elements in list in different order,
thus resulting task as changed.
Fixes#3310
* allow split horizon for route53_zone and refactor
* fix documentation
remove comment
fix version_added
* Remove unused imports
* Only include zone as matching if it has the same privacy setting
* Use `.endswith` instead of indexing into a string
* Update public zone behavior to only create new if there is no matching public zone
* Remove from legacy PEP8 files
* Add ability to use either sec group name or sec group id or combination to ec2_lc. Also fix header size in AWS Guidelines
* Fix ec2 import to only import neccessary packages. Remove pep8 double line
* re-raise exception as expected
The current ThrottlingException handling code hides other actual exceptions from the user, and basically goes infinite loop instead. eg when the api caller doesn't have permission (example below) to use the api, ansible effectively hangs. adding the re-raise stops execution and shows the error to the command line as expected
can test by removing permission to the efs api, and calling the efs: module
The error was: botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the DescribeFileSystems operation: User: <x> is not authorized to perform: elasticfilesystem:DescribeFileSystems on the specified resource
* PEP8 fix
* Remove DryRun parameter in ec2_vpc_igw_facts check_mode
Using DryRun in check mode causes errors, and is not required
(as nothing changes when calling describe_internet_gateways)
Prevents the following error:
```
{"changed": false,
"failed": true,
"msg": "An error occurred (DryRunOperation) when calling
the DescribeInternetGateways operation: Request
would have succeeded, but DryRun flag is set."}
```
* ec2_vpc_igw_facts pep8 tidy up
* [cloud][tests] Create fixtures for using placebo to test boto3-using modules
* Use pytest's importorskip instead of manually skipping on missing deps
* Fix imports in cloudformation module
* Delete unused code
* Add maybe_sleep fixtures to speed up recorded test runs
* Build basic placebo-CFN tests
* Commit placebo recordings of basic stack operations
* Add placebo to test-requires
* Allow unit tests to run regardless of environment by setting a default region
* Use explicit relative import for Python 3 compat
* Use __name__ attribute that works on Python 2 and 3
* add else statement to ensure instids is set
set res_list to None to avoid UnboundLocalError and fix iteration over a nonetype by adding an empty tuple
* make res_list empty tuple by default and check for instids before setting tags (fails otherwise)
Set encrypted to default False, rather than None
Otherwise you get:
```
Invalid type for parameter Encrypted, value: None, type: <type 'NoneType'>, valid types: <type 'bool'>)
```
* update integration tests for updated boto exception message
* integration tests fail on both "test credential" test cases
exception bubbles out of module. instead catch and wrap
* ec2_group does not support updating a security group's description
AWS security group descriptions are immutable.
if ec2_group finds a group that matches by name, but the descriptions do not match, the module does not support this case
previously it would check if the group was used, but would not do anything if it was
old behavior was erroneous because it could make a user expect that the description change of a group was fine when in fact it did not occur
also, it made an expensive check against all ec2 instances for no good reason
* comments not doc strings
* else must have pass w/o doc-string statement
* Catch specific BotoServerException, give context around error when fetching SGs
* python3 compatible exception blocks
* add traceback to fail_json
* two blank lines before first function
purge_subnets|routes were introduced recently.
ensure_subnet_associations now takes purge_subnets as a parameter. A
call to this function was missed when introducing this feature. With
out, results in a "got 5 expected 6" error.
Allows to health-check two services on one host. Before, asking for 2 health checks being present for `http://myhost:8888` and `http://myhost:9999` resulted in just a single health check, as this module considered the two specs as equal.
Bonus: route53_health_check now passes style guidelines
* Fix get_s3_connection (fixes#22317)
Override aws_connect_kwargs rather than prepending to them. Should fix an issue in which `calling_format` is set twice in the kwargs passed to `boto.connect_s3` or `S3Connection` if a bucket name contains a `.`
* Revert "Fix get_s3_connection (fixes#22317)"
This reverts commit 7f61b8bebd2929940495204f1a98d660a55985d8.
* implements alternative way of fixing issue with aws_connect_kwargs for rgw and fakes3 (fixes 22317)
* add comment to explain why the keys are being removed from aws_connect_kwargs
* remove trailing whitespace on comment line
* don't modify an eni without eni_id
fix pep8
remove ec2_eni from pep8 legacy files
remove a couple unused variables in main()
fix the documentation
* fix yaml
* making ec2 pep8
* remove ec2 from pep8 legacy files
* missed a couple
* fix imports and remove iteritems
* making group_id and group_name mutually exclusive and fixing whitespace
Changes to the metadata format were approved here:
https://github.com/ansible/proposals/issues/54
* Update documentation to the new metadata format
* Changes to metadata-tool to account for new metadata
* Add GPL license header
* Add upgrade subcommand to upgrade metadata version
* Change default metadata to the new format
* Fix exclusion of non-modules from the metadata report
* Fix ansible-doc for new module metadata
* Exclude metadata version from ansible-doc output
* Fix website docs generation for the new metadata
* Update metadata schema in valiate-modules test
* Update the metadata in all modules to the new version
Currently the documentation utilizes with_subelements, but it does not parse the results correctly. By changing to with_items: and specifying the proper list, we are able to tag the instances as expected.
Unclear how useful it is to still be catching BotoServerError exceptions
generally, especially when we are now hopefully handling most instances
of throttling, but kept it as it was already in the code. Moved to main
as catching BotoServerErrors inside individual functions will bypass any
AWSRetry.backoff decorator.
Move additional imports to top.
Use python2.6 formatting
A bit of cleanup on imports
Make one more string interpolation Python 2.6 compatible.
* Fix connection_draining idempotency in ec2_elb_lb
Ensure connection_draining types are equivalent when comparing
whether or not connection_draining is being changed.
This means that running `ec2_elb_lb` with connection_draining
set a second time will now report `changed=False`
* ec2_elb_lb: fixed latest Ansible standards compliance
Update to current pep8 standards, fix module imports
and remove module from exclusion file
* fix module doc fields
* More module docs corrections
* More module docs corrections
* More module docs corrections
* More module docs corrections
* correct aliases
* Review comments
* Must quote ':'
* More authors
* Use suboptions:
* restore type: bool
* type should be in the same place
* More tidyups
* authors
* Use suboptions
* revert
* remove duplicate author
* More issues post rebase
* description is only required when group state is present
also note that AWS requires a non-empty description when creating a security group
* clarify description requirement
1. Check less aggressively for resources that take a long time to return
(some resources can take an hour or more).
2. Handle throttling when checking for resources, and back off quickly
when it occurs.
* Add user_data_path parameter to ec2_lc module
* Improve user_data_path parameter documentation
- Specify mutual exclusivity
- Specify `version_added`
- Change module parameter type to `path`
* Use correct comparison for user_data result
* Include traceback on error and use with block
* Only hide user data if provided in file
ensure_tags fails if `tags` is None rather than an empty
dict. Ensure that not passing `tags` parameter is equivalent
to passing an empty dict.
Fixes#21778
Fixes ansible#21796 Prevent users from deleting buckets rather than objects by making object parameter and mode=delobj mutually exclusive in task.
https://github.com/ansible/ansible/issues/21796
The list_elbs call to boto doesn't use any pagination, so any time there
are more ELBs than the API page size, this module will fail. This change
uses the `next_token` attribute of `ResultSet` to check if there are
still more ELBs to return.
Fixes#21361
* Make more use of AWS Exception information
* Use pythonic dict assignments
* Improve documentation formatting to make option names clearer
in descriptions
* Meet ansible line length requirements
* Tidy up flake8 messages
Set the `force` flag to false to:
- protect existing Keys from being overwritten
- check if a Key was already existing by evaluating
the returned `changed` status
CentOS's cloud image wiki page (wiki.centos.org/Cloud/AWS) gives an
example of using their product code to search for AMIs. This commit
adds an optional parameter to ec2_ami_find to allow this filtering.
If versioning is not enabled, then `get_bucket_versioning`
can return an empty dict.
If that happens, the code to enable versioning should still
run!
The logic for suspending versioning was also incorrect, so
have updated that too.
Fixes#20491
Introduces `ignore_nonexistent_bucket` option that does not require ListBuckets permission, so less permissioned IAM roles can still be used with this module.
Fixes#20501
Don't try to create tags on a vpc that you've just removed.
Avoids
```
"msg": "An error occurred (InvalidParameterValue) when calling the CreateTags operation: You must specify one or more tags to create"
```
Although not quite sure why the `create_tags` was being called
as `module.params.get('tags')` *should* have returned `None`.
* Adding an elasticache snapshot module.
Allows user to create, copy, or delete a snapshot.
* Removing unnecessary function
* Make indentation uniform.
* Making requested changes.
Fixing PEP8
Adding a more graceful fail for delete() if the snapshot's state valid (such as when it is in the process of being created).
* PEP8
* Fixing some formatting
move imports
fix parameter alignment
* move imports to the top of the file below documentation
Replaces the usage of expanduser with the type 'path' for the
dest and object options as that functionality is available
in that type automatically.
patch related to #12263
* initial commit of cloudfront_facts.py
* modification as per review from @georgepsarakis
* fixed shippable build error
* fixed shippable build error
* removed wildcard imports and replaced with specific imports from ansible.module_utils.ec2 as advised by @georgepsarakis
* renamed all instances of cloud_front_origin_access_identity to origin_access_identity as advised by @georgepsarakis
* show facts based on alias and distribution id for easy referencing as advised by @ryansb. have done for both distribution and distribution_config
* fixed incorrect logic for default list_distributions, fixed list_distributions_by_web_acl - wasn't passing web_acl_id, fixed list_invalidations keyword args missing DistributionId
* fixed last fact added clobbering all previous facts for an alias or an id
* removed list_ prefix from list keys
* removed unnecessary boto fields. made list_distributions and list_streaming_distributions dictionaries with id/alias as key. fixed list_invalidations.
* initial commit of cloudfront_facts summary
* checks for empty list returned from boto, standardised list naming
* neatened up parameters
* added summary documentation
* refactoring of functions for modularity
* refactoring, neatening code, fix for if cname not present, added try-catch blocks
* more refactoring, cleaning
* more cleaning, allowed streaming distributions to be found by domain name alias
* removed unnecessary line
* fixed tabs
* fixed indentation
* removed trailing whitespace
* removed more whitespace
* more refactoring, modified where invalidations dict is set
* added ETag to summary
* refactored summary list
* moved list out of for loop
* trailing white space
* reverted line lengthening as advised by @willthames
* modified exceptions to multi line as advised by @willthames
* reverted variable spacing to be more pythonic'
* reverted spacing in YAML as advised by @ryansb
* reverted line spacing for parameters for correct blame attribution as advised by @ryansb
* removed white space
* more white space
* reverted line spacings for parameters as advised by @ryansb
* removed spaces between parameters as advised by @ryansb
* Avoid having module documentation links to itself
A lot of modules use M(own_module) in their documentation causing a link
in the documentation to itself.
* Make note more clear now
* Removing cmp to be compatible with Python3
* fix syntax
* Added an overlooked return
* Fixing making things overly complicated
* Simplifying since tags will always be hashable. Don't need to use cmp at all.
* Adding environment variable support for lambda.py in response to feature idea #20479.
Plus a 1-character bug fix.
* fix yaml syntax
* Fixing option name, adding alias, and fixing a line to allow the user to delete environment variables by setting an empty dict.
* Update validate-modules
* Validates ANSIBLE_METADATA
* Ensures imports happen after documentation vars
* Some pep8 cleanup
* Clean up some left over unneeded code
* Update modules for new module guidelines and validate-modules checks
* Update imports for ec2_vpc_route_table and ec2_vpc_nat_gateway
Boolean options that default as `None` but are set to `False` by the user were ignored on update. This change checks to distinguish None & False so that options like multi_az can be turned off during an update.
* Modifying how optional parameters are handled in rds.py. Fixes#20370
Allowing options to be set to false/no. Previously ignored unless set to true/yes.
Added a conditional for invalid parameters since the default is false instead of null for some options (e.g. force_failover, apply_immediately, upgrade).
* Making requested revision.
* Disassociate subnets from route tables before deletion
If a route table still has subnets associated with it, it will fail
to delete:
```
"msg": "The routeTable 'rtb-abcd1234' has dependencies and cannot be deleted."
```
Avoid this by disassociating subnets before route table deletion
* Fix ec2_vpc_route_table flake8 complaints
* Check if EIP exists before deleting it
After deleting the NAT gateway, the EIP sometimes seems to
cease to exist afterwards. Check if it exists before deleting it.
Otherwise you get:
```
Failed to release EIP eipalloc-abdc1234: An error occurred (InvalidAllocationID.NotFound) \
when calling the ReleaseAddress operation: The allocation ID 'eipalloc-abcd1234' does not \
exist", "success": false}
```
* Fix flake8 errors with ec2_vpc_nat_gateway
* added ec2_vpc_igw_facts module
* added cr at end of file
* corrected import json in wrong location
* corrected version added
* added snake_case conversion
* updated documentation and fixed for python 3'
* ec2_vpc_igw_facts: simplify logic
Make module arguments more 'Ansiblish'
Remove unnecessary intermediate variables in results generation
Use `ansible_dict_to_boto3_filter_list` rather than duplicating logic
Use `check_mode` rather than pass a `dryrun` argument
Update for flake8 improvements
* updated documentation
Update exception handling, remove use of iteritems
Update for better flake8 compliance
Use ansible_dict_to_boto3_filter_list rather than
duplicating its implementation
* new AWS module for ec2 VPC vgw facts
* fixed documentation as suggested by reviewers
* changed iteritems to items for py 3
* improvements to bring in line with #19787
* corrected documentation
* updated doco and exception path
* Deprecate ec2_vpc module
The deprecation of ec2_vpc module has been discussed for
2 years and is causing duplication of effort as changes
are implemented for ec2_vpc rather than for the newer
alternatives
* Improve module deprecation documentation
Update the developing modules documentation with the latest
instructions on how to deprecate a module.
* Fixes: #3217 - Add a new parameter for the kms_key_id to the ec2_vol module.
* Fixes: #3217 - Correcting comment string as requested.
* Fixes: #3217 - Adding boto version when kms_key_id is used. Also re-adding accidentally removed comment line.
* Cleanup of EBS volume key pull request
* Adding support for Amazon ECR
This patch adds a new module named ecr, which can create, update or
destroy Amazon EC2 Container Registries. It also handles the management
of ECR policies.
* ecs_ecr: addressed review feeback
* Renaming ecr to ecs_ecr
* Fixed docs
* Removed bad doc about empty string handling
* Added example of `delete_policy`
* Removed `policy_text` option; switched policy to `json` type so
it can accept string or dict
* Added support for specifying registry_id
* Added explicit else after returned if clauses
* Added `force_set_policy` option
* Improved `set_repository_policy` error handling
* Fixed policy comparisons when AWS doesn't keep the ordering stable
* Moved `boto_exception` into the module
* initial commit of cloudfront_facts.py
* modification as per review from @georgepsarakis
* fixed shippable build error
* fixed shippable build error
* removed wildcard imports and replaced with specific imports from ansible.module_utils.ec2 as advised by @georgepsarakis
* renamed all instances of cloud_front_origin_access_identity to origin_access_identity as advised by @georgepsarakis
* added input option requirements where missing. fixed a typo in an error message, added '.' to the end of sentences.
* fixed typo in documentation
* simplified paginated_response method as advised by @georgepsarakis
* set default option to list_defaults when no option specified as advised by @ryansb
* bumped version to 2.3 as advised by @ryansb
* removed double-nesting of ansible_facts as advised by @ryansb
* show facts based on alias and distribution id for easy referencing as advised by @ryansb. have done for both distribution and distribution_config
* made comments clearer
* fixed incorrect logic for default list_distributions, fixed list_distributions_by_web_acl - wasn't passing web_acl_id, fixed list_invalidations keyword args missing DistributionId
* neatened up logic for list_distributions default
* facts now return Etag as well as specific facts. modified paginated_response to handle this. also all requests that use distribution id also list the facts under the corresponding alias now.
* fixed last fact added clobbering all previous facts for an alias or an id
* removed list_ prefix from list keys
* removed unnecessary boto fields. made list_distributions and list_streaming_distributions dictionaries with id/alias as key. fixed list_invalidations.
* fixed incorrectly named list_distributions_by_web_acl to ..web_acl_id. added id/alias dict for list_distributions_by_web_acl_id
* minor doc changes
* removed merge tag
* fixed more of merge
The main purpose of this PR is to add the subnet associations to the
dict returned by ec2_vpc_route_table_facts. This commit also
re-formats code to make it PEP8 compliant.
New module by @tedder for handling granting/revoking access to KMS secrets.
For example:
```
- name: grant user-style access to production secrets
kms:
args:
mode: grant
key_alias: "alias/my_production_secrets"
role_name: "prod-appServerRole-1R5AQG2BSEL6L"
grant_types: "role,role grant"
```
- Consistent capitalisation in the descriptions
- Removed redundant 'optional' notes when this is covered by the Boolean `optional` column
- Clarified `instance_id` description
* set is_public 'true' or 'false'
Despite being a boolean property, https://github.com/melta/boto/blob/master/boto/ec2/image.py:63 sets is_public = True only if the argument is passed in as the string 'true'. Likewise for False/'false'.
This is a workaround for that bug in boto2, to allow the documented parameter to work with valid yaml values.
fixes#5600
* only set is_public if true
Actual doc give this error
```
fatal: [localhost]: FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"command": "facts",
"register": "new_database_facts"
},
"module_name": "rds"
},
"msg": "unsupported parameter for module: register"
}
```
Register should be at the module `rds` level and not at args level in this example :
```
- rds:
command: facts
instance_name: new-database
- register: new_database_facts
+ register: new_database_facts
```
* Add new parameters to taskdefinition module - network_mode and task_role_arn
* Add version_added field for doco
* Change version_added parameter to 2.3
* Allow re-using an existing template when updating a stack by not passing 'template' or 'template_url'. This is a big one for me as our deploy process creates a new stack and then modifies the old one; to avoid changing the resources inside the old one, we have had to avoid using the Ansible module and use the AWS CLI instead in order to pass `--use-previous-template`.
* Split create and update logic into separate functions
* Remove dead `update` variable
PR move of https://github.com/ansible/ansible-modules-core/pull/3588
##### ISSUE TYPE
- Docs Pull Request
##### COMPONENT NAME
ec2_group.py
##### ANSIBLE VERSION
```
ansible 2.0.2.0
config file = /Users/tpai/src/cm-secure/ansible.cfg
configured module search path = Default w/o overrides
```
##### SUMMARY
Make it clear you can specify the created group in the rules list, allowing idempotent use for group<->group networking rules.
This is a really useful feature that isn't obvious enough in the docs.
* Change example syntax on supervisorctl module
* Change example syntax or _ec2_ami_search module
* Change example syntax on cloudformation module
* Change example syntax on ec2 module
* Change example syntax on ec2_facts module
* Change example syntax on ec2_eip module
* Change example syntax on rds module
* Change example syntax on route53 module
* Change example syntax on s3 module
* Change example syntax on digital_ocean module
* Change example syntax on docker_service module
* Change example syntax on cloudformation module
* Change example syntax on gc_storage module
* Change example syntax on gce module
* Change example syntax on gce_mig module
* Change example syntax on _glance_image module
* Change example syntax on _keystone_user module
* Change example syntax on _nova_keypair module
* Change example syntax on _quantum_floating module
* Change example syntax on _quantum_floating_ip_associate module
* Change example syntax on _quantum_network module
* Change example syntax on _quantum_router module
* Change example syntax on _quantum_router_gateway module
* Change example syntax on _quantum_router_interface module
* Change example syntax on _quantum_subnet module
* SQUASH _quantum_subnet
* Add missing quotes
Moving the "check if min_size/max_size/desired_capacity..." code to execute BEFORE the desired_capacity code is used in the following operation:
num_new_inst_needed = desired_capacity - len(new_instances)
Otherwise the following exception occurs when desired_capacity is not specified and you're replacing instances:
num_new_inst_needed = desired_capacity - len(new_instances)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
Stack Trace:
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 3044, in <module>
main()
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 3038, in main
replace_changed, asg_properties=replace(connection, module)
File "/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg", line 2778, in replace
num_new_inst_needed = desired_capacity - len(new_instances)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "ec2_asg"}, "module_stderr": "Traceback (most recent call last):\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 3044, in <module>\n main()\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 3038, in main\n replace_changed, asg_properties=replace(connection, module)\n File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1478229985.74-62334493713074/ec2_asg\", line 2778, in replace\n num_new_inst_needed = desired_capacity - len(new_instances)\nTypeError: unsupported operand type(s) for -: 'NoneType' and 'int'\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}
to retry, use: --limit @
In cases where a CFN stack could not complete (due to lack of
permissions or similar) but also failed to roll back, the gathering of
stack resources would fail because successfully deleted items in the
rollback would no longer have a `PhysicalResourceId` property.
This PR fixes that by soft-failing when there's no physical ID
associated to a resource.
Support the new native YAML format in the CloudFormation API. This means
the existing `template_format` parameter is deprecated. This commit also
adds a warning for the deprecated parameter.
- Don't rewrite the result; this is causing 'changed=true' on update
- Move AWSRetry import to top since it's a decorator, and is needed at definition-time
- removed star-imports, which wasn't possible in Ansible 1.x
- boto doesn't have any of the modern features (most notably, changesets), so this rewrite goes all-in on boto3.
- tags are updateable, at least in boto3. Fix documentation.
- staying with "ansible yaml to json conversion" because I'm trying to keep this scoped properly. The next PR will have AWS-native yaml support.
- documented the output. Tried to leave it backwards-compatible but the changes to 'events' might break someone's flow. However, the existing data wasn't terribly useful so I don't assume it will hurt.
- split up the code into functions. This should make unit testing possible.
- added forward-facing code: 'six' for iterating, started using AWSRetry, common tag conversion.
- add todo list
- Pass `exception` parameter to fail_json
* Restart EC2 instances with multiple network interfaces
A previous bug, #3234, caused instances with multiple ENI's to fail when being
started or stopped because sourceDestCheck is a per-interface attribute, but we
use the boto global access to it (which only works when there's a single ENI).
This patch handles a variant of that bug that only surfaced when restarting an
instance, and catches the same type of exception.
* Default termination_protection to None instead of False
AWS defaults the value of termination_protection to False, so we don't
need to explicitly send `False` when the user hasn't specified a
termination protection level. Before this patch, the below pair of tasks
would:
1. Create an instance (enabling termination_protection)
2. Restart that instance (disabling termination_protection)
Now, the default None value would prevent the restart task from
disabling termination_protection.
```
- name: make an EC2 instance
ec2:
vpc_subnet_id: {{ subnet }}
instance_type: t2.micro
termination_protection: yes
exact_count: 1
count_tag:
Name: TestInstance
instance_tags:
Name: TestInstance
group_id: {{ group }}
image: ami-7172b611
wait: yes
- name: restart a protected EC2 instance
ec2:
vpc_subnet_id: {{ subnet }}
state: restarted
instance_tags:
Name: TestInstance
group_id: {{ group }}
image: ami-7172b611
wait: yes
```
Per #3877, the code to wait for spot instance requests to finish would
hang for the full wait time if any spot request failed for any reason.
This commit introduces status checks for spot requests, so if the
request fails, finishes, or is cancelled the task will fail/succeed
accordingly.
One edge case introduced here is tha if a user terminates the instance
associated with the request manually it won't fail the play, under the
presumption that the user *wants* the instance terminated.
The AWS API requires that any termination policy list that includes
`Default` must end with Default. The attribute sorting caused any list
of attributes to be lexically sorted, so a list like
`["OldestLaunchConfiguration", "Default"]` would be changed to
`["Default", "OldestLaunchConfiguration"]` because default is earlier
alphabetically. This caused calls to fail with BotoServerError per #4069
This commit also adds proper tracebacks to all botoservererror fail_json
calls.
Closes#4069
Previously calculation of the number of instances that have been
terminated assumed all instances were in the first reservation returned
by AWS. If this is not the case the calculated number of instances
terminated never reaches the number of instances and the module always
times out. By unpacking the instances we get an accurate number and the
module correctly exits.
Currently instances with multiple ENI's can't be started or stopped
because sourceDestCheck is a per-interface attribute, but we use the
boto global access to it (which only works when there's a single ENI).
This patch handles multiple ENI's and applies the sourcedestcheck across
all interfaces the same way.
Fixes#3234
If you apply `wait=yes` and use `instance_tags` as your filter for
stopping/starting EC2 instances, this stack trace happens:
```
An exception occurred during task execution. The full traceback is: │~
Traceback (most recent call last): │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1540, in <module> │~
main() │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1514, in main │~
(changed, instance_dict_array, new_instance_ids) = startstop_instances(module, ec2, instance_ids, state, instance_tags) │~
File "/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py", line 1343, in startstop_instances │~
if len(matched_instances) < len(instance_ids): │~
TypeError: object of type 'NoneType' has no len() │~
│~
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_name": "ec2"}, "module_stderr": "Traceb│~
ack (most recent call last):\n File \"/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1540, in <module>\n main()\n File \"/tmp/│~
ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1514, in main\n (changed, instance_dict_array, new_instance_ids) = startstop_instances│~
(module, ec2, instance_ids, state, instance_tags)\n File \"/tmp/ryansb/ansible_FwE8VR/ansible_module_ec2.py\", line 1343, in startstop_insta│~
nces\n if len(matched_instances) < len(instance_ids):\nTypeError: object of type 'NoneType' has no len()\n", "module_stdout": "", "msg": "│~
MODULE FAILURE", "parsed": false}
```
That's because the `instance_ids` variable is None if not supplied
in the task. That means the instances that result from the instance_tags
query aren't going to be included in the wait loop. To fix this, a list
needs to be kept of instances with matching tags and that list needs to
be added to `instance_ids` before the wait loop.
Before this, all spot instance requests would fail because the code
_always_ called module.fail_json when the parameter was set (which it
always was, because the module parameter's default was set to 'stop').
As the comment said, this parameter doesn't make sense for spot
instances at all, so the error message was also misleading.
Due to a mixup of the group/role/user and policy names, policies with
the same name as the group/role/user they are attached to would never be
updated after creation. To fix that, we needed two changes to the logic
of policy comparison:
- Compare the new policy name to *all* matching policies, not just the
first in lexicographical order
- Compare the new policy name to the matching ones, not to the IAM
object the policy is attached to
* Check mode fixes for ec2_vpc_net module
Returns VPC object information
Detects state change for VPC, DHCP options, and tags in check mode
* Early exit on VPC creation in check mode
The default VPC egress rules was being left in the egress rules for
purging in check mode. This ensures that the module returns the correct
change state during check mode.
AWS security groups are unique by name only by VPC (Restated, the VPC
and group name form a unique key).
When attaching security groups to an ELB, the ec2_elb_lb module would
erroneously find security groups of the same name in other VPCs thus
causing an error stating as such.
To eliminate the error, we check that we are attaching subnets (implying
that we are in a VPC), grab the vpc_id of the 0th subnet, and filtering
the list of security groups on this VPC. In other cases, no such filter
is applied (filters=None).
EC2 Security Group names are unique given a VPC. When a group_name
value is specified in a rule, if the group_name does not exist in the
provided vpc_id it should create the group as per the documentation.
The groups dictionary uses group_names as keys, so it is possible to
find a group in another VPC with the name that is desired. This causes
an error as the security group being acted on, and the security group
referenced in the rule are in two different VPCs.
To prevent this issue, we check to see if vpc_id is defined and if so
check that VPCs match, else we treat the group as new.
While from the documentation[1] one would assume that replacing
CAPABILITY_IAM with CAPABILITY_NAMED_IAM; this as empirically been shown
to not be the case.
1: "If you have IAM resources, you can specify either capability. If you
have IAM resources with custom names, you must specify
CAPABILITY_NAMED_IAM."
http://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_CreateStack.html
The `source_dest_check` and `termination_protection` variables are being
assigned twice in ec2.py, likely due to an incorrect merge somewhere
along the line.
The IAM group modules were not receiving the `module` object, but they
use `module.fail_json()` in their exception handlers. This patch passes
through the module object so the real errors from boto are exposed,
rather than errors about "NoneType has no method `fail_json`".
Ceph Object Gateway (Ceph RGW) is an object storage interface built on top of
librados to provide applications with a RESTful gateway to Ceph Storage
Clusters:
http://docs.ceph.com/docs/master/radosgw/
This patch adds the required bits to use the RGW S3 RESTful API properly.
Signed-off-by: Javier M. Mellid <jmunhoz@igalia.com>
The default pagination is every 100 items with a maximum of 1000 from
Amazon. This properly uses the marker returned by Amazon to concatenate
the various pages from the results.
This fixes#2440.
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.
* Fixing compile time errors IRT error exception handling for Python 3.5.
This does not need to be compatible with Python2.4 b/c Boto is Python 2.6 and above.
This is to address this error:
fatal: [site]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to connect to S3: Region does not seem to be available for awsmodule boto.s3. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path"}
Commit 0dd58e9 changed the logic so an exception is thrown (by
`connect_to_aws`) before the `s3 is None` check is performed. This
changes the `None` check to a catch so the old logic can compensate.
* Add more example on how to use module ec2_tags to list tags on an instance
* Add more example on how to use module ec2_tags to list tags on an instance
'key_ids' is referenced before it is assigned, causing the module to fail with a UnboundLocalError instead of failing gracefully with a helpful error message. This very small patch moves the assignment of 'key_ids' to before the variable is referenced.
ec2_elb_lb doesn't react well to AWS API throttling errors. This
implements an exponential backoff operation around some of the AWS API
calls (with random jitter, in line with AWS recommendations) to make
this more resilient.
* Speed up AMI code by not attempting to create the AMI without checking on the name first. Also simplifies code for reporting errors from AMI creation, greatly.
* remove sys.exit
In case role policy was deleted, we did not handle at all if there
was authorization issue to do the deletion. Also add message when
role is not found and the policy is skipped.
route53 creates Record objects using `health check` and `failover`
parameters. Those parameters only became available in boto 2.28.0.
As some prominent LTS Linux releases (e.g.: Ubuntu 14.04) only ship
older boto versions (e.g.: 2.20.1 for Ubuntu 14.04), users are getting
unhelpful error messages like
TypeError: __init__() got an unexpected keyword argument 'health_check'
when running Ansible 2 against their LTS install's default boto.
We improve upon this error message by checking the boto version
beforehand.
Fixesansible/ansible#13646
OCD is making me fix the inconsistency with how None is typed. First Letter Capitalized All Over Now.
cleaning up the default object that was created for the cache_security_groups and removing checks dealing with it.
clean up space
Changing default cache_security_groups from [default] to None.
Now module will assume that if the argument is a string it is already formated as json
and will only try to convert non strings into json string.
Also removed unused 'msg' var declarations and the ifs that set them
fixes#2009
If this is not set, Ansible parses the parameter as a string.
This is fine if the parameter is not provided by the caller, but
if it is set to False or True explicitly, ec2_vol receives this as
the string 'False' or the string 'True', both of which are truthy.
Thus, without this fix, setting the parameter results in encryption
always enabled.
Brandon Davidson
Daniel Shepherd
Will Thames
kwerey
Lujeni
Justin Menga
Jonathan Davila
Sloane Hertel
Jon Hadfield
Rob
Abhijeet Kasurde
mattwwarren
Matt Clay
higginsd
Andrea Tartaglia
Andrej Svenke
Matt Babineau
Bradley Phipps
Łukasz Kostka
Takuya Sato
Brian Coca
Mark Chappell
bentaylornz
Prasad Katti
Ryan Brown
Ricardo Carrillo Cruz
Corey Christous
Nick Ball
Alex Lo
ukaj
Matt Martz
Glen Maetzig
Shaun Brady
mjfroehlich
Joshua C. Randall
Toshio Kuratomi
Marc Mercer
Robin Miller
夏恺(Kai Xia)
John R Barker
Kamil Cholewiński
Andrea Giardini
Robin Miller
Dan Vaida
Michał Masłowski
Valentyn Boginskey
mikedlr
amblina
Will Thames
Philipp Schmiedel
Troy Ready
scottb
Stepan Stipl
Tim Rupp
Willem van Ketwich
Dag Wieers
Ralph Rodkey
Eugene Krokhalev
Pau Rodriguez
Maxime Deravet
naslanidis
Etherdaemon
sebastianneubauer
Steve Kuznetsov
simplesteph
Robert Sax
Ross Williams
Fabio Alessandro Locati
Pascal Bach
David M. Lee
Jasper Lievisse Adriaanse
defunct
BRAMILLE Sébastien
howinator
Emlyn Stokes
Ted Timmons
Enis Ozgen
Kash
Rob White
Tom Paine
joernheissler
Laurent Goujon
Sean Summers
Ivan Nikolov
Chris Soyars
Gaël Lambert
Alex Sergeyev
tedder
Carlos E. Garcia
Yury V. Zaytsev
José Sabater Montes
Adam Chainz
Maarten Bezemer
Andy Freeland
James Cammarata
Fabio Alessandro Locati
Rob White
Sam Doran
Ben Tomasik
einarc
Zaius Dr
Andrew Gaffney
jctanner
mickael-ange
Eduard Iskandarov
Jamie Dyer
Charles Paul
jjshoe
Elena Washington
Pradeep
Denis Tiago
René Moser
Shawn Siefkas
mzizzi
Christopher Kotfila
Abhijit Menon-Sen
Florian Dambrine
Matt Ferrante
Kenny Woodson
Rick Mendes
Ilja Bauer
Richard Adams
Shawn Siefkas
Rick Mendes
Lyle Mantooth
Jasmine Hegman
chrisweaver
Matthew Martin
Michael Baydoun
Kai Kousa
Adrian Moisey
Hrishikesh Barua
Javier M. Mellid
Adam Butler
Toshio Kuratomi
Matt Davis
mansunkuo
Jonathan A. Sternberg
Michael Pappas
@
David Fischer
Roberto Bampi
Doug Luce
Nathan Brown
Lloyd Hazlett
Kaz Cheng
hyperized
cspollar
Bill W
deyvsh
Sumit Roy
Constantin
Dennis Conrad
Marcin
Joel Thompson
Michael Moussa
William Holroyd
Kalle Lehtonen
Christian Aistleitner
Arnaud Lachaume
Ashwanth Kumar
Taras Lipatov
Fabian Fülling
Jonathan Davila
Neil Saunders
Joel
Constantin07
Tom Bamford
Ryan-Neal Mes
Jonathan Davila
Scott Brown
Daniel Petty
Michael Baydoun
Ben Visser
Shawn Silva
techraf
Yoichi Imai
Jason Paige
Bermudez, Jaime
Henrique Rodrigues
Constantin07
autotune
Joost Cassee
krdlab
Mike Buzzetti
Andy Nelson
HAMSIK Adam
Michel Alexandre Salim
Arthur Clement
Jay Rogers
Michel Alexandre Salim