Try SpotterCreate and manage AWS Datapipelines
| "added in version" 1.0.0 of community.aws"
Authors: Raghu Udiyar (@raags) <raghusiddarth@gmail.com>, Sloane Hertel (@s-hertel) <shertel@redhat.com>
Install with ansible-galaxy collection install community.aws:==5.1.0
collections:
- name: community.aws
version: 5.1.0Create and manage AWS Datapipelines. Creation is not idempotent in AWS, so the C(uniqueId) is created by hashing the options (minus objects) given to the datapipeline.
The pipeline definition must be in the format given here U(https://docs.aws.amazon.com/datapipeline/latest/APIReference/API_PutPipelineDefinition.html#API_PutPipelineDefinition_RequestSyntax).
Operations will wait for a configurable amount of time to ensure the pipeline is in the requested state.
# Note: These examples do not set authentication details, see the AWS Guide for details.
# Create pipeline
- community.aws.data_pipeline:
name: test-dp
region: us-west-2
objects: "{{pipelineObjects}}"
parameters: "{{pipelineParameters}}"
values: "{{pipelineValues}}"
tags:
key1: val1
key2: val2
state: present
# Example populating and activating a pipeline that demonstrates two ways of providing pipeline objects
- community.aws.data_pipeline:
name: test-dp
objects:
- "id": "DefaultSchedule"
"name": "Every 1 day"
"fields":
- "key": "period"
"stringValue": "1 days"
- "key": "type"
"stringValue": "Schedule"
- "key": "startAt"
"stringValue": "FIRST_ACTIVATION_DATE_TIME"
- "id": "Default"
"name": "Default"
"fields": [ { "key": "resourceRole", "stringValue": "my_resource_role" },
{ "key": "role", "stringValue": "DataPipelineDefaultRole" },
{ "key": "pipelineLogUri", "stringValue": "s3://my_s3_log.txt" },
{ "key": "scheduleType", "stringValue": "cron" },
{ "key": "schedule", "refValue": "DefaultSchedule" },
{ "key": "failureAndRerunMode", "stringValue": "CASCADE" } ]
state: active
# Activate pipeline
- community.aws.data_pipeline:
name: test-dp
region: us-west-2
state: active
# Delete pipeline
- community.aws.data_pipeline:
name: test-dp
region: us-west-2
state: absent
name:
description:
- The name of the Datapipeline to create/modify/delete.
required: true
type: str
tags:
aliases:
- resource_tags
default: {}
description:
- A dict of key:value pair(s) to add to the pipeline.
type: dict
state:
choices:
- present
- absent
- active
- inactive
default: present
description:
- The requested state of the pipeline.
type: str
region:
aliases:
- aws_region
- ec2_region
description:
- The AWS region to use.
- For global services such as IAM, Route53 and CloudFront, I(region) is ignored.
- The C(AWS_REGION) or C(EC2_REGION) environment variables may also be used.
- See the Amazon AWS documentation for more information U(http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region).
- The C(ec2_region) alias has been deprecated and will be removed in a release after
2024-12-01
- Support for the C(EC2_REGION) environment variable has been deprecated and will
be removed in a release after 2024-12-01.
type: str
values:
default: []
description:
- A list of parameter values (dicts) in the pipeline definition.
elements: dict
suboptions:
id:
description: The ID of the parameter value
type: str
stringValue:
description: The field value
type: str
type: list
objects:
default: []
description:
- A list of pipeline object definitions, each of which is a dict that takes the keys
I(id), I(name) and I(fields).
elements: dict
suboptions:
fields:
description:
- Key-value pairs that define the properties of the object.
- The value is specified as a reference to another object I(refValue) or as a
string value I(stringValue) but not as both.
elements: dict
suboptions:
key:
description:
- The field identifier.
type: str
refValue:
description:
- The field value, expressed as the identifier of another object.
- Exactly one of I(stringValue) and I(refValue) may be specified.
type: str
stringValue:
description:
- The field value.
- Exactly one of I(stringValue) and I(refValue) may be specified.
type: str
type: list
id:
description:
- The ID of the object.
type: str
name:
description:
- The name of the object.
type: str
type: list
profile:
aliases:
- aws_profile
description:
- A named AWS profile to use for authentication.
- See the AWS documentation for more information about named profiles U(https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html).
- The C(AWS_PROFILE) environment variable may also be used.
- The I(profile) option is mutually exclusive with the I(aws_access_key), I(aws_secret_key)
and I(security_token) options.
type: str
timeout:
default: 300
description:
- Time in seconds to wait for the pipeline to transition to the requested state, fail
otherwise.
type: int
access_key:
aliases:
- aws_access_key_id
- aws_access_key
- ec2_access_key
description:
- AWS access key ID.
- See the AWS documentation for more information about access tokens U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
- The C(AWS_ACCESS_KEY_ID), C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY) environment variables
may also be used in decreasing order of preference.
- The I(aws_access_key) and I(profile) options are mutually exclusive.
- The I(aws_access_key_id) alias was added in release 5.1.0 for consistency with the
AWS botocore SDK.
- The I(ec2_access_key) alias has been deprecated and will be removed in a release
after 2024-12-01.
- Support for the C(EC2_ACCESS_KEY) environment variable has been deprecated and will
be removed in a release after 2024-12-01.
type: str
aws_config:
description:
- A dictionary to modify the botocore configuration.
- Parameters can be found in the AWS documentation U(https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore.config.Config).
type: dict
parameters:
default: []
description:
- A list of parameter objects (dicts) in the pipeline definition.
elements: dict
suboptions:
attributes:
description:
- A list of attributes (dicts) of the parameter object.
elements: dict
suboptions:
key:
description: The field identifier.
type: str
stringValue:
description: The field value.
type: str
type: list
id:
description:
- The ID of the parameter object.
type: list
secret_key:
aliases:
- aws_secret_access_key
- aws_secret_key
- ec2_secret_key
description:
- AWS secret access key.
- See the AWS documentation for more information about access tokens U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
- The C(AWS_SECRET_ACCESS_KEY), C(AWS_SECRET_KEY), or C(EC2_SECRET_KEY) environment
variables may also be used in decreasing order of preference.
- The I(secret_key) and I(profile) options are mutually exclusive.
- The I(aws_secret_access_key) alias was added in release 5.1.0 for consistency with
the AWS botocore SDK.
- The I(ec2_secret_key) alias has been deprecated and will be removed in a release
after 2024-12-01.
- Support for the C(EC2_SECRET_KEY) environment variable has been deprecated and will
be removed in a release after 2024-12-01.
type: str
description:
default: ''
description:
- An optional description for the pipeline being created.
type: str
endpoint_url:
aliases:
- ec2_url
- aws_endpoint_url
- s3_url
description:
- URL to connect to instead of the default AWS endpoints. While this can be used
to connection to other AWS-compatible services the amazon.aws and community.aws
collections are only tested against AWS.
- The C(AWS_URL) or C(EC2_URL) environment variables may also be used, in decreasing
order of preference.
- The I(ec2_url) and I(s3_url) aliases have been deprecated and will be removed in
a release after 2024-12-01.
- Support for the C(EC2_URL) environment variable has been deprecated and will be
removed in a release after 2024-12-01.
type: str
aws_ca_bundle:
description:
- The location of a CA Bundle to use when validating SSL certificates.
- The C(AWS_CA_BUNDLE) environment variable may also be used.
type: path
session_token:
aliases:
- aws_session_token
- security_token
- aws_security_token
- access_token
description:
- AWS STS session token for use with temporary credentials.
- See the AWS documentation for more information about access tokens U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
- The C(AWS_SESSION_TOKEN), C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN) environment
variables may also be used in decreasing order of preference.
- The I(security_token) and I(profile) options are mutually exclusive.
- Aliases I(aws_session_token) and I(session_token) were added in release 3.2.0, with
the parameter being renamed from I(security_token) to I(session_token) in release
6.0.0.
- The I(security_token), I(aws_security_token), and I(access_token) aliases have been
deprecated and will be removed in a release after 2024-12-01.
- Support for the C(EC2_SECRET_KEY) and C(AWS_SECURITY_TOKEN) environment variables
has been deprecated and will be removed in a release after 2024-12-01.
type: str
validate_certs:
default: true
description:
- When set to C(false), SSL certificates will not be validated for communication with
the AWS APIs.
- Setting I(validate_certs=false) is strongly discouraged, as an alternative, consider
setting I(aws_ca_bundle) instead.
type: bool
debug_botocore_endpoint_logs:
default: false
description:
- Use a C(botocore.endpoint) logger to parse the unique (rather than total) C("resource:action")
API calls made during a task, outputing the set to the resource_actions key in the
task results. Use the C(aws_resource_action) callback to output to total list made
during a playbook.
- The C(ANSIBLE_DEBUG_BOTOCORE_LOGS) environment variable may also be used.
type: bool
changed:
description: whether the data pipeline has been modified
returned: always
sample:
changed: true
type: bool
result:
description:
- Contains the data pipeline data (data_pipeline) and a return message (msg). If
the data pipeline exists data_pipeline will contain the keys description, name,
pipeline_id, state, tags, and unique_id. If the data pipeline does not exist then
data_pipeline will be an empty dict. The msg describes the status of the operation.
returned: always
type: dict