community / community.aws / 5.1.0 / module / data_pipeline Create and manage AWS Datapipelines | "added in version" 1.0.0 of community.aws" Authors: Raghu Udiyar (@raags) <raghusiddarth@gmail.com>, Sloane Hertel (@s-hertel) <shertel@redhat.com>community.aws.data_pipeline (5.1.0) — module
Install with ansible-galaxy collection install community.aws:==5.1.0
collections: - name: community.aws version: 5.1.0
Create and manage AWS Datapipelines. Creation is not idempotent in AWS, so the C(uniqueId) is created by hashing the options (minus objects) given to the datapipeline.
The pipeline definition must be in the format given here U(https://docs.aws.amazon.com/datapipeline/latest/APIReference/API_PutPipelineDefinition.html#API_PutPipelineDefinition_RequestSyntax).
Operations will wait for a configurable amount of time to ensure the pipeline is in the requested state.
# Note: These examples do not set authentication details, see the AWS Guide for details. # Create pipeline - community.aws.data_pipeline: name: test-dp region: us-west-2 objects: "{{pipelineObjects}}" parameters: "{{pipelineParameters}}" values: "{{pipelineValues}}" tags: key1: val1 key2: val2 state: present
# Example populating and activating a pipeline that demonstrates two ways of providing pipeline objects - community.aws.data_pipeline: name: test-dp objects: - "id": "DefaultSchedule" "name": "Every 1 day" "fields": - "key": "period" "stringValue": "1 days" - "key": "type" "stringValue": "Schedule" - "key": "startAt" "stringValue": "FIRST_ACTIVATION_DATE_TIME" - "id": "Default" "name": "Default" "fields": [ { "key": "resourceRole", "stringValue": "my_resource_role" }, { "key": "role", "stringValue": "DataPipelineDefaultRole" }, { "key": "pipelineLogUri", "stringValue": "s3://my_s3_log.txt" }, { "key": "scheduleType", "stringValue": "cron" }, { "key": "schedule", "refValue": "DefaultSchedule" }, { "key": "failureAndRerunMode", "stringValue": "CASCADE" } ] state: active
# Activate pipeline - community.aws.data_pipeline: name: test-dp region: us-west-2 state: active
# Delete pipeline - community.aws.data_pipeline: name: test-dp region: us-west-2 state: absent
name: description: - The name of the Datapipeline to create/modify/delete. required: true type: str tags: aliases: - resource_tags default: {} description: - A dict of key:value pair(s) to add to the pipeline. type: dict state: choices: - present - absent - active - inactive default: present description: - The requested state of the pipeline. type: str region: aliases: - aws_region - ec2_region description: - The AWS region to use. - For global services such as IAM, Route53 and CloudFront, I(region) is ignored. - The C(AWS_REGION) or C(EC2_REGION) environment variables may also be used. - See the Amazon AWS documentation for more information U(http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region). - The C(ec2_region) alias has been deprecated and will be removed in a release after 2024-12-01 - Support for the C(EC2_REGION) environment variable has been deprecated and will be removed in a release after 2024-12-01. type: str values: default: [] description: - A list of parameter values (dicts) in the pipeline definition. elements: dict suboptions: id: description: The ID of the parameter value type: str stringValue: description: The field value type: str type: list objects: default: [] description: - A list of pipeline object definitions, each of which is a dict that takes the keys I(id), I(name) and I(fields). elements: dict suboptions: fields: description: - Key-value pairs that define the properties of the object. - The value is specified as a reference to another object I(refValue) or as a string value I(stringValue) but not as both. elements: dict suboptions: key: description: - The field identifier. type: str refValue: description: - The field value, expressed as the identifier of another object. - Exactly one of I(stringValue) and I(refValue) may be specified. type: str stringValue: description: - The field value. - Exactly one of I(stringValue) and I(refValue) may be specified. type: str type: list id: description: - The ID of the object. type: str name: description: - The name of the object. type: str type: list profile: aliases: - aws_profile description: - A named AWS profile to use for authentication. - See the AWS documentation for more information about named profiles U(https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html). - The C(AWS_PROFILE) environment variable may also be used. - The I(profile) option is mutually exclusive with the I(aws_access_key), I(aws_secret_key) and I(security_token) options. type: str timeout: default: 300 description: - Time in seconds to wait for the pipeline to transition to the requested state, fail otherwise. type: int access_key: aliases: - aws_access_key_id - aws_access_key - ec2_access_key description: - AWS access key ID. - See the AWS documentation for more information about access tokens U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). - The C(AWS_ACCESS_KEY_ID), C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY) environment variables may also be used in decreasing order of preference. - The I(aws_access_key) and I(profile) options are mutually exclusive. - The I(aws_access_key_id) alias was added in release 5.1.0 for consistency with the AWS botocore SDK. - The I(ec2_access_key) alias has been deprecated and will be removed in a release after 2024-12-01. - Support for the C(EC2_ACCESS_KEY) environment variable has been deprecated and will be removed in a release after 2024-12-01. type: str aws_config: description: - A dictionary to modify the botocore configuration. - Parameters can be found in the AWS documentation U(https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore.config.Config). type: dict parameters: default: [] description: - A list of parameter objects (dicts) in the pipeline definition. elements: dict suboptions: attributes: description: - A list of attributes (dicts) of the parameter object. elements: dict suboptions: key: description: The field identifier. type: str stringValue: description: The field value. type: str type: list id: description: - The ID of the parameter object. type: list secret_key: aliases: - aws_secret_access_key - aws_secret_key - ec2_secret_key description: - AWS secret access key. - See the AWS documentation for more information about access tokens U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). - The C(AWS_SECRET_ACCESS_KEY), C(AWS_SECRET_KEY), or C(EC2_SECRET_KEY) environment variables may also be used in decreasing order of preference. - The I(secret_key) and I(profile) options are mutually exclusive. - The I(aws_secret_access_key) alias was added in release 5.1.0 for consistency with the AWS botocore SDK. - The I(ec2_secret_key) alias has been deprecated and will be removed in a release after 2024-12-01. - Support for the C(EC2_SECRET_KEY) environment variable has been deprecated and will be removed in a release after 2024-12-01. type: str description: default: '' description: - An optional description for the pipeline being created. type: str endpoint_url: aliases: - ec2_url - aws_endpoint_url - s3_url description: - URL to connect to instead of the default AWS endpoints. While this can be used to connection to other AWS-compatible services the amazon.aws and community.aws collections are only tested against AWS. - The C(AWS_URL) or C(EC2_URL) environment variables may also be used, in decreasing order of preference. - The I(ec2_url) and I(s3_url) aliases have been deprecated and will be removed in a release after 2024-12-01. - Support for the C(EC2_URL) environment variable has been deprecated and will be removed in a release after 2024-12-01. type: str aws_ca_bundle: description: - The location of a CA Bundle to use when validating SSL certificates. - The C(AWS_CA_BUNDLE) environment variable may also be used. type: path session_token: aliases: - aws_session_token - security_token - aws_security_token - access_token description: - AWS STS session token for use with temporary credentials. - See the AWS documentation for more information about access tokens U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). - The C(AWS_SESSION_TOKEN), C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN) environment variables may also be used in decreasing order of preference. - The I(security_token) and I(profile) options are mutually exclusive. - Aliases I(aws_session_token) and I(session_token) were added in release 3.2.0, with the parameter being renamed from I(security_token) to I(session_token) in release 6.0.0. - The I(security_token), I(aws_security_token), and I(access_token) aliases have been deprecated and will be removed in a release after 2024-12-01. - Support for the C(EC2_SECRET_KEY) and C(AWS_SECURITY_TOKEN) environment variables has been deprecated and will be removed in a release after 2024-12-01. type: str validate_certs: default: true description: - When set to C(false), SSL certificates will not be validated for communication with the AWS APIs. - Setting I(validate_certs=false) is strongly discouraged, as an alternative, consider setting I(aws_ca_bundle) instead. type: bool debug_botocore_endpoint_logs: default: false description: - Use a C(botocore.endpoint) logger to parse the unique (rather than total) C("resource:action") API calls made during a task, outputing the set to the resource_actions key in the task results. Use the C(aws_resource_action) callback to output to total list made during a playbook. - The C(ANSIBLE_DEBUG_BOTOCORE_LOGS) environment variable may also be used. type: bool
changed: description: whether the data pipeline has been modified returned: always sample: changed: true type: bool result: description: - Contains the data pipeline data (data_pipeline) and a return message (msg). If the data pipeline exists data_pipeline will contain the keys description, name, pipeline_id, state, tags, and unique_id. If the data pipeline does not exist then data_pipeline will be an empty dict. The msg describes the status of the operation. returned: always type: dict