Sagemaker Operator Async¶
SageMakerProcessingOperatorAsync starts a processing job on AWS Sagemaker and polls for the status asynchronously.
A processing job is used to analyze data and to run your data processing workloads, such as feature
engineering, data validation, model evaluation, and model interpretation
SageMakerProcessingOperatorAsync
.
preprocess_raw_data = SageMakerProcessingOperatorAsync(
task_id="preprocess_raw_data",
aws_conn_id=SAGEMAKER_CONN_ID,
config=test_setup["processing_config"],
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/amazon/aws/example_dags/example_sagemaker.py
SageMakerTransformOperatorAsync starts a transform job and polls for the status asynchronously. A transform job uses a
trained model to get inferences on a dataset and saves these results to an Amazon S3 location that you specify.
SageMakerTransformOperatorAsync
.
test_model = SageMakerTransformOperatorAsync(
task_id="test_model",
aws_conn_id=SAGEMAKER_CONN_ID,
config=test_setup["transform_config"],
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/amazon/aws/example_dags/example_sagemaker.py
SageMakerTrainingOperatorAsync starts a model training job and polls for the status asynchronously.
After training completes, Amazon SageMaker saves the resulting model artifacts
to an Amazon S3 location that you specify.
SageMakerTransformOperatorAsync
.
train_model = SageMakerTrainingOperatorAsync(
task_id="train_model",
aws_conn_id=SAGEMAKER_CONN_ID,
print_log=False,
config=test_setup["training_config"],
)
# https://github.com/astronomer/astronomer-providers/tree/main/astronomer/providers/amazon/aws/example_dags/example_sagemaker.py