AWS Cloud Data Platform Engineer
Summary: The AWS Cloud Data Platform Engineer role involves developing and maintaining data pipelines and models using various cloud technologies, primarily within AWS and Databricks. The position requires expertise in tools such as Terraform, Snowflake, and Airflow, along with strong Python proficiency. The role is based in Glasgow and requires on-site presence for 2-3 days a week. The contract is set to last until December 31, 2026, with a competitive daily rate.
Key Responsibilities:
- Develop and maintain data models using dbt.
- Write complex SQL transformations to clean and structure data.
- Implement data testing and validation, including schema tests and data quality checks.
- Manage data documentation and lineage within DBT.
- Create and deploy DAGs in Airflow on Kubernetes.
- Design and develop scalable data pipelines using Databricks and Apache Spark.
- Transform and process large datasets using PySpark, SQL, and Delta Lake.
- Implement data ingestion from multiple sources, including APIs and streaming.
- Utilize AWS services such as CloudFormation, IAM, S3, and Glue.
- Work with DevOps tools like GitLab, Jenkins, and SonarQube.
Key Skills:
- Proficiency in AWS services (CloudFormation, IAM, S3, Glue, Lambda, etc.).
- Experience with Terraform and Snowflake.
- Strong knowledge of Airflow and Kubernetes.
- Expertise in Python and PySpark.
- Familiarity with Databricks and Apache Spark.
- Ability to write complex SQL queries.
- Understanding of data testing and validation processes.
- Experience with GitLab, Jenkins, and SonarQube.
Salary (Rate): £375 per day
City: Glasgow
Country: UK
Working Arrangements: hybrid
IR35 Status: inside IR35
Seniority Level: undetermined
Industry: IT
We are a Global Recruitment specialist that provides support to the clients across EMEA, APAC, US and Canada. We have an excellent job opportunity for you.
Role Title: AWS Cloud Data Platform Engineer
Location: Glasgow
Duration: 31/12/2026
Days on site: 2-3
Rate: £375 per day all inc. (PAYE through Umbrella)
Role Description:
Must have Skills:
AWS: Cloud formation, Service Catalog, IAM, S3, Glue, Lambda, Knowledge python SDK, boto, EKS, ECS, ECS, VPC, SUBNETS, NACLs.
Terraform: Modules, providers, terraform enterprise, Snowflake terraform provider knowledge.
Snowflake: Snow sight, Python SDK for snowflake, Stage, storage integration, Data marketplace, direct shares, listings, catalog linked database, Apache Iceberg, external tables, Catalog integrations, Snowpipe, external volume, Cortex ai,privatelink.
Airflow: Dag creation, operators, connections, deployment on Kubernetes, Astronomer.
DBT:
Build and maintain data models using dbt
Write complex SQL transformations to clean and structure data
Develop modular, reusable data pipelines
Implement data testing and validation (eg, schema tests, data quality checks)
Manage data documentation and lineage within DBT
SPARK: Distributed computing fundamentals, PYSPARK, GLUE JOB USING PYSPARK.
Devops: Gitlab, Jenkins, Kubernetes, SonarQube.
Databricks:
Design and develop scalable data pipelines using Databricks and Apache Spark
Build and optimize data workflows for batch and Real Time processing
Transform and process large datasets using PySpark, SQL, and Delta Lake
Implement data ingestion from multiple sources (APIs, streaming, databases)
PYTHON PROFIENCY NEEDED
If you are interested in this position and would like to learn more, please send through your CV and we will get in touch with you as soon as possible. Please note, candidates are often Shortlisted within 48 hours.
Similar Jobs
Apologies, there are currently no similar jobs available.