We are seeking an experienced Data Engineer with deep expertise in AWS-based data engineering, ETL pipeline
design, and large-scale data management.
The ideal candidate will have a strong foundation in Python, SQL, and AWS data services, and will be responsible for
architecting and maintaining data systems that power advanced analytics and machine learning initiatives.
You will work closely with data scientists, analysts, and DevOps teams to ensure robust, scalable, and secure data
infrastructure across the organization.
Key Responsibilities
• Design, build, and optimize data ingestion and transformation pipelines using AWS-native tools (Glue,
Lambda, Step Functions, EMR, etc.).
• Develop and manage data lakes and data warehouses using Amazon S3, Redshift, and Athena.
• Implement ETL/ELT workflows for structured, semi-structured, and unstructured data at scale.
• Create and maintain data models and schemas that support analytics and ML workflows.
• Work with AWS Glue Catalog, Lake Formation, and AWS DataBrew for data cataloging, cleaning,
and governance.
• Collaborate with data scientists to prepare feature-ready datasets for model training and inference.
• Implement data quality, validation, and monitoring mechanisms to ensure accuracy and reliability.
• Optimize data performance and costs through partitioning, compression, and indexing strategies.
• Automate deployment and orchestration of data workflows using AWS CodePipeline, Docker, and CI/CD
best practices.
• Ensure compliance, data security, and access control in alignment with AWS IAM and governance
standards.
Required Skills
• 3+ years of hands-on experience in Data Engineering, with at least 3 years in AWS
environments.
• Proficiency in Python (pandas, PySpark, boto3) and SQL for data transformation and orchestration.
• Expert-level knowledge of AWS data services:
o AWS Glue, AWS Lambda, Amazon S3, Amazon Redshift, Athena, Kinesis, and Step Functions.
• Strong understanding of data lake and data warehouse architectures.
• Experience designing and maintaining end-to-end data pipelines in production environments.
• Familiarity with event-driven and batch data processing workflows.
• Strong background in data modeling, schema design, and query optimization.
• Experience integrating data pipelines with ML/AI environments (e.g., SageMaker or Bedrock).
• Excellent problem-solving and debugging skills in distributed data systems.
Preferred Skills
• Experience with AWS Glue Studio, Redshift Spectrum, or Athena Federated Queries.
• Exposure to streaming data frameworks (Kinesis Data Streams, Firehose).
• Understanding of data governance, metadata management, and data lineage tracking.
• Experience using Terraform or CloudFormation for AWS infrastructure automation.
• Knowledge of Docker and containerized ETL workloads.
• Experience with Agile delivery, code review, and cross-functional collaboration.
Education
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related
fields.
• Preferred: Certification as an AWS Certified Data Engineer – Associate / Professional, or AWS Certified
Solutions Architect.
Python, SQL, AI Agents, AWS Glue, Lambda, S3, Redshift, Athena, Kinesis, Step Functions, PySpark, etc.