We are looking for an experienced Data Engineer with 6–10 years of hands-on expertise in building scalable data analytics, data warehousing, and cloud-based data solutions. The ideal candidate will be proficient in Spark (Scala/Python), Azure Data Factory, Azure Databricks, and modern data engineering practices. This role is based in Hyderabad (Hybrid) and requires strong experience across Azure services, large-scale data processing, ETL frameworks, and end-to-end data pipeline development.
Design, develop, and optimize large-scale data analytics and data warehouse solutions using Azure Data Factory and Azure Databricks.
Build and maintain data lake architectures to process multi-format, high-volume datasets.
Translate solution designs into fully functional implementation packages on the Azure platform.
Manage distributed data processing workflows and integrate solutions with service-oriented architectures.
Work extensively on Azure Storage Gen2, Azure DevOps pipelines, and Azure analytics services.
Develop high-performance data pipelines using Spark (Scala/Python) and T-SQL.
Implement ETL workflows using tools such as Informatica, SSIS, or Talend.
Collaborate in Agile SDLC environments, ensuring cloud deployments follow best practices.
Work with streaming technologies (Kafka) for real-time data ingestion (good to have).
Ensure adherence to best practices around cloud, DevOps, testing, and modern CI/CD processes.
6–10 years of hands-on experience in data engineering, data analytics, and cloud-based data solutions.
Strong proficiency in Azure Data Factory (ADF) and Azure Databricks is mandatory.
Deep understanding of large-scale data distribution, data lake architectures, and multi-format processing.
Minimum 5 years of experience working with Azure cloud stack and Azure DevOps.
Proficient coding experience in Spark (Scala/Python) and T-SQL.
Solid knowledge of Azure Storage Gen2, Azure SQL, Azure Function App, Logic App, and related analytics services.
Prior ETL development experience using Informatica, SSIS, Talend, or similar tools.
Experience in cloud deployments (preferably Microsoft Azure) within Agile environments.
Good to have familiarity with Kafka streaming and Azure infrastructure for real-time data ingestion.
Strong problem-solving skills, ability to work in hybrid environments, and proven project execution experience.
Spark, Scala, Python, Azure, Azure Data Factory, Azure Databricks, Azure DevOps, Azure Storage Gen2, Azure SQL, Azure Function App, Logic App, T-SQL, Informatica, SSIS, Talend, Kafka, Cloud Deployments, Data Engineering, Data Warehousing, ETL, Data Lake, Agile SDLC