Azure Data Engineer with Databricks expertise (5-6 years). Responsible for managing and optimizing data pipelines, building scalable data solutions, and leveraging Azure services and Databricks for data processing and analytics. Collaborates with cross-functional teams to design and implement data architecture, ensuring data quality, security, and performance. Works on data integration, transformation, and analysis within the Azure cloud environment.
Design and implement data pipelines using Azure Data Factory, Databricks, and other Azure services.
Develop and maintain ETL processes for large datasets.
Optimize and manage Azure Databricks clusters.
Integrate data from various sources into Azure Data Lake and Azure SQL Data Warehouse.
Automate data workflows and ensure data accuracy and security.
Collaborate with data scientists and analysts for advanced analytics solutions.
Azure Data Factory, Databricks, Azure Data Lake, Azure SQL Data Warehouse, Data Integration, Data Transformation, Spark, Python, Scala, SQL, Azure Synapse Analytics, Data Modeling, Data Governance.