We are seeking a Senior Data Engineer with experience building and contributing to the design of database systems, both normalized transactional systems and dimensional reporting systems. Strong experience with SQL Server as a database engine, as well as Microsoft and Databricks technology experience implementing Big Data with Advanced Analytics solutions. Successful candidates will have experience and skill in providing solutions for storing, retrieving, transforming and aggregating data to support line of business applications as well as reporting systems. The Data Engineer will work with customers to deliver solutions utilizing strong business, technical and data modelling skills. This position will represent our organization's approach to data visualization and information delivery solutions and as such must demonstrate proficiency with Power BI and Azure Data Services.
Key Responsibilities:
Design and implement general architecture for complex data systems.
Translate business requirements into functional and technical specifications.
Design and implement lakehouse architecture.
Develop and manage cloud-based data architecture and reporting solutions.
Apply data modelling principles for relational and dimensional data structures.
Design Data Warehouses following established principles (e.g., Kimball, Inmon).
Create and manage source-to-target mappings for ETL/ELT processes.
Mentor junior engineers and contribute to architectural decisions and code reviews.
Minimum Qualifications:
Bachelor’s Degree in Computer Science, Computer Engineering, MIS, or related field.
6+ years of experience with Microsoft SQL Server and strong proficiency in T- SQL, SQL performance tuning (Indexing, Structure, Query Optimization).
6+ years of experience in Microsoft data platform development and implementation.
3+ years of experience in consulting, with a focus on analytics and data solutions.
3+ years of experience with Databricks, including Unity Catalog, Databricks SQL, Workflows, and Delta Sharing.
3+ years of experience with Power BI or other competitive technologies.
Proficiency in Python and Apache Spark.
Develop and manage Databricks notebooks for data transformation, exploration, and model deployment.
Expertise in Microsoft Azure services, including Azure SQL, Azure Data Factory (ADF), Azure Data Warehouse (Synapse Analytics), Azure Data Lake, and Stream Analytics.
Preferred Qualifications:
Experience with Microsoft Fabric.
Familiarity with CI/CD pipelines and infrastructure-as-code tools like Terraform or Azure Resource Manager (ARM).
Knowledge of taxonomies, metadata management, and master data management.
Familiarity with data stewardship, ownership, and data quality management.
Expertise in Big Data technologies and tools:
Big Data Platforms: HDFS, MapReduce, Pig, Hive.
General DBMS experience with Oracle, DB2, MySQL, etc.
NoSQL databases such as HBase, Cassandra, DataStax, MongoDB, CouchDB, etc.
Experience with non-Microsoft reporting and BI tools, such as Qlik, Cognos, MicroStrategy, Tableau, etc.
Additional Skills & Experience to Prioritize
Strong hands-on experience in Enterprise Data Modeling.
Experience designing Canonical Data Models (Common Data Models) that unify data from multiple source systems.
Experience building or contributing to Semantic Layers, Business Data Models, Ontology, or similar enterprise-wide data abstraction frameworks.
Ability to standardize data coming from multiple applications with different schemas, naming conventions, data types, and structures.
Experience working with multiple business stakeholders to understand business processes and define a common data model across the organization.
Exposure to Data Modeling tools such as Erwin is highly preferred.
Candidates who have implemented data modeling or semantic layer solutions natively in Databricks are also a strong fit.
Good understanding of data governance, metadata management, and business terminology mapping will be an added advantage.
Revised Screening Focus
While Databricks remains a mandatory skill, interviews and profile screening should now place greater emphasis on:
Practical experience in data modeling, rather than only data engineering or ETL development.
How candidates approached designing a common/canonical data model across multiple systems.
Experience creating semantic layers or translating technical data into business-friendly models.
Ability to bridge the gap between business requirements and source system data structures.
Strong communication, stakeholder management, and consensus-building skills, as this role involves working with multiple teams across the organization.
Candidate Profile We Are Looking For
The ideal candidate should be someone who has worked beyond traditional data engineering and has experience defining how enterprise data should be structured and represented across multiple systems. This role is becoming more focused on business-centric data modeling and architecture, while retaining a solid foundation in Databricks and modern data platforms.
Microsoft SQL Server, T-SQL, SQL Performance Tuning, Query Optimization, Indexing, Database Design, Data Engineering, Data Architecture, Data Modeling, Relational Data Modeling, Dimensional Data Modeling, Data Warehousing, Kimball Methodology, Inmon Methodology, ETL, ELT, Source-To-Target Mapping, Lakehouse Architecture, Big Data, Advanced Analytics, Databricks, Unity Catalog, Databricks SQL, Databricks Workflows, Delta Sharing, Databricks Notebooks, Python, Apache Spark, Power BI, Microsoft Azure, Azure SQL, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Stream Analytics, Cloud Data Architecture, Reporting Solutions, Business Intelligence, Microsoft Fabric, CI/CD, Terraform, Azure Resource Manager, Metadata Management, Master Data Management, Data Stewardship, Data Quality Management, HDFS, MapReduce, Pig, Hive, Oracle, DB2, MySQL, HBase, Cassandra, DataStax, MongoDB, CouchDB, Qlik, Cognos, MicroStrategy, Tableau, Consulting, Technical Specification, Functional Specification, Code Review, Mentoring