Lead Data Engineer

Job Summary

We are seeking a highly skilled and motivated Lead Data Engineer to oversee the design, development, and deployment of enterprise-grade data pipelines and analytics platforms. This individual will lead a team of Data Engineers, ensuring best practices in design, performance, scalability, and governance. In addition to driving current Data Engineering efforts, the role will play a pivotal part in expanding the team’s scope towards next-generation AI technologies, where data serves as the foundation for innovation.

Responsibilities

Team leadership and Oversight

• Supervise and mentor a team of Data Engineers, setting clear goals, KPIs, and development plans.
• Review, approve, and provide feedback on data pipeline and data model designs.
  Foster a culture of continuous improvement, innovation, and accountability.

Data Engineering Excellence

• Lead the development, optimization, and deployment of reliable data pipelines to Production.
• Ensure adherence to best practices in architecture, coding standards, security, and compliance.
• Troubleshoot and resolve complex data pipeline and integration issues

Innovation & AI Expansion

• Explore and integrate emerging technologies such as Retrieval-Augmented Generation (RAG), text-to-SQL, and other AI-driven data solutions.
• Work closely with the Data/AI Lead to identify opportunities where data can fuel advanced AI applications and business insights.
• Stay updated with the latest industry trends in Data, Cloud, and AI

Collaboration & Governance

• Partner with stakeholders across business and IT to align solutions with organizational objectives.
• Ensure compliance with data governance, lineage, and quality frameworks.
• Collaborate with DevOps and Data Governance teams for smooth deployment and lifecycle management

Requirements

Education

Bachelor in Computer Science or its equivalent

Experience

• 5+ years in Data Engineering (or adjacent roles) delivering production-grade data solutions; 2+ years as a tech lead or people lead.
• Proven track record reviewing and approving data architecture/designs and promoting to Production in governed environments

Technical Expertise

• Strong experience in ETL/ELT pipeline development, cloud-based data platforms (Azure, AWS, or GCP), and data modeling.
• Proficiency in SQL, Python, Spark, Synapse/Fabric, Databricks, or equivalent.
• Knowledge of MLOps, containerization (Docker, Kubernetes) is an advantage.
• Exposure to AI technologies such as LLMs, RAG, and text-to-SQL preferred

Leadership & Growth Mindset

• Proven ability to lead, mentor, and inspire technical teams.
• Strong problem-solving, critical thinking, and decision-making abilities.
• Open to learning, adaptable to new technologies, and proactive in applying innovative solution

Certification (Preferred but not mandatory):

• Microsoft Certified: Azure Data Engineer Associate / Azure Solutions Architect
• Google Cloud Professional Data Engineer / AWS Certified Data Analytics – Specialty
• Databricks Certified Data Engineer Associate / Professional
• Certifications in AI/ML or MLOps (e.g., TensorFlow, OpenAI, AWS ML, Azure AI Engineer) are a plus