Data Engineering Lead 2387751 | Noida, Uttar Pradesh

Job description

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities:

Data Modeling & Analytics Engineering Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas

Build curated data products that support reporting, analytics, AI, and machine learning use cases

Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains

Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication

Collaborate with business stakeholders to translate requirements into analytical and reporting solutions

Databricks Platform Engineering Design and implement modern data architectures leveraging Databricks Lakehouse principles

Develop and maintain Bronze, Silver, and Gold data layers

Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows

Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed (CDF)

Delta optimization techniques Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions

Context Layer & Semantic Layer Development

Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics

Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms

Enable self-service analytics through Power BI, Tableau, and AI-driven applications

Manage semantic models and governance processes to ensure reporting consistency

Establish metric certification, lineage tracking, and data governance standards

Performance Optimization & Cost Management Analyze query execution plans and identify performance bottlenecks

Optimize Spark workloads for scalability, reliability, and cost efficiency

Implement cluster sizing, autoscaling, caching, broadcast joins, and Adaptive Query Execution (AQE) strategies

Tune Delta Lake tables through file compaction, partition optimization, and storage management

Monitor warehouse and cluster utilization to improve performance and control cloud expenses

Enhance dashboard and reporting performance through optimized data models and query design

Data Governance & Quality Management Develop and implement data quality frameworks, validation processes, and automated monitoring

Ensure compliance with enterprise security, governance, and regulatory requirements

Support metadata management, data cataloging, and lineage initiatives

Establish monitoring and alerting for data pipelines, processing failures, and data anomalies

Partner with governance teams to drive data stewardship and certification processes

Technical Leadership & Collaboration Collaborate with data scientists, analysts, engineers, product owners, and business leaders

Mentor and guide junior analytics engineers and data engineers

Conduct architecture reviews and recommend technology best practices

Drive adoption of enterprise data standards, frameworks, and governance processes

Lead technical design discussions and influence data platform strategy and roadmap

Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications:

Graduate degree or equivalent experience

Bachelor's degree in Computer Science, Information Systems, Data Engineering, Analytics, or a related field

7+ years of experience in Data Engineering, Analytics Engineering, Business Intelligence, or related disciplines

3+ years of hands-on experience with Databricks and Delta Lake technologies

Hands-on experience with Power BI, Tableau, or similar BI platforms

Experience building enterprise-scale ETL/ELT pipelines

Experience implementing semantic layers and enterprise reporting solutions

Experience with performance tuning of Spark workloads and large-scale analytics environments

Experience with cloud platforms such as Azure, AWS, or Google Cloud

Knowledge of CI/CD, DevOps practices, and version control systems such as Git

Solid understanding of Databricks Lakehouse architecture and Medallion (Bronze/Silver/Gold) design patterns

Solid understanding of data governance, metadata management, and data quality frameworks

Solid expertise in dimensional data modeling, including Star and Snowflake schema design

Advanced proficiency in SQL and Spark/PySpark development

Proven excellent problem-solving, communication, and stakeholder management skills

Preferred Qualifications:

Master's degree in Computer Science, Data Science, Analytics, or a related discipline

Databricks certifications (Data Engineer Associate, Professional, or equivalent)

Experience with Databricks Unity Catalog and enterprise data governance implementations

Experience supporting AI/ML, Generative AI, or advanced analytics use cases

Experience designing enterprise semantic models and metric layers

Healthcare, insurance, or highly regulated industry experience

Demonstrated experience leading technical teams, mentoring engineers, and driving enterprise data strategy

Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation

Key Competencies:

Data Architecture & Modeling

Analytics Engineering

Databricks & Delta Lake

Data Governance & Quality

Performance Optimization

Semantic Layer Design

Cloud Data Platforms

Technical Leadership

Stakeholder Management

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.