Tech Lead Data Engineer

Job Title: Tech Lead – Data Engineering (GCP BigQuery & Databricks)

Location: Bangalore

Experience: 8+ Years

Role Overview

We are looking for a highly hands-on Tech Lead – Data Engineering to lead the design, development, and delivery of scalable, cloud-native data platforms using GCP BigQuery and Databricks.

The role requires strong technical expertise in Python, SQL, Apache Spark, and modern data engineering practices, along with the ability to lead a team, drive technical decisions, conduct code reviews, and ensure high-quality production delivery.

The ideal candidate should have experience building end-to-end data pipelines, working with multi-cloud data platforms, and implementing modern data architecture patterns such as Lakehouse and Medallion Architecture.

Key Responsibilities

• Lead and mentor a team of Data Engineers, providing technical guidance and ensuring successful project delivery.
• Design, develop, and optimise scalable data pipelines using Python, SQL, Apache Spark, GCP BigQuery, and Databricks.
• Drive end-to-end data engineering activities, including source analysis, data ingestion, transformation, and data integration.
• Work with batch, streaming, CDC, APIs, files, and database-based ingestion methods.
• Build and optimise ETL/ELT pipelines and ingestion frameworks for large-scale data processing.
• Implement Medallion Architecture (Bronze, Silver, Gold) and Lakehouse design patterns.
• Work on interoperability between BigQuery and Databricks using Delta Lake, Apache Iceberg, Delta UniForm, BigLake, and Unity Catalog.
• Conduct code reviews, design reviews, performance tuning, and troubleshooting of complex data engineering issues.
• Establish engineering best practices covering coding standards, testing, CI/CD, documentation, data quality, and monitoring.
• Collaborate with stakeholders and clients to understand requirements, assess complexity, estimate effort, and plan delivery.
• Ensure data governance, security, access control, lineage, and cloud cost optimisation.
• Troubleshoot data pipeline failures, performance bottlenecks, and production issues.
• Drive continuous improvement in engineering processes, platform reliability, and team productivity.

Required Skills & Experience

• 8+ years of hands-on experience in Data Engineering, Data Warehousing, or Analytics Engineering.
• Proven experience in technically leading a Data Engineering team.
• Strong hands-on expertise in GCP BigQuery and Databricks.
• Advanced proficiency in Python, SQL, and Apache Spark.
• Experience designing and developing scalable ETL/ELT pipelines and data ingestion frameworks.
• Strong understanding of batch processing, streaming, CDC, APIs, and database ingestion.
• Experience with Lakehouse architecture and Medallion Architecture.
• Working knowledge of BigQuery and Databricks interoperability, Delta Lake, Apache Iceberg, Delta UniForm, BigLake, and Unity Catalog.
• Hands-on experience with Apache Airflow, dbt, Apache Kafka, and CDC.
• Good understanding of cloud infrastructure, compute, storage, networking, IAM, security, and cost optimisation.
• Experience with CI/CD, monitoring, data quality, and production troubleshooting.
• Strong technical leadership, mentoring, communication, and problem-solving skills.
• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.

What We Offer

• Hands-on exposure to AWS, Azure, and GCP.
• Support for professional certifications.
• Opportunities to develop technical and leadership skills.
• Medical insurance coverage for self and family.
• Provident Fund benefits.