Data Engineer/Senior Data Engineer
Search Jobs
Search by job, company or skills
Search by job, company or skills
Role: Data Engineer/Senior Data Engineer
Location: Kuala Lumpur(Onsite)
Duration: 12 Months (Renewable) Contract
Databricks.
Implement scalable data processing and transformation logic in PySpark and SQL on Databricks (Delta
Lake / lakehouse patterns). Ingest and integrate data from diverse batch and streaming sources into
ADLS Gen2 and Delta Lake.
Build curated, documented datasets for analytics and ML (support dimensional models where
appropriate).
Implement automated data quality checks, unit/integration tests, and validation gates. Monitor pipeline
health, implement logging, alerting and dashboards; perform RCA and production incident remediation.
Optimize pipeline performance and cost: cluster sizing, job scheduling, partitioning, caching and query
tuning.
Implement and maintain CI/CD for data jobs and infrastructure (Terraform/ARM/Bicep, Azure DevOps or
GitHub Actions). Candidate knows Cantonese .
Interested candidates can connect on +6586533349 (WhatsApp chat only)
Job ID: 151799695
Skills:
Pyspark, Sql, databricks, Data Engineer
Skills:
Data Modeling, Dashboarding, Sql, ELT, Data Quality, Data Visualization, Python, Etl, SaaS Enterprise business models, Data processing performance tuning, Logging specifications, Product analytics, Anomaly detection solutions
Skills:
Github, Pyspark, Data Lineage, Data Warehouse, ELT, Data Lake, Databricks, Star Schema, Etl, Slowly Changing Dimensions, Role-Based Access Control, Attribute-Based Access Control, Snowflake Schema, Unity Catalog, Delta Tables, Fact and Dimension tables, GitHub Actions, Lakehouse, CI CD pipelines
Skills:
Adf, Pyspark, Microsoft Power Bi, Scala, Sql, ELT, Databricks, Microsoft Azure, AWS, Etl, Airflow, Data ingestion techniques, Data integration best practices, Delta Lake, Databricks Workflows
Skills:
Hadoop, Pyspark, Spark, Data Warehousing Concepts, UNIX, Python, Sql
