Data Engineer (AI) - (ETL, AWS Glue, Airflow, LLM)

Responsibilities:

• Design, build, and deploy autonomous AI agents that can independently execute multi-step data engineering tasks — ingestion, transformation, validation, and remediation — with minimal human intervention.

• Architect multi-agent systems where specialized agents collaborate to handle complex data workflows (e.g., one agent for data quality, another for schema evolution, another for pipeline optimization).

• Build agentic RAG systems that can autonomously retrieve, reason over, and act on enterprise data to answer complex business queries and generate reports.

• Develop tool-using AI agents that interact with AWS services (Glue, Redshift, Athena, S3, Lambda) to autonomously diagnose and fix pipeline failures, optimize queries, and manage infrastructure.

• Implement agent orchestration frameworks using LangGraph, CrewAI, AutoGen, or AWS Bedrock Agents to coordinate multi-step reasoning and task execution.

• Design self-healing data pipelines where AI agents detect anomalies, determine root cause, and autonomously apply corrective actions based on learned patterns.

• Build intelligent data stewards — agents that autonomously enforce data governance policies, manage metadata, classify sensitive data, and ensure IFRS17 compliance.

• Create conversational data agents that allow business stakeholders to interact with data through natural language — querying, exploring, and generating insights without writing SQL.

• Develop agent memory and knowledge management systems using vector databases, knowledge graphs, and long-term context stores to improve agent reasoning over time.

• Implement human-in-the-loop patterns for high-stakes decisions — agents that know when to escalate, seek approval, or flag uncertainty.

• Build evaluation and observability frameworks for agent performance — tracking reasoning quality, task completion rates, cost efficiency, and hallucination detection.

• Design guardrails and safety mechanisms to ensure agents operate within defined boundaries and don't take unintended destructive actions.

Key Requirements:

Bachelor's degree in computer science, AI/ML, Data Science, or related field • Minimum 5 years of experience in data engineering with ETL, data modelling, and data architecture on AWS • Minimum 3 years of hands-on experience with AWS — Glue (PySpark), S3, RDS Postgres, Redshift, Athena, Airflow • Minimum 2 years of hands-on experience building LLM-powered applications and AI agents

Strong proficiency in Python with deep knowledge of agentic AI frameworks — LangChain, LangGraph, CrewAI, AutoGen, or AWS Bedrock Agents

Experience designing and implementing multi-agent architectures with tool use, planning, and memory

Hands-on experience with prompt engineering, function calling, and structured output from LLMs (Claude, GPT-4, Llama, Titan)

Experience with RAG pipelines — embeddings, vector databases (OpenSearch, Pinecone, pgvector), chunking strategies, and retrieval optimization.

Strong understanding of agent design patterns — ReAct, Plan-and-Execute, Reflection, Tool Use, Multi-Agent Collaboration

Experience with AWS AI services — Bedrock, SageMaker, Lambda, Step Functions for agent deployment

Experience with KIRO to build data pipelines

Experience with MCP (Model Context Protocol) for connecting agents to enterprise tools and data sources

Strong SQL skills and ability to build complex queries

Rajkumar Renganathan

Quess Selection & Services, Singapore

EA Licence Number: 23C2060 Registration ID is R1983662

Disclaimer: The company is committed to ensuring the privacy and security of your information. By submitting this form, you consent to the collection, processing, and retention of the information you provide. The data collected (which may include your contact details, educational background, work experience and skills) will be used solely for the purpose of evaluating your qualifications for the position you're applying for. Your data will be stored securely and retained for the duration necessary to fulfill our hiring process. If you are not selected for the position, your data will be kept on file for a limited period in case future opportunities arise. You have the right to access, correct, or delete your data at any time by contacting us at Quess Singapore | A Leading Staffing Services Provider in Singapore (quesscorp.sg)