About The Team

The Marketplace Intelligence and Data team's mission is to build sustainable and efficient data and intelligence products to facilitate Shopee's business development. The team is responsible for Shopee e-commerce data warehouse construction, merchant and operation data product construction, all-link traffic data, product algorithms, including product release, control, information optimization, SPU library and its comparison business, marketing algorithms, including Merchandising, Product Selection, Recommendation Algorithm, Evaluation Algorithms, User Profiling, and in addition, basic AI capabilities, such as Machine Translation, Speech Algorithm, Image Algorithm, and Real-person Authentication.

As a Data Annotation Quality Analyst at Shopee, you will play a critical role in ensuring the accuracy and quality of annotated data used to enhance our AI models. You will be responsible for developing and implementing annotation quality pipelines, designing and introducing Standard Operating Procedures (SOPs) to local annotators, and evaluating the overall quality of annotations generated by our teams by leveraging data analysis. The annotated data includes not only training data but also LLM model evaluation data used for benchmarking and performance assessment. You will also leverage the latest AI technologies such as LLMs and AI agent APIs to optimize the overall efficiency of the whole process.

Job Description

Annotation Quality Pipeline Development:

Collaborate with cross-functional teams to create and implement annotation quality pipelines that ensure consistent and accurate data labeling.
Develop and refine quality control processes to identify and rectify issues in annotated datasets.

SOP Design and Documentation:

Design, document, and communicate clear Standard Operating Procedures (SOPs) for data annotation teams.
Train and educate annotation teams on SOPs to ensure adherence to quality standards.

Quality Evaluation:

Conduct regular quality assessments of annotated data to identify discrepancies, inconsistencies, or errors.
Collaborate with annotation teams to provide feedback and guidance for improving annotation quality.

Requirements

Currently pursuing a bachelor's degree or higher in a related field (e.g., Information Systems , Data Science, Statistics, Data Analytics etc).
Knowledge of data labeling techniques.
Knowledge of SQL, Python, Data Analysis and Machine learning is a plus.
Ability to leverage AI tools to enhance day-to-day tasks and improve efficiency.
Strong problem-solving skills with an attention to detail.

Data Quality Assurance Engineer Intern, Marketplace Intelligence and Data