Key Responsibilities:
- ETL Development
- Design develop and maintain ETL pipelines using Python and PySpark
- Extract transform and load data from multiple structured and unstructured sources
- Build reusable and scalable data processing frameworks
- Ensure data quality validation and consistency
- PySpark Big Data Processing
- Develop and optimize PySpark jobs for large scale data processing
- Work with Spark DataFrames and RDDs
- Implement transformations aggregations and joins in Spark
- Optimize jobs for performance and scalability
- Python Development
- Develop backend logic and data processing scripts using Python
- Write modular reusable and efficient code
- Integrate APIs and automate workflows
- Data Management Integration
- Work with data lakes and warehouses S3 HDFS Redshift Hive
- Handle file formats like Parquet ORC JSON CSV
- Perform data cleansing enrichment and transformation
- Collaboration Support
- Work with data engineers analysts and business stakeholders
- Debug and troubleshoot ETL data pipeline issues
- Participate in Agile Scrum ceremonies
- Maintain documentation and coding standards
Technical Requirements:
- Primary skills Domain Finacle Core Functional Finacle Core WMS Grand Master Technology ETL Data Quality ETL Others Technology Machine Learning Python
Additional Responsibilities:
- Core Skills
- 2 5 years of experience in Python and ETL development
- Hands on experience with PySpark mandatory
- Strong understanding of data processing and pipelines
- Solid knowledge of SQL and database concepts
- Technical Skills
- Experience with Apache Spark ecosystem
- Good knowledge of data structures and algorithms basic to intermediate
- Familiarity with Big Data technologies Hadoop Hive
- Experience with version control Git
- Understanding of REST APIs and integrations
Preferred Skills:
Technology->AI-Data science->PYTHON,Technology->Big Data - Data Processing->PySpark,Technology->ETL & Data Quality->ETL - Others
