Log In
Sign Up
Description:
We are seeking an experienced **Python – PySpark Developer** with **Jupyter Notebook** expertise to join our dynamic team in **Dubai, UAE**. The ideal candidate should have at least 10 years of experience in software development, with a strong focus on big data technologies and Python programming.
### **Key Responsibilities**:
Design, develop, and maintain large-scale **ETL pipelines** using **PySpark** and **Python**.
Develop, test, and maintain efficient, reusable, and reliable code in **Jupyter Notebooks** for data analysis and visualization.
Collaborate with data scientists and analysts to support their data processing and analysis needs.
Optimize performance of PySpark jobs and workflows to handle large datasets.
Ensure data accuracy and integrity by implementing and maintaining robust data validation procedures.
Write well-designed, efficient, and testable code following best practices.
Work with cross-functional teams to understand and translate business requirements into technical solutions.
Perform troubleshooting, debugging, and performance tuning of applications.
Stay updated with the latest trends and technologies in big data and cloud environments.
### **Key Requirements**:
Minimum 10+ years of experience in software development, with expertise in **Python** and **PySpark**.
Extensive experience with **Jupyter Notebooks** for data analysis and development.
Proven experience in developing and optimizing large-scale **data processing pipelines**.
Strong knowledge of **distributed computing** concepts and experience with **Apache Spark**.
Familiarity with **cloud platforms** like **AWS**, **Azure**, or **Google Cloud**.
Proficiency in working with **Hadoop**, **HDFS**, and other big data technologies is a plus.
Strong problem-solving skills with a proactive and results-driven approach.
Excellent communication skills and the ability to work in a fast-paced environment.
