Senior Python Data Scraping Engineer (Freelance) in India at Jobgether

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Python Data Scraping Engineer (Freelance) based in India.

As a Senior Python Data Scraping Engineer, you will contribute to advanced data extraction initiatives that power innovative AI-driven applications. In this hands-on freelance role, you will design and execute robust web scraping workflows, transforming complex web data into high-quality, structured datasets. Working remotely within a flexible project-based environment, you will leverage modern scraping tools, cloud technologies, and automation frameworks to solve challenging technical problems at scale. The role offers the opportunity to work with dynamic websites, large datasets, and emerging AI technologies while maintaining the highest standards of data quality and reliability. If you are passionate about Python, automation, and building resilient data pipelines, this is an exciting opportunity to apply your expertise on impactful projects.

• Design, develop, and maintain end-to-end web scraping workflows that efficiently collect structured data from complex and dynamic websites.
• Build reliable data extraction solutions using Python and modern scraping frameworks while adapting techniques to handle JavaScript-rendered content, APIs, and evolving website structures.
• Implement scalable data collection processes through automation, batching, parallel processing, and monitoring to ensure efficient execution across large datasets.
• Validate, clean, normalize, and structure extracted data while maintaining strict quality standards and ensuring consistency across multiple data sources.
• Troubleshoot anti-bot mechanisms, optimize scraping performance, and continuously improve the reliability and resilience of extraction pipelines.
• Utilize cloud infrastructure, containerization technologies, and automation frameworks to support scalable and maintainable data processing workflows.
• Collaborate effectively within a distributed project environment, documenting processes and delivering high-quality datasets that meet project specifications.

• Minimum of 5 years of professional experience in data engineering, web scraping, automation, software development, or a related technical field.
• Strong expertise in Python for web scraping using frameworks such as BeautifulSoup, Selenium, or similar technologies.
• Proven experience extracting data from dynamic websites, JavaScript-rendered applications, REST APIs, and complex HTML structures.
• Solid understanding of data cleaning, normalization, validation, and delivery of structured datasets in formats such as CSV, JSON, or Google Sheets.
• Experience overcoming anti-bot protections, managing proxies, and maintaining stable scraping solutions for changing website architectures.
• Hands-on experience with cloud platforms such as AWS (or equivalent), Docker, and scalable automation workflows.
• Familiarity with modern AI and LLM frameworks such as LangChain, OpenRouter, or similar technologies is highly desirable.
• Strong analytical thinking, troubleshooting skills, attention to detail, and the ability to work independently with minimal supervision.
• Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related technical discipline is preferred.
• Upper-intermediate (B2) English proficiency or higher is required. A GitHub portfolio showcasing relevant projects is considered an advantage.

• Freelance, fully remote opportunity based in India.
• Flexible part-time workload of approximately 10–20 hours per week during active project phases.
• Competitive compensation of up to $30 per hour equivalent, depending on experience, contribution level, and project scope.
• Opportunity to work on innovative AI-focused projects involving advanced data extraction and automation technologies.
• Exposure to modern Python ecosystems, cloud infrastructure, containerization, and LLM-powered development tools.
• Flexible work environment with the autonomy to manage your schedule while collaborating with a global technical team.
• Opportunity to expand your expertise through challenging, real-world data engineering and web scraping projects.