Python or JS Developer Needed for Adzuna Job Scraper Improvement (Fetch All Job Data, JSON/HTML Parsing

₹1000-1001 INR

Open

Posted

about 1 hour ago

•

Ends in 6 days

₹1000-1001 INR

Paid on delivery

I am looking for a skilled Python or JS developer with expertise in web scraping, Selenium, and HTML parsing to enhance and complete a job scraping tool for the Adzuna job portal. Project Goals: code must fetch all job listings from Adzuna for a given search query URL (e.g., [login to view URL]). Handle cases where JSON-LD data is missing by falling back to HTML parsing to extract all relevant details. Current Issues: my code scrapes some job data (titles, descriptions, company names, etc.), but many fields remain blank. Not all job listings are fetched; the script currently stops at ~157 jobs out of ~410 listed. Fallback HTML parsing is insufficient, leaving many fields incomplete. Requirements: Comprehensive Data Extraction: Fetch all job listings from Adzuna for the provided URL. Extract key fields, including: Job Title Company Name Description (cleaned and free of HTML tags) Location Posting Date ( Ensure no blank fields Pagination Support: Ensure the scraper navigates through all pages of the search results, capturing every job listing. Fallback Mechanism: If JSON-LD is unavailable or malformed, extract data reliably using HTML selectors for titles, company names, descriptions, etc. Error Handling: Implement robust error handling for issues like missing elements, timeouts, and unexpected HTML structures. Log skipped or problematic jobs for review. Output: Save the data to a CSV file with all columns accurately populated. Optimization: Improve the scraper's speed and memory efficiency, ensuring compatibility with the existing Selenium + Chrome WebDriver setup. Deliverables: A Python script that fetches all job listings from Adzuna for the given URL. Accurate and complete CSV output containing all fetched job data. Detailed comments in the code for clarity. Skills Required: Python or JS Web Scraping (Selenium, BeautifulSoup, etc.) JSON-LD and HTML Parsing CSV Data Handling Debugging and Error Handling

Project ID: 38977005

About the project

13 proposals

Open for bidding

Remote project

Active 45 mins ago

Place your bid

Bid amount

₹

INR

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

13 freelancers are bidding on average ₹17,155 INR for this job

@jinufeb14

With a distinguished record of 10+ years in software engineering, ranking among the top 0.03% on Freelancer, my name is Jinu and I'm here to optimize your Adzuna job scraper proficiently. Offering expertise in Python, JavaScript, Web Scraping (including BeautifulSoup & Selenium), JSON/HTML Parsing, CSV handling, Debugging and Error Handling; I believe I have all it needs for this project. The current shortcomings faced by the tool are effortlessly dealt with in my skill set. Whether it's fetching all job data for a given query URL or improvising for missing JSON-LD data by falling back to HTML parsing, I'm adept at handling every aspect of these requirements effectively. Equipped with experience in pagination support, fallback mechanisms, error handling, and optimizing the scraper's speed and memory efficiency (especially for your existing Selenium + Chrome WebDriver setup), rest assured that every possible measure will be taken to rectify the issues.

₹1,001 INR in 7 days

4.9

(32 reviews)

5.3

@navinfeb15

Hello there, I specialize in Python and JavaScript, focusing strongly on web scraping using Selenium and HTML parsing with BeautifulSoup. I've tackled similar challenges, ensuring comprehensive data extraction and perfecting fallback mechanisms when JSONLD is missing. For your Adzuna job portal tool, I propose: 1. Comprehensive Data Extraction: I'll ensure the code fetches all job listings by enhancing the existing HTML parsing and filling in missing fields like location and posting date, ensuring every field is populated. 2. Pagination Support: I'll adjust the script to navigate through all pages effectively, capturing every listing without stopping prematurely. 3. Enhanced Fallback Mechanism: Implement improved HTML selectors to fill gaps when JSONLD data is unavailable, ensuring complete and accurate information extraction. 4. Robust Error Handling: I'll introduce advanced error catching strategies for timeouts, and unexpected HTML, and maintain logs for skipped jobs for easy review. 5. Output Optimization: Ensure the data is saved to a CSV with all details accurately filled, optimizing both speed and memory use within the existing Selenium and Chrome WebDriver framework. I have previously worked on projects requiring such detailed extraction and processing, like scraping job portals and ecommerce sites, which align well with your project's goals. I'm eager to refine the tool, ensuring it operates flawlessly and meets all your deliverables, including a fullycommented Python script. Could you please specify the desired frequency of scraping and any specific fields of primary importance? Looking forward to making your project a success!

₹1,010 INR in 1 day

5.0

(18 reviews)

4.6

@ShekharTyagi345

With 8 years of experience in web development and a strong background in Python and JavaScript, I am confident that I am the best fit to enhance and complete the job scraping tool for the Adzuna job portal. How I will be completing this project: I will start by analyzing the existing code to understand the current issues and requirements. Then, I will enhance the scraper to fetch all job listings from Adzuna, ensuring comprehensive data extraction and handling pagination support. I will also implement a fallback mechanism for HTML parsing and robust error handling for unexpected scenarios. What tech stack I will be following: I will be using Python for scripting, Selenium for web scraping, BeautifulSoup for HTML parsing, and CSV for data handling. I will leverage my expertise in JSON-LD and HTML parsing to extract key fields accurately and optimize the scraper for speed and memory efficiency. I have worked on similar solutions in the past, where I have successfully tackled complex web scraping challenges and delivered accurate and complete data outputs. My experience and skills make me well-equipped to handle the requirements of this project. By following a structured approach and leveraging the right tech stack, I am confident in delivering a Python script that fetches all job listings from Adzuna, generates an accurate CSV output, and includes detailed comments for clarity. Overall, I am excited about the opportunity to

₹1,000 INR in 7 days

5.0

(8 reviews)

2.5

@namitajain118

I have 8 years of experience in web development and data scraping, making me the perfect fit for this project. I have the relevant skills in Python, web scraping, Selenium, and HTML parsing to enhance and complete the job scraping tool for the Adzuna job portal. How I will be completing this project: - I will enhance the existing code to fetch all job listings from Adzuna for the given search query URL. - I will handle cases where JSON-LD data is missing by falling back to HTML parsing to extract all relevant details. - I will ensure comprehensive data extraction by fetching key fields such as job title, company name, description, location, and posting date without any blank fields. - I will implement pagination support to navigate through all search result pages and capture every job listing. - I will create a fallback mechanism to reliably extract data using HTML selectors if JSON-LD is unavailable or malformed. - I will implement robust error handling for missing elements, timeouts, and unexpected HTML structures and log skipped or problematic jobs for review. - I will optimize the scraper's speed and memory efficiency to work seamlessly with the existing Selenium + Chrome WebDriver setup. What tech stack I will be following: - Python for scripting - Web Scraping tools like Selenium and BeautifulSoup - JSON-LD and HTML parsing techniques - CSV data handling for accurate output - Debugging and error handling for a smooth e

₹1,000 INR in 7 days

5.0

(6 reviews)

2.7

@swifttecnologia

Hello! How are you? I hope so. My name is Kairo, I am a back-end developer specializing in high-quality systems, apis, automations, I have about 10 years of experience in the area, and I am interested in your project. I look forward to hearing from you so we can discuss more details.

₹1,001 INR in 1 day

5.0

(3 reviews)

2.3

@sabokadiriba

Hello, I am a skilled Python and JavaScript developer with extensive experience in web scraping, including using tools like Selenium, BeautifulSoup, and HTML parsing. I am confident I can help enhance your Adzuna job scraping tool to fetch all job listings and complete missing fields. I will ensure comprehensive data extraction, implement a fallback mechanism for missing JSON-LD data, and improve the speed and efficiency of the scraper. My approach will also include robust error handling, pagination support, and accurate CSV data output. I look forward to discussing how I can help you resolve the current issues and deliver a reliable solution for scraping all job listings. Best regards, Saboka Diriba

₹1,000 INR in 7 days

5.0

(2 reviews)

0.9

@aminsyed46

Hi Karthik B., We are an expert team which have many years of experience on Job Skills We are an expert team which have many years of experience on JavaScript, Python, Web Scraping, BeautifulSoup, Selenium Lets connect in chat so that We discuss further. Thank You

₹1,000 INR in 7 days

0.0

(0 reviews)

0.0

@richardl846

Hi Client, I understand you're looking to improve your Adzuna job scraper. With my experience in Python web scraping and Selenium, I can help you tackle the issues you're facing. Here's how I can assist: Complete Data Extraction: I've developed scrapers that reliably fetch all job listings, ensuring no data is missed. Fallback Parsing: When JSON-LD data is missing or malformed, I implement HTML parsing to extract all relevant details. Pagination Handling: I ensure scrapers navigate through all pages, capturing every job listing. Error Handling: I incorporate robust error handling to manage missing elements, timeouts, and unexpected HTML structures. Optimized Output: I save the data to a CSV file with all columns accurately populated. Let's discuss your project in more detail. I can provide a tailored solution to enhance your scraper's performance and reliability. Best regards, Richard

₹1,000 INR in 2 days

0.0

(0 reviews)

0.0

@amannn00007

I will enhance your job scraping tool to reliably fetch all job listings from Adzuna, handle pagination, improve data extraction (using both JSON-LD and HTML parsing), and ensure no blank fields, with robust error handling and optimized performance for Selenium and Chrome WebDriver. The final output will be a CSV with accurate job data and comprehensive code comments.

₹1,001 INR in 7 days

0.0

(0 reviews)

0.0

@imadrahhali

Hello, I am excited to assist with enhancing your Adzuna job scraper to ensure comprehensive, accurate, and efficient data extraction. With extensive experience in Python, Selenium, BeautifulSoup, and data parsing, I can address the current challenges and deliver a robust solution. What I’ll Deliver: - Comprehensive Data Extraction: A script capable of fetching all job listings from Adzuna, including titles, company names, descriptions (cleaned), locations, and posting dates. - Pagination Support: Seamless navigation through all pages to capture every job listing. - Fallback Mechanism: Reliable HTML parsing for scenarios where JSON-LD data is missing or malformed. - Error Handling: Robust mechanisms to handle missing elements, timeouts, and unexpected HTML structures, with logging for skipped/problematic jobs. - Optimized Output: A well-structured CSV file with complete and accurate data. - Code Comments: Clear documentation to ensure maintainability. Skills I Bring: - Proficiency in Python and web scraping tools like Selenium and BeautifulSoup. - Expertise in handling JSON-LD and HTML parsing. - Strong debugging and optimization skills for performance and reliability. Let’s collaborate to enhance the tool’s capabilities and ensure it meets all your requirements. I’m confident in delivering quality results within your timeline. Best regards

₹1,000 INR in 7 days