Web crawling and scraping jobs jobs
...trying to learn things online by gathering the CC of Videos into TXT files and read it instead. I need a way to automate the gathering of the text files with the close captioning. Here is a video explaining: Job Posting: YouTube Caption Extraction Bot Developer Position Overview: Looking for a Python developer to create a YouTube channel caption extraction bot with specific requirements for handling large-scale video processing. Project Requirements: Create a bot that processes YouTube channel video pages (e.g., @TheCookingFoodie/videos) Core Functionalities: Process all videos from a channel's video page Extract and save: Video titles Video URLs Closed captions (CC) Save data in text files
I need a skilled Python developer to create a web scraper for me. The scraper should be able to extract structured data, like tables, from various e-commerce websites. The project requires a deep understanding of web scraping techniques and best practices in Python.
I'm looking for a Python-based email validation checker that can efficiently check millions of email addresses against...a Python-based email validation checker that can efficiently check millions of email addresses against the Kucoin sign-up page. This tool needs to: - Utilize proxies: It should be able to run with proxies and handle the load of millions of checks with a budget of $200 worth of proxies. - Store results: The checker should store the results of the email validation in a file. The output format for the results file should be TXT. Ideal candidates for this project would be those with extensive experience in Python, web scraping, and proxy handling. A deep understanding of email validation and experience working with cryptocurrency exc...
Please Sign Up or Login to see details.
I need an “AI” that can generate realistic images. The data scrapper should be capable of scraping the entire internet to gather unique details and use random generation to create distinct images. I want the AI to output a variety of image types, including portraits, landscapes, and abstract art. Ideal skills for this job include: - Experience in machine learning - Proficiency in data scraping - Knowledge in image generation - Familiarity with different art styles - Skills in random generation algorithms
I'm looking for a service to add members from one Telegram group to another (scraping usernames from public groups and adding them to my group). 1) What is the price for 1k members? 2) What is the timeline to finish? - NO smm panel/bots... (I will provide you public groups to take members from and add to my group) - This will be an ongoing project, as I plan to place many additional orders later. - Only professionals with prior experience in this type of work !
...profiles on X (formerly Twitter) for posts related to memecoin launches and delivers real-time alerts via Telegram. The bot must be fast, efficient, and built using free or open-source tools, avoiding paid APIs or developer tools. Responsibilities: • Profile Monitoring on X: • Track a predefined list of X accounts, including influencers, crypto enthusiasts, and meme-focused profiles. • Focus on posts mentioning memecoin launches, using hashtags like #memecoin and phrases such as “launch,” “new coin,” or other relevant crypto-related terms. • Real-Time Detection: • Implement functionality to scan posts in real-time to detect mentions of new tokens, key hashtags, or phrases signaling a memecoin launch. &bu...
Development of a Cost-Efficient Microservice for Color-Based Apparel Search and Price Comparison Project Overview: We are developing a Minimum Viable Product (MVP) website focused on fashion, styling, and makeup. The platform will allow fashion artists to create profiles and showcase their work while offering users practical tools to enhance their styling experience. As part of this MVP, we are seeking an experienced freelancer to develop a lightweight and cost-effective microservice that enables users to search for apparel by color and compare prices and availability across major online retailers. The freelancer will develop a simple yet functional microservice with the following features: Color-Based Search: Users can search for apparel by s...
extract data from web ejar SA, and put it into excel sheet
I need an app to continually monitor when a driving test becomes available and then sends an SMS and/or email with the details of the earliest available appointment, if there is one The app will need to be able to login to the DVSA website (I will give my credentials) which has a visual CAPTCHA security screening This is the URL: This link is to the rebooking site for an existing appointment where you can look for an appointment at a date earlier than the existing booking so it will need to know that data too (to be provided). The app doesn't need to make the rebooking, only to alert when one an earlier appointment is available. It needs to monitor for multiple users ar a time
I'm in need of a recruitment agency that can assist in the mass hiring of mid-level professionals pr...agency that can assist in the mass hiring of mid-level professionals primarily in the Information Technology, Healthcare, and Sales sectors. Key Responsibilities: - Posting jobs in valid and verified indeed accounts for US. - Streamline the hiring process to improve efficiency -Candidate database creation Challenges: - The primary obstacle I'm facing is the difficulty in finding qualified candidates with valid Indeed Employer Accounts Ideal Skills: - Extensive network and resources in the Information Technology, Healthcare, and Sales industries - Proven track record in sourcing mid-level professionals - Ability to understand and mee...
I'm in need of an expert who can scrape live video data from video streaming services. The data should then be converted into an API, with the format being JSON. Ideal Skills: - Proficiency in web scraping - Experience in API development - Understanding of JSON format Your primary task will be to collect data from live videos on various streaming platforms and turn it into a usable API. Please note that the specific types of data to be collected have not been specified yet. Prior experience working with video data is a plus. Please bid if you have the necessary skills and experience. Looking forward to your proposals.
...database of key contacts and their email addresses from the company lists provided. The work involves creating a clean, organized spreadsheet with accurate information about each company’s key personnel and their contact details. Scope of Work: You will receive a names of few companies + 1 link containing lists of other companies. For each company, gather the following information (where available): Company Name Key People (CEO, Founder, Marketing Head, Sales Head, or equivalent) Job Titles Contact Details (email addresses, phone numbers) LinkedIn Profile URLs (if available) Organize the data into a structured Google Sheet or Excel file. Requirements: - Experience in data scraping, web research, or lead generation. - Attention to detail and...
...configured, and user flows are defined and partially tested. Now, we need your expertise to connect the pieces and bring everything together into a fully functional MVP. This tasks will focus on integrating the frontend with the database, implementing role-based authentication, and ensuring seamless workflows for all user roles (Guest, Registered, Admin). What’s Done So Far: We’ve built a solid foundation: The database is live in Supabase with role management, data inputs, and calculations fully set up. Key APIs like ScrapingBee (for data extraction) and FRED (for rate data) are ready and configured. The frontend uses React.js with TailwindCSS, DaisyUI, and ShadCN for layouts and visuals. We also ha...
Here’s a detailed design for the Pathways to America website, highlighting functionality, user flow, and design elements: Website Design for Pathways to America 1. Homepage Header • Logo: Pathways to America (modern and inclusive design with the tagline “Paving the Way to US Citizenship”). • Navigation Bar: • Home • About Us • Job Categories • How It Works • Apply Now • Donate • Contact Us Hero Section • Background Image: A diverse group of professionals in various industries (healthcare workers, engineers, construction workers, etc.). • Tagline: “Achieve Your American Dream. Apply for Citizenship Today!” • Call-to-Action Buttons: • “Apply Now” (scrolls ...
I'm seeking a web designer to modernize the design aesthetics of my existing website and update the logo. Key tasks include: - Revamping the layout to a clean, minimalist style with improved typography using sans-serif fonts. - Simplifying the menu for easier access to key sections such as "Jobs" and "Resources". - Creating a visual hierarchy with larger headings and clear subheadings. - Modernizing the color palette and incorporating high-quality images and icons to break up text. - Ensuring the site is fully responsive and optimized for mobile. - Ideally, a QR code will be generated for easy site access. The ideal freelancer for this job should have: - Extensive experience in web design with a focu...
create some dashboard heat maps and other items to show repairs on propertyies by different job types and appliances I want someone to create me a dashboard where the user can select from a drop down the heat map type and they heat map displays the data ie if the user selects jobs as the filter it should show map of certain area and then show heat maps of jobs that have been done in a certain area if the user selects job type as the filter it should show map of certain area and then show heat maps of job type that have been done in a certain area ie plumbing if the user selects Asset as the filter it should show map of certain area and then show heat maps of asset that have failed in a certain area ie plumbing This needs to be...
...website that provides foreclosure and HUD home listings. The website should offer a seamless user experience with a $X week-long trial, followed by an ongoing monthly subscription fee. Model Site: Core Features & Requirements: -User Registration & Subscription Management: -Seamless user sign-up with email verification -Trial period implementation ($1 for 7 days) -Automated monthly billing after trial ends -User dashboard to manage subscriptions -Subscription plan management (upgrade/downgrade/cancel) -Integration with popular payment gateways (Most likely ) Foreclosure & HUD Home Listings: -Intuitive search and filtering options -Map integration for location-based searches -Detailed property pages with images, descriptions, and pricing -Automated da...
I'm seeking a professional to scrape my WordPress website for 1100 entries related to our different offices. The data needs to be exported into an Excel file, without any s...professional to scrape my WordPress website for 1100 entries related to our different offices. The data needs to be exported into an Excel file, without any specific filters or sorting. In addition to data scraping, I need a thorough code review of my WordPress site. This should encompass: - Documentation and review of custom code - Optimization where necessary - Plugin review for updates - Recommendations for optimization This project requires a deep understanding of PHP, as well as experience with WordPress and data scraping. The ideal freelancer will have a strong background in w...
...that uses GPT API running on a local machine using local GPUs. The front end is single screen PyQT5. We now need to scale this to a web service running on , converting the PyQT5 UI to Gradio for the UI/orchestration layer so we can run batches on multiple instances and easily scale GPU and CPU resources as needed. We've scoped this initial deployment at about 80 hours for a developer with these proven skills (modal and gradio), with future work to continue based on performance. Our app batch uses batch downloaded docs. via API, parses and uses LLM to generate a docx report and set of .csv files. Analyzed content is based in the biosciences area and in future phase we'll be adding functionality to analyze bio-sequences using Bla...
We have 11,000 manufacturer SKU's that we would like to do price comparison across 10 different ecommerce websites. 100% accuracy is a must. 5 star freelancers only apply.
...seeking an experienced and detail-oriented developer to build an automation tool that will facilitate the extraction of real estate data from multiple platforms, including Zillow, Redfin, StreetEasy, and others. This tool will allow users to download property listings and seamlessly extract additional zoning and building-related data from Zola NYC and NYC Open Portal, all within an easy-to-use Excel interface. Responsibilities: Download Listings: Create a system that can pull real estate listings data from sources such as Zillow, Redfin, StreetEasy, and other relevant platforms. Enable users to paste or upload property listing data into an Excel sheet for further processing. Data Extraction: Automate the extraction of zoning data for each pr...
I'm in need of a Python expert who can maintain my web scraping code. The code is currently set up to collect data from various news websites on a daily basis. Key Responsibilities: - Ensuring the scraping code runs smoothly and efficiently every day. - Troubleshooting any issues that arise with the code or the websites being scraped. - Implementing updates to the code as needed, such as changes in website structure or scraping requirements. Ideal Skills: - Strong Python programming skills. - Experience with web scraping, particularly from news sites. - Problem-solving skills for troubleshooting code issues. - Ability to adapt to changes in website structures. Your expertise will help ensure consistent and reliable data colle...
I'm in need of a skilled web scraper and automation expert to assist with a unique project. The task involves scraping phone numbers from a specific directory site. The project requires: - Proficient skills in web scraping, specifically targeting directory sites. - Experience with automation, particularly sending messages via WhatsApp. - Ability to create a system that triggers a WhatsApp message whenever a new phone number is detected. The ideal candidate would have a keen eye for detail, understand the importance of data privacy, and be able to deliver a seamless, efficient system.
I'm looking for a proficient web scraping expert skilled in Javascript, Node.js, and Python. Your main tasks will include: - Scraping profile information from various social media platforms - Data mining from Shopify Ideal candidates should be well-versed in data extraction techniques, have a keen eye for detail and able to handle large datasets.
...I need a Python developer to modify and improve an existing script. Tasks will include: ✅ Adding small features to an existing Python script ✅ Debugging & fixing minor issues ✅ Making small modifications based on clear instructions ? Payment per task: Small fixes/modifications: $5-$10 Medium feature additions: $15-$30 More complex tasks: Higher rates possible ? First task will be a $5 paid test to check work quality. Requirements: ✔ Python scripting experience (loops, functions, API calls) ✔ Basic understanding of web scraping or automation ✔ Ability to follow instructions and deliver clean code ✔ Clear communication in English ? To apply, answer these questions: 1️⃣ How many years of Python experience do you have? 2️⃣ Have you worked with web ...
...website identical to OpenSouq in terms of features and design. Key Requirements: Core Website Features: User account registration (individuals and businesses). Classified ads system organized by categories (Real Estate, Cars, Jobs, etc.). Advanced internal search engine with various filters. User dashboards to manage ads and updates. Rating and review system for users and sellers. UI/UX Design: Responsive and attractive design compatible with all devices (Desktop and Mobile). Simplified and user-friendly navigation. Mobile Applications Development: Fully integrated mobile apps for iOS and Android. Features such as push notifications, location-based services, and direct image uploads. Hosting and Infr...
...Personal/Sensitive Data in the Database - Cron Jobs Run from the Same Server - Lack of "GitIgnore" File - Lack of code documentation - No Git History of the Code - Presence of Hardcoded Variables in the Code - Presence of Secrets/Sensitive Information in the GitHub Repository - Showcase Website Integrated into the Platform - Presence of Variables in Multiple Locations in the Code - Security of User Accounts - User Account Creation All of these issues are top priority. The platform's security is a significant concern, especially: - Non-Encryption of Personal/Sensitive Data in the Database - Presence of Secrets/Sensitive Information in the GitHub Repository The ideal freelancer for this project should have extensive experience with Ruby on Rails, and a stro...
I'm seeking a developer who can create different micro bots for me. The ma...micro bots for me. The main purpose of these bots will be to automate various actions, primarily data scraping tasks. Key Responsibilities: - Develop around 5 bots each month, focusing on web scraping and APIs data extraction. - Ensure long-term, reliable performance of the bots. - Adapt to potential new tasks and requirements over time. Ideal Skills and Experience: - Proficiency in Python and AI technologies. - Strong experience in creating micro bots for data scraping. - Ability to understand and adapt to different data sources. Please note, the specific websites or APIs to scrape data from will be discussed and decided together. I...
I'm in need of a Python expert who can maintain my web scraping code. The code is currently set up to collect data from various news websites on a daily basis. Key Responsibilities: - Ensuring the scraping code runs smoothly and efficiently every day. - Troubleshooting any issues that arise with the code or the websites being scraped. - Implementing updates to the code as needed, such as changes in website structure or scraping requirements. Ideal Skills: - Strong Python programming skills. - Experience with web scraping, particularly from news sites. - Problem-solving skills for troubleshooting code issues. - Ability to adapt to changes in website structures. Your expertise will help ensure consistent and reliable data colle...
I am looking to extract product information from a website into an Excel sheet. I just need the product name, description, and price from the product listings on the site. Please let me know if you're experienced with web scraping and can finish this task. I just need an Excel Sheet and not a tool. Thank you!
...with potential employers. The platform should provide features for both workers and clients, making it easier to find and manage construction projects. Key features for workers: - Profile Creation: Workers should be able to create detailed profiles showcasing their skills, experience, and availability. - Job Notifications: The app should send notifications about relevant job opportunities based on the worker's profile and location. - Skill Verification: There should be a feature for skill verification to ensure that workers' claimed skills are validated. Key features for clients: - Post Jobs: Clients should be able to easily post job listings. - View Worker Profiles: Clients should have the ability to browse and view workers' p...
I'm looking for a web scraping expert who can extract data on clothing products from various ecommerce platforms. Ideal skills: - Proficiency in web scraping tools and software - Experience with ecommerce platforms - Understanding of data analysis The scope of the project includes: - Scraping clothing items' data - Providing insights and trends from the scraped data
I'm facing multiple issues with my Search Console and need an expert in SEO to help me resolve them. The issues include: - Indexing problems: Some of my pages are not getting indexed and I'm experiencing crawling anomalies. - Crawl errors: I'm encountering 404 errors and 500 server errors. - Other issues: I've got problems with canonical tags and redirections. The ideal candidate should have a deep understanding of SEO and Search Console. They should have experience in resolving indexing problems, crawl errors, and other SEO issues. I am looking for someone who can help me get my website back on track and improve its performance in search engines.
...balancing and improving system communication. The application is partially developed, and additional features need to be integrated. Responsibilities: Implement printer load balancing: Assign specific printers for black-and-white or color jobs. Reroute print jobs to available printers based on defined conditions. Develop and enhance communication between the application and external systems. Refine and optimize existing workflows for better performance. Add additional features as needed during development. Ensure the software is efficient, scalable, and user-friendly. Requirements: Strong expertise in and Node.js. Proficiency in JavaScript for desktop application development. Experience with task allocation and...
...Here are some bullet points to help you hire a developer specifically for fetching option chain stocks and index data using the Dhan API for your Tredex View project: ### **Responsibilities:** - **API Integration:** - Integrate with the **Dhan API** or other relevant financial APIs. - Develop and manage API calls to retrieve real-time stock, option chain, and index data. - **Data Handling:** - Write scripts or software to format and store the fetched data appropriately. - Ensure data is saved in preferred formats such as CSV, JSON, or directly into databases. - **Automation:** - Set up cron jobs or schedulers to automate the data fetching process. - Ensure regular updates and timely retrieval of data. - **Data Storage:** - Sto...
I'm looking for a freelancer to help me with extracting data from a website. The data is spread across different web pages, and I need it to be exported into separate Word documents for each page. Only bid if you can do the job. Data to Extract: - Text content - Images (to be saved in JPEG format) all in one document each Document Organization: - Each Word document should have separate sections for text and images. Ideal Skills and Experience: - Prior experience with web scraping. - Proficiency in creating and formatting Word documents. - Attention to detail for data accuracy and organization.
I need a scraping expert to debug my Python script that's supposed to be fetching financial data from APIs. Currently, the script is failing to retrieve some data. Ideal Skills and Experience: - Proficient in Python and data scraping - Experienced with financial data APIs - Strong debugging skills - Knowledge of data parsing and handling - familiar with yahoo finance data
I'm seeking a freelancer to compile a detailed, monthly updated database of USA credit cards for a public website or app. Key Requirements: - image of the card. - Information on Interest Rates and Fees: Understand and gather data concerning the cost of owning the card. - Rewards and Benefits: Identify the perks associated with each card including cash back, travel points, etc. - Eligibility Requirements: Document what is needed to qualify for each credit card. Ideal Skills: - web scraping, cleaning and storing data - Familiarity with web/app data requirements Please note: This project will require a commitment to quarterly updates.
I need a skilled web scraper who can extract data from a members area of a website. The data should be compiled and delivered in an Excel file. Website Scraping of Members Area Ideal Skills: - Web scraping expertise - Proficiency in Excel - Attention to detail
...three Python scripts for an analysis project involving NASDAQ data . These scripts will scrape stock prices and dividends from free stock data site (i.e., ), place the data in a single-sheet Excel file, and create a line graph similar to an attachment I will provide (). Script 1: Download stock prices (for a year - Download Historical ) and dividend data (for a year) for a given stock Script 2: save data to a single-sheet Excel file Script 3: Create a chart in the Excel file The ideal freelancer for this project should have: - Extensive experience with Python, particularly with libraries for web scraping and data analysis. - Proficiency in creating and manipulating Excel files with Python. - Skills in data visualization, particularly in...
I'm in need of a skilled programmer who can create an independent bot for me. The bot's main task will be to retrieve cer...will be to retrieve certain information from a secure website. Key Requirements: - The bot should be able to log into the site with provided credentials. - It should collect specific user data, particularly user activity and user messages. Ideal Skills: - Proficiency in programming languages such as Python, Java, or similar. - Prior experience in web scraping and bot development. - Understanding of data security and ethical considerations in data retrieval. - Ability to deliver a reliable, efficient, and independent bot. Please note: The target website is secure and requires login credentials. The success...
Hi. I would like to scrape oddsportal for game results,closing and opening all odds import them into excel. I am interested in soccer matches. The scraper should be able to scrape data according to date i.e. I would like to input a date range and sport (soccer) and get the match details for those matches. Further details will be discussed with bidder. I should also mention that date range for scrapping data could be quite long (i.e. months) so the output xls could be quite big. It is very important to me that scraping is done as quickly as possible (seconds) with as less clicks as possible and without hanging-up or crippling my PC while scraping. PROJECT DETAILS: Sport:----------------------Football Bookmaker:------------------Bet365, BWIN, P...
I'm looking for an expert in AI tools to scrape a website for me. The goal is to extract specific contact information and organise it neatly in an Excel sheet. Data to be Scraped: - Email addresses - Phone numbers - Names - Zip Codes The ideal candidate should have prior experience in web scraping and data management. Proficiency in using AI tools for this task is crucial. Data Formatting: - I need the data to be sorted into separate sheets for each type of information. Please provide examples of previous similar tasks you have completed successfully.
I'm seeking a proactive recruiter to assist me in finding and applying for mid-level automated testing QA jobs. Key Responsibilities: - Search for mid-level automated testing QA jobs - Tailor my skills to match each job requirement - Submit applications on my behalf - Handle recruiter calls and arrange interviews The ideal candidate will have: - A strong understanding of automated testing - Experience in recruiting for QA positions - Excellent communication skills
I am looking for an expert in automation and data scraping to set up scenarios in MAKE that link with my social media accounts and chat GPT. The goal is to have CSV files automatically publishing across all platforms with AI generated lifestyle images featuring my product. Additionally, I need assistance with data scraping for emails in my group. Ideal Skills: - Proficiency in using MAKE for automation - Experience with social media management - Ability to work with CSV files - Familiarity with AI image generation - Skills in data scraping Social Media Platforms: - Facebook - Instagram - TikTok CSV File Format: - Comma-separated values Type of AI-Generated Images: - Lifestyle images with my product
...Increase the visibility and organic traffic of a German-language WordPress website on Google using legal (White Hat) SEO practices. The site is correctly integrated with Google Search Console (GSC) and uses RankMath for SEO optimization. Despite previous efforts in SEO and page speed optimization, the AI-generated content is underperforming in Google rankings. A strategic approach focusing on resolving GSC errors and enhancing overall site visibility is required. --- ### Scope of Work: 1. **Google Search Console (GSC) Error Analysis & Resolution** - Analyze all reported errors and warnings in GSC. - Identify root causes and implement solutions directly within the WordPress environment. - Ensure all fixes are aligned with Google&r...
...with Python to automate data pipelines and manipulate data. - Utilizing SQL for query optimization, managing joins, and performance tuning. - Employing Postgres for relational data modeling and management. - to produce clear reports or dashboards showcasing insights and progress. The ideal candidate will also be able to: - Understand or utilize AI-based techniques for data scraping. - Conduct exploratory data analysis, performing data cleaning and feature engineering as necessary. A critical part of the role will be translating business objectives into technical requirements. Hence, strong verbal and written communication skills are necessary for interacting with stakeholders. Your ability to produce clear reports or dashboards showcasi...
...expertise in API integrations, web scraping, and webhooks to work on a complex recruitment automation project. The core of this project involves integrating with the Manatal recruitment platform API, processing data from external services like OpenAI, and automating workflows to enhance candidate evaluation. This includes fetching candidate data, dynamically associating job descriptions, processing resumes via OpenAI's API, and updating scores and custom fields directly in the Manatal platform. Additionally, the project requires monitoring assessment completion through webhook events and triggering automated reminders or updates based on specific conditions. The ideal candidate must have strong experience with RESTful APIs, managing...
