Web scraping in python using scrapy jobs
I'm seeking an expert in Life Cycle Assessment (LCA) and Gabi software to evaluate the environmental impact of an Enhanced Oil Recovery (EOR) project. The process primarily involves gas injection, specifically Carbon Dioxide (CO2). Key Responsibilities: - Conduct a comprehensive LCA using Gabi software - Focus on the environmental implications of CO2 gas injection in EOR - Provide insightful reports and recommendations Ideal Skills and Experience: - Proven experience in LCA, preferably in the oil and gas industry - Proficient in using Gabi software - Strong understanding of EOR processes and gas injection techniques - Excellent report writing and communication skills.
...images, descriptions, and pricing -Automated data updates from external foreclosure listing sources - I.e. scraping or third party API -Backend Administration Panel: -Manage property listings (add/edit/delete) -Track subscriptions and payments -Analytics dashboard for user activity and revenue -User support/ticketing system integration Technology Preferences: - Based on WordPress CMS preferred (for ease of management and scalability) - Custom plugin development if required - Secure and scalable hosting setup recommendation - Mobile-responsive design (optimized for all devices) Security & Compliance: -Secure user data storage (GDPR/CCPA compliance) - SSL certification and best practices for web security - Payment gateway security (PCI compliance) Performance Optimiz...
I'm trying to learn things online by gathering the CC of Videos into TXT files and read it instead. I need a way to automate the gathering of the text files with the close captioning. Here is a video explaining: Desired Programming Language: Python C# (preferred) Or whatever is best for this type of work * Do not bid if you have under 3 reviews, you will be ignored * Need Source Code when Job is Complete, After any Bug fixes, and Updates * Milestone release upon completion of project & is bug free. * Please post "I've read everything" when applying, so I'll know you actually read this and give you priority response. * I'm looking for best deal possible.
...task description for developing a web application with three main interfaces: Admin, Vendor, and Customer. The application will require secure login systems, functionality for managing flight and vendor information, and real-time tracking for shipments. To begin, I will follow the outlined steps: Use the search_code tool to find relevant files or code snippets that may already exist in the current working directory, which could help in understanding the structure and requirements for the new application. If no relevant files are found, I will consider creating new files based on the specifications provided. After gathering information, I will proceed to create a comprehensive plan using the brainstorm_plan tool. I will start by searching for relevant files ...
I'm seeking a professional to scrape my WordPress website for 1100 entries related to our different offices. The data needs to be exported into an Excel file, without any specific filters or sorting. In addition to data scraping, I need a thorough code review of my WordPress site. This should encompass: - Documentation and review of custom code - Optimization where necessary - Plugin review for updates - Recommendations for optimization This project requires a deep understanding of PHP, as well as experience with WordPress and data scraping. The ideal freelancer will have a strong background in website code review and optimization.
We have a python app that uses GPT API running on a local machine using local GPUs. The front end is single screen PyQT5. We now need to scale this to a web service running on , converting the PyQT5 UI to Gradio for the UI/orchestration layer so we can run batches on multiple instances and easily scale GPU and CPU resources as needed. We've scoped this initial deployment at about 80 hours for a developer with these proven skills (modal and gradio), with future work to continue based on performance. Our app batch uses batch downloaded docs. via API, parses and uses LLM to generate a docx report and set of .csv files. Analyzed content is based in the biosciences area and in future phase we'll be adding functionality to analyze bio-sequences us...
I have a Python-based AI project that currently runs smoothly on Windows. I need a skilled developer to help me enhance its data processing capabilities and improve its accuracy. Additionally, the project needs to be adapted for seamless operation on Mac. Key Responsibilities: - Update data processing functionalities to enhance the project's accuracy. - Port the project from Windows to Mac, ensuring compatibility and performance stability. Ideal Skills and Experience: - Proficient in Python, with extensive experience in AI development. - Strong knowledge of data processing techniques and accuracy improvement strategies. - Mac OS software development experience is a must. - Familiarity with AI accuracy measurement tools would be a plus.
deliver a utility for log monitoring in python for the next 8 hours ...
I'm seeking an experienced Python developer to upgrade my Django-based web application from Python 2.7 to the latest version. This project is strictly an upgrade task - I'm not looking to add new features or enhance existing ones. Ideal Skills: - Extensive experience with Python and Django - Proven track record of upgrading Python applications - Familiarity with web application architecture Please note that the primary focus of this project is not to modify the application's functionality, but to ensure it's fully compatible with the latest Python version. Your ability to work meticulously and understand the nuances of this upgrade will be key to the success of this project.
PapAPP was conceived to address significant challenges in academic communication and organization encountered by parents and students. By centralizing academic information, such as grades, schedules, and assignments, into one accessible and user-friendly platform, PapAPP aims to simplify the management of academic responsibilities and keep parents informed of their child's tests and assignments via messages so they can accompany their child throughout their study. A non-functional prototype has been developed in Miro, detailing key workflows, user interfaces, and screen designs. This prototype serves as the foundation for the development process and reflects the intended functionalities of the final product. The main goals of PapAPP are to reduce stress between parents a...
...to Twist and email. We’re open to using any AI platform that effectively meets these needs. Responsibilities: Build and integrate a voice agent capable of managing calls (inbound and outbound). Enable the agent to interact with a knowledge base to answer queries accurately. Develop functionality to interface with RingCentral for transferring, holding, and parking calls. Utilize to automate sending call transcripts to Twist and email. Troubleshoot and optimize performance to ensure reliability. Provide clear documentation for ongoing use and future enhancements. Requirements: Experience with API integration and voice agent development. Familiarity with AI voice platforms (e.g., ElevenLabs, RingCentral, or others). Strong programming skills (Python, JavaScript, or ...
We want to create a fully custom-coded LMS website inspired by (only referencing its layout and user flow) using Cursor AI. This means no WordPress, Wix, or Figma—just custom HTML/CSS/JS with code generation and editing via Cursor AI. After deploying on Netlify, we’ll later revise images/text to avoid any direct duplication or trademark elements from the original. Reference Pages (for structure and design approach only): Main: Landing: Menu 1 (Company Intro): Menu 2 (Landing Page Intro): Menu 3 (Purchase Page):
...solutions for businesses. The role involves building and deploying AI agents and leveraging various AI models to develop impactful web applications tailored to business needs. Key Responsibilities: • Design, build, and deploy AI agents to address real-world business challenges. • Develop and integrate AI-driven solutions using frameworks such as TensorFlow, PyTorch, or other relevant platforms. • Work with language models (e.g., GPT, BERT, LLaMA) to create applications for natural language processing, automation, and decision-making. • Leverage tools like n8n, , or other automation platforms to streamline workflows and enhance efficiency. • Develop scalable web applications with AI components tailored to business operations. • St...
We have 11,000 manufacturer SKU's that we would like to do price comparison across 10 different ecommerce websites. 100% accuracy is a must. 5 star freelancers only apply.
...and formatted for easy access within Excel. Provide clear, user-friendly functionality to paste or upload listings and automatically trigger the data extraction process. Skills Required: Strong experience with web scraping or API integration to pull data from Zillow, Redfin, StreetEasy, and other platforms. Proficiency in Excel automation, preferably with VBA or Python (pandas, openpyxl) to integrate the data extraction functionality. Familiarity with Zola NYC and NYC Open Portal to access zoning and building-related data. Knowledge of real estate terminologies and zoning regulations in NYC. Experience with data validation to ensure accurate and reliable data extraction. Ability to work autonomously and deliver high-quality, maintainable code. Preferr...
I'm in need of a Python expert who can maintain my web scraping code. The code is currently set up to collect data from various news websites on a daily basis. Key Responsibilities: - Ensuring the scraping code runs smoothly and efficiently every day. - Troubleshooting any issues that arise with the code or the websites being scraped. - Implementing updates to the code as needed, such as changes in website structure or scraping requirements. Ideal Skills: - Strong Python programming skills. - Experience with web scraping, particularly from news sites. - Problem-solving skills for troubleshooting code issues. - Ability to adapt to changes in website structures. Your expertise will help ensure consistent an...
I'm seeking a skilled Python programmer to automate the handling of mixed data inputs into an Excel spreadsheet. Historical trades will be used as a reference point to retrieve data based on a number of conditions and variables from various financial instruments utilising historical data from 2016 onwards. This process will need to be scheduled to complete on weekly and ad hoc basis utilising API connection for historical price capture. Once the above is completed the same datasets will be used to simulate and output data based on the conditions set out in the simulation. Some understanding of trading terminology and bar/candle charts would be useful as this will would make it easier to comprehend the conditions and variables but is not a pre-requisite. Key Requireme...
...convert customer input into text. Text-to-Speech to relay ChatGPT's responses to the customer. Call tracking (duration in seconds) for billing purposes. AI Integration: Integrate OpenAI's ChatGPT API to process transcribed input and generate conversational responses. Ensure seamless, natural conversation flow. Backend Development: Create a backend system (Python with Flask/FastAPI or Node.js with Express) to: Handle API calls to Twilio, OpenAI, and Stripe. Store customer details, PaymentIntent IDs, and call logs in a database (SQLite/PostgreSQL). Implement a webhook to monitor call status and trigger payment capture when the call ends. Web Interface: Develop a simple web page where customers can: Register and input payment details. View th...
I'm in need of a skilled web scraper and automation expert to assist with a unique project. The task involves scraping phone numbers from a specific directory site. The project requires: - Proficient skills in web scraping, specifically targeting directory sites. - Experience with automation, particularly sending messages via WhatsApp. - Ability to create a system that triggers a WhatsApp message whenever a new phone number is detected. The ideal candidate would have a keen eye for detail, understand the importance of data privacy, and be able to deliver a seamless, efficient system.
I'm looking for a proficient web scraping expert skilled in Javascript, Node.js, and Python. Your main tasks will include: - Scraping profile information from various social media platforms - Data mining from Shopify Ideal candidates should be well-versed in data extraction techniques, have a keen eye for detail and able to handle large datasets.
Project Title: ? Python Developer Needed for Script Modifications (Task-Based, Fixed Price) Project Description: I need a Python developer to modify and improve an existing script. Tasks will include: ✅ Adding small features to an existing Python script ✅ Debugging & fixing minor issues ✅ Making small modifications based on clear instructions ? Payment per task: Small fixes/modifications: $5-$10 Medium feature additions: $15-$30 More complex tasks: Higher rates possible ? First task will be a $5 paid test to check work quality. Requirements: ✔ Python scripting experience (loops, functions, API calls) ✔ Basic understanding of web scraping or automation ✔ Ability to follow instructions and deliver clean code ✔ Clear communication in Eng...
Development of a Python Package for LinkedIn Ad Campaign Creation and Management Overview: We are seeking a highly skilled Python developer to create a comprehensive Python package / SDK that automates the creation of LinkedIn Ad campaigns, and provides robust functionality for managing and analyzing existing campaigns. This package will empower users to create, monitor, and manage LinkedIn Ad campaigns tailored to a specific target list provided by the user. Project Objectives: Python Package / SDK Development Build a Python package that interfaces with the LinkedIn Ads API to perform end-to-end ad campaign management. Design the package to be modular, user-friendly, and extensible for future enhancements. Key Features: 1) Campaign Creation Allow use...
I'm looking for a Python developer to create a ball tracking application for live cricket matches. This app's primary objective is to enhance viewer experience by providing real-time data on the ball's motion. Key Features: - The app should capture various data points during the ball's motion, including the speed of the ball, its trajectory, and its spin and swing. - It should be able to support mobile apps and TV broadcasts. Ideal Skills: - Proficient in Python - Experience in developing sports-related applications - Familiarity with live data tracking systems - Understanding of cricket and its ball dynamics would be an advantage. Please contact on below mentioned contact numbers for more Information : Amit Wakade : +91 8329850086 What�...
Looking to tweak a health check python script which is not properly functioning
I need a Python script modified to integrate with AWS's Route 53 API to react to health checks. The script currently pings for health checks and then emails me an alert if there are 3 consecutive failures. Then emails me again once everything is back online. I want to add functionality to this. Upon failure, also ping an outside IP address like 8.8.8.8 to confirm if the failure is internally related vs external. If 8.8.8.8 fails, do not take any action. If 8.8.8.8 succeeds, then tap into the Route 53 API and make a DNS record change. Eventually, when the remote host comes back online, the health checker should check for 30 out of 30 successful "pings". Upon the success, I want the DNS change to be reverted to its original state, which we can define in the cod...
Project Title: Custom Crypto Trading Bot Development for Exchange Project Description: I’m looking for an experienced developer to create a custom crypto trading bot that integrates with the Exchange using its API. The bot should be able to execute trades automatically based on pre-defined strategies and should include the following features: Key Features: 1. Exchange Integration: • Full integration with the API. • Secure handling of API keys (read and trade permissions only, no withdrawal access). 2. Trading Features: • Support for multiple trading pairs (e.g., BTC/USDT, ETH/USDT). • Ability to implement trading strategies (e.g., scalping, grid trading, dollar-cost averaging, etc.). • Risk management tools, such as stop-loss and take-pro...
...with a strong background in backend, frontend, and blockchain technology to deploy SherlockSol, an AI-powered fraud detection system. The ideal candidate will have expertise in Python, React.js, and blockchain APIs and be proficient in automating workflows using task schedulers and email services. Key Responsibilities: Backend Development: Integrate Solscan API to fetch wallet and transaction data from the Solana blockchain. Implement rule-based and ML-enhanced fraud detection systems. Set up Celery with Redis for task scheduling. Frontend Development: Create an interactive React.js dashboard. Visualize wallet risk scores, fraud summaries, and transaction trends using libraries like Recharts. Task Automation and Email Service: Automate email...
I'm seeking a developer who can create different micro bots for me. The main purpose of these bots will be to automate various actions, primarily data scraping tasks. Key Responsibilities: - Develop around 5 bots each month, focusing on web scraping and APIs data extraction. - Ensure long-term, reliable performance of the bots. - Adapt to potential new tasks and requirements over time. Ideal Skills and Experience: - Proficiency in Python and AI technologies. - Strong experience in creating micro bots for data scraping. - Ability to understand and adapt to different data sources. Please note, the specific websites or APIs to scrape data from will be discussed and decided together. I'm looking for a long-term collaboration, so rel...
I'm in need of a Python expert who can maintain my web scraping code. The code is currently set up to collect data from various news websites on a daily basis. Key Responsibilities: - Ensuring the scraping code runs smoothly and efficiently every day. - Troubleshooting any issues that arise with the code or the websites being scraped. - Implementing updates to the code as needed, such as changes in website structure or scraping requirements. Ideal Skills: - Strong Python programming skills. - Experience with web scraping, particularly from news sites. - Problem-solving skills for troubleshooting code issues. - Ability to adapt to changes in website structures. Your expertise will help ensure consistent an...
I am looking to extract product information from a website into an Excel sheet. I just need the product name, description, and price from the product listings on the site. Please let me know if you're experienced with web scraping and can finish this task. I just need an Excel Sheet and not a tool. Thank you!
when one dropdown is clicked other dropdown are opened..
I have a Python script used for monitoring uptime of a specific protocol. It needs some quick tweaking, and would also like to get it to make a DNS change via the Route 53 API (AWS) when the uptime check fails. The script should then revert the DNS change once uptime is restored. Key Requirements: - Tweaking of the existing Python script to improve its functionality and performance. - Implementing a new feature to interact with the Route 53 API. - The script should be able to initiate a DNS change if a failure is detected. Ideal Skills: - Strong proficiency in Python. - Prior experience with AWS Route 53 and API is required. - Excellent problem-solving skills to enhance script performance and functionality.
I have a Python script used for monitoring uptime of a specific protocol. It needs some quick tweaking, and would also like to get it to make a DNS change via the Route 53 API (AWS) when the uptime check fails. The script should then revert the DNS change once uptime is restored. Key Requirements: - Tweaking of the existing Python script to improve its functionality and performance. - Implementing a new feature to interact with the Route 53 API. - The script should be able to initiate a DNS change if a failure is detected. Ideal Skills: - Strong proficiency in Python. - Prior experience with AWS Route 53 and API is required. - Excellent problem-solving skills to enhance script performance and functionality.
I am looking for a developer who can develop portable USDT Flashing Software in main network (windows based, preferably windows 10), the requirements are mentioned below, 1) The software can perform or generate flash of the USDT, as well as the flashed USDT should be transferable from/to unlimited wallets with blockchain confirmations is appreciated. since it is flash btc it must be visible in BTC wallet address for 365 days. There will be no transaction fee while sending from "flash s/w" to "btc wallet" as it will be no confirmations. There will be no limitation for daily transfer for usdt It can’t be tracked as fake transaction. should take normal time to send btc one wallet to another wallet in normal internet connection. 2) These usdt ...
I'm looking for a professional who can help me extract a cutting pattern from a designed picture and visualize it in 3D. The picture is in JPEG/PNG format and the cutting pattern will be used for garment manufacturing. The final cutting pattern needs to be delivered in PDF format. Ideal skills and experience for the job: - Proficiency in pattern making & garment manufacturing - 3D modeling skills - Ability to work with JPEG/PNG files - Proficient in creating & delivering PDF files - Previous experience with creating cutting patterns is a plus Task: Design the Top and Bottom wear in this picture, show it in 3D, also, I want to fetch the cutting pattern
I'm seeking a data science person who loves playing with data and analyzing using data science techniques. The data primarily revolves around numerical, specifically stock market data. Key skills: Data Science, Statistics, Machine learning, Data Analysis, Python
...skilled Python developer with expertise in computer vision and image processing to develop a solution for extracting individual objects (nails) from an image or a set of images. The ideal candidate will have experience with segmentation techniques and be able to deliver precise results in a variety of image conditions. Responsibilities: • Develop a Python-based solution to: • Identify and isolate specific objects (nails) within an image. • Segment and extract each object as a separate image file. • Use computer vision techniques (e.g., edge detection, contouring, or deep learning) to ensure accuracy. • Handle varying image conditions, such as lighting, angles, and backgrounds. • Provide clean, well-documented, and reusable...
My WebSocket connection is experiencing slowness during data transmission. The connection is established with a cloud server and is handling both text and binary data. I need an expert Python developer to analyze and optimize the existing code to enhance the speed of data transmission. Ideal Skills: - Proficient in Python - Experience with WebSocket - Knowledge of optimizing data transmission The task involves a quick fix on the provided code to make the connection faster.
...social media posts along with a scheduled datetime (Python datetime object in UTC). I also have a system in place to submit posts to Mastodon. What I need is a mechanism that: - Reads the scheduled datetime for each post in my DB that has the 'scheduled' flag in its status field - Triggers the submission process once the scheduled datetime has arrived The details are as follows: - The mechanism should check for scheduled posts every minute. This probably requires Celery. - No logging or monitoring of the posting mechanism is necessary. - If a post fails to send at the scheduled time, it should retry after a delay. Ideal skills and experience for this job include: - Proficiency in Django (and perhaps minimal Postgres) - Knowledge of...
I need a short Python program that can extract transaction details from my PDF bank statements into both CSV and Excel formats. All my statements are in the same format, so consistency won't be an issue. Ideal Skills: - Proficiency in Python - Experience with PDF data extraction - Knowledge of CSV and Excel programming This project is a great opportunity for a skilled Python developer with experience in data extraction and a keen eye for detail.
I'm looking for a web scraping expert who can extract data on clothing products from various ecommerce platforms. Ideal skills: - Proficiency in web scraping tools and software - Experience with ecommerce platforms - Understanding of data analysis The scope of the project includes: - Scraping clothing items' data - Providing insights and trends from the scraped data
I'm seeking a proficient Excel developer with strong skills in VBA and Macros. The job involves creating an automated workflow to transform data from a single Excel file into three separate files. Key Tasks: - Importing data from one Excel file - Creating three files using VBA and Macros - Data cleaning - Data merging - Data filtering The data filtering will involve: - Applying numerical ranges - Text matching - Date ranges Ideal candidates should have a robust understanding of data transformation processes and extensive experience with Excel VBA and Macros.
...this project. 2. Use RETS system to download 20 years worth of historical data from an existing real estate MLS server. 3. Use RESTFUL API to set up daily, incremental downloads of the same data. The existing database is a real estate MLS using a very well established RESTFUL API standard called RESO Web API Core 1.0 (). The reason for using the RETS system to do the initial download is that the number of historical records in some of the data fields has proven to be impractical (tens of millions of records) to download using the API as it is simply too slow. I am open to alternative methods here though. There will be a limited number of transforms (calculated fields) based upon the existing data structure, perhaps 15-20. Ideally, once the database...
I'm in need of a seasoned software engineer to review and enhance the firmware of our home automation system. The task primarily involves optimizing performance, with a special focus on the Flask web framework and its security measures. Key Responsibilities: - Conduct a comprehensive review of the firmware - Identify performance bottlenecks and areas for optimization - Specifically address the security measures within the Flask framework - Implement necessary code changes in Python, MySQL, Mosquitto, and Flask Ideal Skills: - Extensive experience with firmware review and optimization - Proficiency in Python, MySQL, Mosquitto, and Flask - Strong understanding of home automation systems - Experience with optimizing the Flask web framew...
I'm seeking an experienced algo developer to create a Python algorithm for trend following in stock trading. Key Requirements: - Proficient in Python and algorithm development - Extensive knowledge of trend following strategy - Experience in stock trading algorithms - Good understanding of stock market trends Please provide examples of similar projects you've completed in your bid.
I am looking for help with creating a simple Python strategy to be used for crypto trading on Binance. The script is basically a simple trade management and hedging script that will allow me to open and manage multiple orders in hedge mode. The key input variables to be specified by the user are: Trading Symbol Upper Limit Price Lower Limit Price Take Profit (In Percentage) Stop Loss (In Percentage) Initial Trade: Long or Short Hedge Multiplier Maximum Number of Hedges If the user specifies Initial Trade = Long, the trading engine will seek to open BUY STOP orders at the Upper Limit Price and undertake SELL STOP orders at the Lower Limit Price (as a hedge) Let’s assume that the current market price is $100. The user specifies Initial Trade = L...
I'm looking for a skilled developer or a team to build a robust SaaS platform...portfolios. - Automated Financial Reporting: The system should be capable of generating financial reports without manual intervention. - Workflow Automation: This includes primarily task assignment features. - Data Visualization Dashboards: The dashboards will primarily display data in tables and grids. Ideal Skills and Experience: - Proficient in Angular for the frontend. - Strong backend experience with Python (Django) or Node.js. - Knowledgeable in databases, particularly PostgreSQL. - Previous experience in developing financial or accounting software. - Excellent communication and project management skills. This is a great opportunity for you to showcase your skill...
