Web crawling and scraping using python jobs
I'm looking for a skilled developer who can help me with the backend of my N9K platform. The backend should be built using Python/Django, Lunux and C Lnaguage and it should cater to both web and mobile applications. Key Requirements: - Proficient in Python and Django - Proficient in C and Linux - Experience in developing backends for both web and mobile applications - Capability to implement user authentication and management, data storage and retrieval, and API integration Please note, the specific functionalities for the backend have not yet been finalized, but the aforementioned points will be key aspects of the project. The ideal candidate would be flexible and able to adap...
I'm looking for a Python expert who can conduct research on harmonic patterns, particularly the Gartley pattern, using historical daily stock prices from the last 5 years of 10 companies listed on the Malaysian stock market. The research should be conducted on Google Colab and should provide clear entry levels, stop-loss and take-profit levels. Ideal Skills and Experience: - Proficiency in Python and Google Colab - Experience with stock market analysis - Understanding of harmonic patterns, particularly the Gartley pattern - Ability to present data in a clear, concise manner Please note, the analysis should solely use historical stock prices.
I'm looking for a skilled software developer to create a desktop application tailored for productivity. This application will primarily target Windows users. Key Requirements: - Experti...tailored for productivity. This application will primarily target Windows users. Key Requirements: - Expertise in Windows desktop application development. - Strong background in creating productivity tools. - Ability to design intuitive and user-friendly interfaces. - Experience with software testing and debugging. Ideal Skills: - Proficiency in languages like C#, Java, or Python. - Familiarity with UI/UX design principles. - Knowledge of productivity software trends. Your role will be crucial in delivering a high-quality product that meets user needs and stands out ...
I have an existing scraper built in python for the website (CWH). It scrapes the following information: - product description, price, images - SEO metadata - Additional metadata Currently, all the scraped data is stored in a CSV file. I am looking for a developer who has expertise in python / scraping and postgres to do the following: [Phase / Milestone 1] 1. Convert the existing scraper (code attached - /site-packages removed for space) to load the data in a postgres database. 2. Scrape reviews of products from the (CWH) website and store them in the database. 3. Look up the availability of the product in specific stores and write the boolean value in the postgres database. (code attached - ) [Phase
I'm in need of a Python script that can download an Excel report from a public website every hour and upload it to a designated portal folder without any modifications. - **Task**: Script should perform the following tasks - Download an Excel report from a public website every hour. - Upload the report to a specific portal folder without any modifications. Ideal Skills: - Proficient in Python programming. - Prior experience with web scraping and automating tasks. - Familiarity with working with Excel files in Python. - Knowledge of file upload protocols and processes.
My existing Python script requires a skilled developer to optimize its performance, particularly its user interface response time. Key Requirements: - Expertise in Python programming - Experience with performance optimization - Proficient in enhancing command-line interface scripts The script is currently running with a command-line interface and the primary performance issues are noticeable in the user interface response time. The goal of this project is to improve the overall efficiency of the script, ensuring a smoother user experience.
Summary: Need to migrate an existing website and add proper structure, access control, admin section. Basically need a proper skeleton created to be able to add additional functionality later. # Software Requirements Specification for Online Worksheet Practice Website Structure ## 1. Overview This document outlines the software requirements for an online worksheet practice website to be developed using CodeIgniter 4. The website will provide educational resources in the form of worksheets, categorized by grade level and subject, similar to and education.com. It will include features for various user access levels, content organization, and administration. A simple working version of this website is already available (). The developer
I'm seeking a proficient mobile app developer or a dedicated team to create a uniqu...MetaHuman integration for real-time lip-syncing responses The ideal candidate or team should possess an impressive portfolio and experience in: - Cross-platform app development, with a strong focus on Flutter or React Native - Leveraging Speech recognition (STT) and Text-to-Speech (TTS) solutions - Integrating with Unreal Engine and MetaHuman - Backend development, proficient in Python (Flask/Django) or Node.js The project demands flawless synchronization between user voice inputs, NLP processing, and animated responses from the MetaHuman. If your background includes creating interactive or AI-driven applications, I would be eager to collaborate. Please provide yo...
I'm in need of a versatile professional skilled in Squarespace and Canva. Your primary task will be to set up product pages on Squarespace, particularly focusing on integrating local delivery options and managing courier charges. You will also be tasked with creating a series of Canva social media posts and setting up an e-learning course in Squarespace (I will provide the written content). Additional responsibilities will include various general marketing tasks, such as scraping email addresses. Ideal Skills: - Proficiency in Squarespace and Canva - Experience in e-learning course setup - Knowledge of basic marketing tasks - Skills in data scraping Understanding of product fulfillment and delivery integration on e-commerc...
...infrastructure that processes and visualizes data using our existing Python and SQL assets. The system must ensure automation, security, scalability, and stability while seamlessly integrating with the our website. Current Status Existing Resources: - SQL Database is created and mapped; all datatypes were harmonized. - All required Python and SQL scripts have been developed and tested locally. - An Azure workspace has been created to host the infrastructure. Infrastructure Capabilities Required: 1. Data Ingestion The infrastructure must support both scheduled and event-driven batch data ingestion from multiple APIs and other external sources. The ingested data is supposed to be parsed and transformed...
...utilizing OpenAI APIs, Speech-to-Text (STT), and Text-to-Speech (TTS) services. The goal is to have an automated system that calls customers, offers services, and engages in conversation to address customer inquiries effectively. The project is already partially developed: The system is built using Issabel and Asterisk. The call center module in Issabel is set up and functional. OpenAI APIs have been integrated for dynamic conversational responses. Current Issue: The project needs to be finalized. Specifically, we need the call center module to handle the following: Import large .csv files containing 1,000-5,000 phone numbers. Automatically call customers listed in the .csv file. Offer services and respond to their questions dynamically u...
I'm working on a Python-based automation script focusing on network tasks, specifically tracing network paths (traceroute). I need a skilled Python developer with experience in network automation to assist me in developing this script. Ideal Skills: - Proficiency in Python - Experience in developing automation scripts - Knowledge in network automation tasks - Familiarity with traceroute and similar network tools
I'm looking for a script written in either Python or PowerShell that can take hundreds of folders containing files and divide them into sets of specified size. The script will be run on a Windows operating system. Key requirements: - The script should be capable of dividing folders with files into near sets of specified size. - Experience with Python or PowerShell is a must. - Windows OS Please note that the script needs to be efficient, as I have a large number of folders to process.
HM Interface Designed by Matlab program will send step moto...file communicates will Matlab program. Program Interface will show and control current angle/step, degrees to Jog, motion jog +/- and X Y Z current position each of joints. On the program Human Machine Interface will have buttons for Teach New Position, Modify Position, call program and delete program, etc. . Program will make Auto Calibration, Manuel Calibration and Show Calibration values for each joints. Program section will be inputs and outputs toggle buttons for entering values servos and digital outputs. MATLAB Program will provide us as design view with written codes not just as .exe file. Program will be open source and developed by every users. Program will allow users...
...GPT model using existing scripts, competitor content, and audience insights to create high-quality, engaging scripts for YouTube and TikTok. The model will be used to generate short-form scripts, visual suggestions, and SEO-optimized content for job seekers at all levels. Project Scope: 1. Data Preparation: • Organize and format existing scripts, competitor analysis, and audience insights into JSONL format. • Ensure the dataset reflects the brand’s tone: professional, fun, and disruptive. 2. Model Fine-Tuning: • Fine-tune OpenAI GPT (preferably curie or davinci) to match the specific tone and content needs of the Mr. Jobs brand. • Include hooks, short-form video scripts, and engaging content...
...primarily focused on mobile. The platform should deliver customized content, quizzes with real-time feedback, and personalized recommendations. Key Requirements: - Frontend Development: For mobile, I'm open to either Flutter or React Native. For web, I prefer React.js or Angular. - Backend Development: Proficiency in Node.js, Python, or Java is essential. The backend will need to support a CMS, user management, and a question bank. - AI/ML: The platform should utilize adaptive learning techniques and predictive analytics for personalized learning experiences. - Video Hosting: Experience with AWS S3 and Vimeo API integration is a plus. - Analytics: The platform should offer real-time tracking and reporting capabilities. Ideal candi...
...modification to my Python script which presently downloads data from Yahoo and adjusts for inflation. The script needs to be able to adjust for inflation on a specified date, rather than the current date. Key Modifications: - Allow manual input for the date used for inflation adjustment. - Add two new columns to the dataset: 1. A column for the inflation ratio (%), calculated based on the market value and the adjusted value. 2. A column that calculates the result of the formula R=ax+by+cz, with a, b, c being arbitrary constants and x, y, z being data pulled from the current row. I've attached the current script and my attempts at modification. The script will be validated and tested in Jupyter Notebook. Ideal skills for this job in...
...a robust integration between Python and the NinjaTrader 8 trading platform using a DLL integration approach. The goal is to automate trading operations seamlessly, facilitate data access, and streamline account management. Key Features: 1. Automated login to NinjaTrader 8 from Python 2. A stable connection maintained by a keep-alive mechanism 3. Comprehensive account information and trading capabilities access, including: - Monitoring account balance and positions - Order management (placing, modifying, cancelling) - Receiving real-time execution updates 4. Extensive market data functionality: - Downloading historical bar data - Creating real-time price feeds - Accessing market depth information Deliv...
Seeking a freelancer to generate B2B leads targeting the Tech sector in Malaysia. Source: Leads from Linkedin. Most important is EMAIL ADDRESS. No Free email account. Targeting 1000, 5000 & 10,000 leads . PM for discussion
...transport rental business in Bali. Responsibilities Design and implement email automation strategies, including: 1. Journey Emails (Pre-Delivery, Trip, Post-Booking/Review, Re-engagement). 2. Monthly Newsletters and Seasonal Campaigns. 3. Seasonal Campaigns / Promotional Emails Key task: 1. Manage and segment subscriber lists for targeted messaging. 2. Monitor email performance and optimize for engagement. 3. Create visually appealing assets using tools like Figma. Requirements: 1. Proficiency in Mailerlite or similar tools. 2. Strong copywriting and visual storytelling skills. 3. Understanding of the customer journey and behavior segmentation. 4. Familiarity with Figma for creative design. 5. Analytical mindset to track and im...
I'm looking for a dedicated train...looking for a dedicated trainer who can guide me through 10 classes - 4 focused on Python and 6 on LangChain. My ultimate goal is to create a multi-agent project together. I have some coding experience from the past and I'm getting back into it, but I would classify myself as a beginner with Python. Your teaching should be structured as step-by-step tutorials, building my skills and confidence gradually. Key areas of focus will be: - Learning Python from the ground up - Understanding LangChain with an emphasis on process automation - Hands-on coding practice to reinforce learning I would appreciate a trainer who can make complex concepts accessible and engaging, helping me to develop both my c...
I'm looking for a Python developer who can help me code a soccer game based on this Instagram reel: Key Features to Implement: - Selection of two logos for teams - Selection of two music tracks for when a goal is scored Ideal Skills and Experience: - Proficient in Python programming - Experience with game development - Ability to understand and replicate gameplay mechanics from a video - Creative input for team logos and music tracks would be a plus
I'm looking for an experienced WordP...WordPress theme. The task involves building a new site for my travel agency, which operates land transfers and tours, using this theme. Content from an existing website will need to be migrated to the new one, including text, images, and customer reviews. Key Responsibilities: - Building a new travel agency site using the Travel Tour theme - Migrating content from an existing website - Providing links to other websites built with this theme Ideal Candidate: - Previous experience with the Travel Tour - Travel Booking WordPress theme - Strong WordPress development skills - Ability to provide links to prior work with this theme Please only bid if you have experience with this specific theme, and can provide lin...
I'm seeking a Python expert with extensive experience in working with both REST and GraphQL APIs. The project's primary focus is to ensure real-time synchronization and reporting with Google Sheets and MongoDB. Key Responsibilities: - Implement integrations with both REST and GraphQL APIs using Python. - Ensure Google Sheets and MongoDB sync in real-time. Ideal Skills: - Proficiency in Python. - Experience with REST and GraphQL APIs. - Google Sheets API and MongoDB integration expertise. - Pandas and Webscraping experience - Twitter API experience a bonus The project will involve pulling live data from API ( both REST and GraphQL) and then organising the data in google sheets t...
I'm seeking an expert who can determine the ideal number of clusters for a text datase...an expert who can determine the ideal number of clusters for a text dataset. The project involves k-means clustering using k-fold cross-validation, complete with preprocessing and evaluation in python only. Key Tasks: - Use the k-means algorithm exclusively - Conduct k-fold cross-validation - Preprocess the dataset through tokenization, stop words removal, and lemmatization - Evaluate clustering results using the silhouette score and elbow method Ideal Skills and Experience: - Proficient in Python or R - Extensive experience with k-means clustering - Strong background in text data preprocessing and analysis - Familiarity with k-fold...
I'm looking for a skilled PHP developer to create a dynamic web scraper for trading data. The scraper needs to collect price data from various sources, namely: - Stock market websites - Cryptocurrency exchanges - Forex trading platforms The data should be collected in real-time, so experience with PHP and web scraping is a must.
We need someone to scrape website data and social media posts. Please download every Tiktok post from Return Home: @returnhomenor?lang=en We will need the videos, text description, hashtags, and all comments (and their associated data). In doing so, we ask that you create an Excel spreadsheet, a folder with video downloads, and a folder of Word documents that include the comments on each video. We provide a detailed description below. 1. Excel Spreadsheet: Please create an excel spreadsheet with the following columns. Column A: Label: “Video title” In this column, please provide the video title to match the title of the video download. For example, if you look at this TikTok video (@returnhomenor/video/7164090596796157230?lang=en), you would
I'm in need of a freelancer who can assist me with data entry of realtor information (name, company name, phone number, email, city) from multiple websites onto an excel sheet. This data needs to be formatted in a way that it can easily be imported into my CRM. Key Task...entry of realtor information (name, company name, phone number, email, city) from multiple websites onto an excel sheet. This data needs to be formatted in a way that it can easily be imported into my CRM. Key Tasks: - Source realtor information from multiple websites - A combination of web scraping and manual data entry - Processing of more than 500 entries Ideal Skills & Experience: - Proficient in Excel - Experience with web scraping - Detail-oriented to ensure data accura...
...data by scraping three specific webpages and processing files from an online consultation system with restricted access (user and password protected). The dashboard must include the functionality to download PDF reports and present the data in a user-friendly and analytical format. A detailed explanation of the webpages to be scraped is provided in the attached PDF. 1. Key Responsibilities: Design and implement an online dashboard with secure user authentication and restricted access. Scrape data from three specified websites, utilizing anticaptcha to bypass CAPTCHA requirements. Integrate uploaded files from an online consultation system. Enable the generation and downloading of PDF reports directly from the dashboard. Perform d...
...a Back-End Developer to complete a Construction Project Management web platform. The front-end is mostly done, using HTML/CSS/JavaScript, and now requires a skilled professional to handle the back-end tasks, primarily using Flask (Python) and MySQL. Your key responsibilities will include: - Developing a secure user registration and login system, managing two specific user roles: Admin and General User. - Designing and building a robust MySQL database to store project and user data. - Enabling dynamic project entry by division (there are 16 divisions, each with unique input fields). - Creating an API to handle form submissions and display project metrics in real-time. - Ensuring the platform is scalable for fu...
I'm looking for a skilled WordPress developer to create a custom plugin for related posts. The plugin should be able to: - Search external websites based on a list I provide - Identify similar articles using the keywords in my site's post titles - Automatically posting 3-5 related titles with hyperlinks on the original site The ideal freelancer for this project should have a solid understanding of WordPress plugin development, web scraping, and SEO practices. Experience with creating similar plugins in the past will be a big plus. The objective is to increase my site's interactivity and improve its SEO ranking through relevant external links.
...client local machine, fetch a python script from Azure Blob Storage (my preferred remote encrypted file system) and execute it. The aim is to ensure IP protection, so any residue of the script on the client machine should be erased after the container run stops. Key Requirements: - The use of free tier or easily available technology is a must. This includes but not limited to Azure Dev Ops, GIT, and Docker. - The client machine for this project should be Windows. - The Python script needs to communicate its results back to the client machine by displaying results in a console. - Any residue of the script on the client machine should be erased Automatically within the container after the container run stops. Ideal Skills and Experience: - Proficiency ...
Title: Looking for an API Integration Expert (ChatGPT and Twitter/X) for Crypto Market Content Project Description: I am looking for a professional or developer with experience in API integration to set up an automated system that connects the ChatGPT API with my Twitter/X account. The main objective is to create and post content related to the cryptocurrency market, with a particular focus on the most relevant meme coins. I want the system to automate informative posts, price updates, trend analysis, and personalized responses to users interested in this topic. Project Requirements: API Integration: Set up and connect the ChatGPT (OpenAI) and Twitter/X APIs. Content Automation: Design a system that regularly publishes updates on the cryptocur...
I need a software engineer to create a Python SDK/package for scraping Reddit data. The package should accept either an email address or a LinkedIn profile URL as input to identify and retrieve data from a person's Reddit account. Key Responsibilities: - Build a Python package that takes a person's identifier (an email address or LinkedIn profile URL, both are suitable as options) as input. - Implement logic or utilize third-party APIs to identify the person's Reddit account. - Scrape the person's Reddit activity, including posts, comments, and likes. - Structure the retrieved data for easy access by the package user. The choice of third-party APIs for identifying Reddit accounts is open. The freelancer may choose whatever they think...
...with migrating product data from my existing website hosted on City Hive () to Shopify. Since City Hive lacks a direct export feature, this project requires scraping the product data and preparing it for import into Shopify, ensuring all essential details are included. Key Project Requirements: Data Extraction: Scrape or extract product data from the website, including product titles, descriptions, pricing, images, categories, brands, and options/variants. Example product page: Legend Vineyard Exclusives LVE Cabernet Sauvignon. Ensure that product categories (e.g., Wine, Spirits), brands (e.g., John Legend), and relevant attributes (e.g., Cabernet Sauvignon, Red Wine, Napa Valley) are captured. CSV File Preparation for Shopify: Format the data according t...
I'm looking for a skilled freelancer to set up a Mac and create an automated daily report from stock broker accounts. This project involves: - Using Python to extract and analyze stock data - Integrating with the broker API (specific API to be determined) - Incorporating volatility indicators (to be specified) - Ensuring the final product can be easily imported into Excel The ideal candidate should be detail-oriented and accurate in data reporting. Proficiency in Python and Excel is essential, as is experience in financial data analysis. If you can deliver reliable reports, I would love to hear from you.
I'm looking for a talented developer to create a bespoke LLM t...a talented developer to create a bespoke LLM tool. This tool should allow users to upload PDFs, interact with them through a chatbot interface, and receive context-aware answers based on the content of their documents. Key Features: - Text extraction from PDFs - Summarization of PDF content - Context-aware Q&A capabilities The user interface, built with and React, should prioritize visual appeal while ensuring the backend is constructed using Python. Ideal skills and experience for the job: - Proficiency in Python for backend development - Extensive experience with and React for frontend development - Prior work with LLM tools and PDF interaction - Strong ...
...you will be responsible for designing, developing, and deploying a Selenium-based API that automates job applications on various job boards. The role involves working with browser automation, API integration, and handling dynamic input requirements for multiple users. The ideal candidate is experienced in Python, Selenium, and Docker and is passionate about solving complex problems with efficient and scalable solutions. Key Responsibilities: API Development: Design and implement a RESTful API to accept job URLs and user-specific data. Integrate API endpoints for managing push notifications and responses. Browser Automation: Use Selenium to automate form filling and job application workflows. Identify required f...
I need an expert in Artificial Intelligence, ChatGPT, Python, Machine Learning: can be expert in any of them The job is to collect data from the internet (NOT from one particular website, but from the worldwide web in general) and populate an Excel sheet that I provide The data is about businesses, and will be used on a business website
...orchestrating local development and test environments Helm: For creating and managing charts for Kubernetes applications Kubernetes: The primary container orchestration platform Any one Public Cloud Platforms: STACKIT AWS (Amazon Web Services) Google Cloud Azure GitLab CI/CD: For implementing and maintaining CI/CD pipelines Terraform or OpenTofu: For infrastructure as code and management Linux: For system administration Scripting languages: Bash Python Monitoring and logging tools: DataDog Prometheus Grafana Cloud-native technologies: Service meshes (e.g., Istio) Kubernetes operators Networking technologies: Specifically for Kubernetes environments Strong focus on containerization, Kubernetes, cloud platforms, infras...
I'm seeking an AI Python Developer to create a sophisticated backlink tool. This tool should be able to track backlink quality, identify link-building opportunities and generate backlink reports. It needs to integrate seamlessly with data from Google Search Console, Ahrefs, and Moz. Key Requirements: - Expertise in Python and AI - Proven experience in developing SEO tools The ideal candidate will have a strong background in Python development and a comprehensive understanding of SEO and backlinking strategies. Your ability to create an intuitive, expert-level tool with advanced features will be key to the project's success.
I need a Python-based program that integrates with the Odds API and Airtable, calculates hit percentages for specific prop types (e.g., assists, points, rebounds, threes), and updates Airtable with the results. This program will run locally on my computer and will be delivered as an executable file .exe for windows. Key Deliverables: - A working Python script or executable that: - Retrieves data from the Odds API (player name, prop category, odds, and line value). - Filters the data based on a specific odds threshold (-400 to -1200). - Checks Airtable to avoid duplicate records (player name, prop category, and line value). - Inserts the data into Airtable if the combination of player, prop, and line doesn't already exist. - Retr...
I'm facing connection issues when trying to extract data from Python-SQLite to Power BI. The specific problem is a database authentication failure. I'm currently using username and password for authentication. Skills and Experience Required: - Proficient in Python and SQLite - Experienced in Power BI - Strong understanding of database connections and authentication - Excellent problem-solving skills to troubleshoot connection issues
Create a Simple Octoparse Template for Web Scraping Job Description: I need someone familiar with Octoparse to create a simple template that I can use to scrape a local copy of a website. This is an older website with no dynamic content, logins, or complex elements—it’s straightforward and static. Scope of Work: Set up the Octoparse template to start on a page listing all business categories. Configure the template to navigate into each category, extract business listings, and handle pagination if applicable. Capture basic business details like name, address, phone, and website (if listed). Key Information: The site is hosted locally on my computer, so I’ll run the template myself. The template should be reusable and easy fo...
I'm looking for a skilled developer to help with two tasks. Firstly, I need an information scraper built or fixed to pull numerical...looking for a skilled developer to help with two tasks. Firstly, I need an information scraper built or fixed to pull numerical values from a specific source and store them in my database. Secondly, I am experiencing issues with my billing. Card bills are being generated, but my database doesn't update on recurring charges. This issue needs to be fixed promptly. Ideal Skills: - Proficient in web scraping tools and techniques - Strong knowledge of database management - Experienced with and troubleshooting payment gateway issues - Good understanding of recurring billing systems - Ability to deliver prompt ...
I'm looking for a data scrapper who can extract specific information from Twitter and Facebook for me. Twitter Data: - I need the Username and bio, Number of followers, and Name of Twitter users. - I want this information from both personal and business profiles. Facebook Data: - I want to scrape Business profile follower links from Facebook. Ideal Skills: - Prior experience in data scraping from social media platforms - Proficiency in data analysis - Familiarity with Python or other scraping tools - Knowledge of ethical data scraping practices 800K records / 2 days delivery
I need a freelancer who can extract text content from a specific website - Google Trends. Key Responsibilities: - Extract relevant search terms from Google Trends - Organize the data in a systematic and easily understandable format Ideal Skills: - Proficiency in web scraping and data extraction - Familiarity with Google Trends - Excellent data organization skills Experience: - Previous work related to web data extraction is preferable - Experience with Google Trends is a plus
...prices and stock status on my WordPress store with a specific e-commerce site. Currently, I manually add products and check stock status but I'm looking to fully automate this process. Key Responsibilities: - Web scraping product information from a specified e-commerce site, not via API. - Daily synchronization of prices and stock status. - Importing various product details including Title, SKU, Price, Stock status, Attributes (options to pick), Description, and Images. Tools & Plugins: - The integration with my WordPress store should be done using the WP All Import plugin. Ideal Skills & Experience: - Proven experience in web scraping. - Strong knowledge of WordPress and WP All Import. - E-commerce ...
