Developer Needed for Review Aggregation Project

$250-750 USD

종료됨

게시됨

1년 이상 전

$250-750 USD

제출할때 지불됩니다

Description: We are seeking a highly-skilled Python web scraping developer to create a sophisticated review aggregation tool. The primary goal is to comprehensively scrape and aggregate reviews from multiple major platforms for specific professional websites. Primary Responsibilities: Web Scraping Tool Creation: Develop a tool that effectively scrapes reviews from multiple prominent online review platforms. Implement strategies to bypass scraping restrictions without violating any terms of service or ethical guidelines. Ensure adherence to [login to view URL] and employ delay strategies, changing user agents, etc., to minimize the risk of bans. Database Management: Design a database to store scraped reviews, associated metadata (e.g., review date, author, source), and website information (URL, active/inactive status, last scraped date). Implement an automated mechanism to regularly update each site's reviews and recalibrate the overall ratings based on newly aggregated reviews. Automatic detection of site activity: If a site becomes unresponsive over multiple scrapes or a set duration (e.g., 3 months), mark it as "inactive". Automatic Discovery and Validation: The tool should feature an automated system for discovering new professional sites related to a given industry. Integration with popular search engine APIs or equivalent to locate potential websites. Use Natural Language Processing (perhaps leveraging popular AI models) to validate that discovered sites truly belong to the targeted professional category. Automated Rating System: Design a rating mechanism that calculates an overall score for each website based on the aggregated reviews. This score should consider weighted coefficients for reviews based on their source (i.e., certain platforms might carry a different weight compared to others). Regular Updates & Maintenance: The scraping tool should be set to operate at regular intervals to ensure that data remains current. Implement checks to validate whether older sites are still active and to re-activate those that have regained activity. Friendly Web UX Features: Launch scraping, maintenance, discover a new site from a category/sector using popular search engine APIs. Add sites manually or using a CSV file, access to the database via the UX, and change the coefficients for each source regarding site rating. Required Skills and Experience: Profound expertise in Python and associated web scraping frameworks such as popular Python scraping libraries. Strong SQL database knowledge and experience with database optimization. Familiarity with NLP for content validation. Previous experience in circumventing scraping limitations without breaching ethical or legal standards. Demonstrated ability to automate complex processes. Additional Notes: Please provide examples of similar projects you've undertaken.

Python

Web Scraping

PHP

Software Architecture

MySQL

프로젝트 ID: 37085631

프로젝트 정보

82 제안서

원격근무 프로젝트

활동 중 1년 전

돈을 좀 벌 생각이십니까?

이메일 주소

프리랜서 입찰의 이점

예산 및 기간 설정

작업 결과에 대한 급여 수급

제안의 개요를 자세히 쓰세요

무료로 프로젝트에 신청하고 입찰할 수 있습니다

82 이 프로젝트에 프리랜서들의 평균 입찰은 $504 USD입니다.

@OutsourceMan

I understand that you are looking for a highly-skilled Python web scraping developer to create a sophisticated review aggregation tool. With 18+ years of experience in web and app development, CnELIndia is the perfect choice for this project. We specialize in PHP, WordPress, WooCommerce and data scraping. We have also developed software architecture before so we are well aware of the importance of having a functioning database to store scraped reviews, associated metadata (e.g., review date, author), and website information (URL, active/inactive status). Additionally, our software is set to operate at regular intervals to ensure that data remains current. Our team's greatest strength lies in its ability to automate processes for regular updates & maintenance as well as making sure older sites remain active. This makes us an ideal choice for this project as it requires regular monitoring of reviews from multiple prominent online review platforms to ensure they are up-to-date. Plus our system automatically detects site activity so we can mark inactive sites as such if they go offline after multiple scrapes or over a set duration (e.g., 3 months).

$500 USD 7일에

4.9

(569 건의 리뷰)

8.9

@AwaisChaudhry

Hi Good afternoon , I have read the brief details on your job listing . I see you have been looking for someone experienced with Python, PHP, MySQL, Software Architecture and Web Scraping. Its been 8 years since I have been working on freelancer.com, I have 9 years of experience doing similar jobs. I would request you to check my profile and review projects, feedbacks of projects related to those skills. Questions: 1. These are all the requirements of your job or do you have more? If yes, Please provide detailed requirements in chat and let me review and get back with queries. 2. Do you currently have anything done or this job has to be done from scratch? 3. What is the timeline to get this job done? 4. Are you open to use 3rd party APIS for it even if they are paid? Why Choose Me? 1. I have done more than 250 major projects only on freelancer.com. 2. I have not received a single bad feedback since last 5-6 years. 3. You will find 5 star feedback on last 100+ major projects which shows my clients are happy with my work. Portfolio: https://www.freelancer.com/u/AwaisChaudhry Timings: 9am - 9pm Eastern Time (I work as a full time freelancer) Please initiate the chat so we could discuss it in detail and we will continue from there. Thanks! Awais

$750 USD 14일에

4.9

(124 건의 리뷰)

8.6

@chirgeo

Hi there, I'm George, an experienced Python-based web crawler with a background creating sophisticated solutions for aggregating reviews from multiple platforms. I've worked with a wide range of scraping techniques, and am comfortable evading restrictions without breaching terms of service. I understand your need to build an automated system to detect, validate and review professional websites. My previous projects have included working with databases, NLP, and automating complex processes. To giver you added peace of mind, I have already created a custom web scraping tool. In order to get started, I would need to gain more insight into the following points: 1. Who are your target users for the review aggregation tool? 2. Which platforms are you targeting for scraping reviews? 3. What automated mechanisms will be used to regularly update and recalibrate review ratings? 4. How is the automated discovery and validation system planned to work? 5. What are the design requirements for the web UX features? Let me know when you have the answers to these questions and I can provide a more accurate quote. I look forward to hearing from you. Best, George

$750 USD 3일에

5.0

(164 건의 리뷰)

8.1

@umg536

Hi there, I'm bidding on your project "Developer Needed for Review Aggregation Project" Being an expert in Python and matlab programming I can do this project for you. please leave a message on my chat so we can discuss the budget and deadline of the project. Also, have a look at my profile for my reviews and portfolios. Thanks Note: This is not an automated bid and i'm ready to discuss your project right now. .. .

$750 USD 5일에

5.0

(45 건의 리뷰)

7.3

@devops724

Hello there! My name is Hossein, and I'm a highly experienced Python developer with a proven track record of delivering successful projects. I've worked on a variety of projects including web scraping tool creation, database management and automatic discovery and validation of new professional sites related to a given industry. I believe that I have the skills necessary to successfully complete your project. If you give me the chance to work on this project, I guarantee you will not be disappointed with the results. I would be more than delighted to discuss this project further with you if given the chance. Please don't hesitate to contact me if you have any questions or would like to see more examples of my work. Best regards!

$500 USD 7일에

5.0

(96 건의 리뷰)

7.4

@uumarkhalid31

Hello! You are looking for a highly-skilled Python web scraping developer to create a sophisticated review aggregation tool. As a Python developer, I have a lot of experience in web scraping which you can see in my profile. Please feel free to ask any questions regarding the project. I am looking forward to working with you. Thank you-

$350 USD 4일에

4.9

(263 건의 리뷰)

7.3

@AITSoft

Hi, I saw that you need help with Python, PHP, Software Architecture, Web Scraping and MySQL. I have 6 years of experience working on these frameworks. I believe I can help you with it. I would request you to have a look at my porfolio, customer feedbacks from my profile. If you find me worthy of doing your job. Please start the chat and lets discuss it. Regards Shamshad

$750 USD 11일에

4.9

(46 건의 리뷰)

6.9

@kozaktr

Hello. As you can check on my profile, I have done tons of web scraping projects so far and as reading your requirements, I became sure that I am the fittest person for your project. I can do it in Selenium (python or c#) or Puppeteer (NodeJS) and you are free to choose any of these scraping tools. Python, Web Scraping, PHP, MySQL and Software Architecture is my mail skill sets. I am available at any time, so feel free to contact me.

$500 USD 7일에

4.8

(64 건의 리뷰)

6.8

@ayesha0124

Hi, I can help u as i have done several similar jobs related to Python, Software Architecture, PHP, MySQL and Web Scraping, I have read the details and furthermore discuss about it, plz discuss with me in detail. Regards

$750 USD 7일에

5.0

(24 건의 리뷰)

6.7

@c0909h1179

Hi there, I am the best here! Please check out my profile and see what others have to say about the work I've done related to the skills you're looking for. Hope to work together soon. Thanks!

$500 USD 7일에

5.0

(41 건의 리뷰)

6.7

@datascientist90

I'm a senior Python & ML developer and owner & founder of Dedeoglu Dev Company. Kindly send me a message to get in touch with me, Thanks, Yusuf

$500 USD 7일에

5.0

(27 건의 리뷰)

6.6

@Devrits

Yo! I am skilled Python software engineer with skills including MySQL, Python, Web Scraping, Software Architecture and PHP. - web scraping, data mining, data extraction, data transformation, - writing scripts and utilities, - image / file downloading. - scraping info from websites into SQL-like DB or any other format (txt, CSV, EXCEL, JSON etc). - file transformation. Web scraping technologies: - Selenium Webdriver - create autotest, scraping information from different websites - Scrapy - a gathering of information variety of websites like amazon, airbnb, etc (python) - Xpath - analysis and using XPath to navigate in an XML document - beautiful soup + requests - collecting information from the given websites. Skillset: - Java – advances (new tools implementation, web applications creation, bug fixing, code review) - JavaScript – basic (new script creation, support existing solution). - Groovy – basic (creation simple CRUD web application using Grails Framework). - Python (knowledge: PyGame, Django, scrapy, beautiful soup) - Spring (Core, MVC, Security, JPA) – new application creation, support existing web applications. - Oracle SQL – creating new scripts, support and tuning existing scripts. - HTML + CSS – simple page creation / support existing Please contact me to discuss more about this project. Thanks & regards

$250 USD 4일에

4.5

(70 건의 리뷰)

7.0

@simonklemp

Hello David, I hope this message finds you well. I will be able to help you with your project. I read through the description and it sounds like I am just the right person for it. I have the right skill set with sophisticated experience in Python, Web Scraping, and Software Architecture. As a next step I would like to discuss some further questions with you. Either over the chat or a call, whatever you prefer. For my references and qualifications: I graduated in Computer Science with highest grades from one of the most reputable universities in Germany. Currently, I am enrolled in the Master's program. During my studies, I already published a paper on the subject of Artificial Intelligence. Further, I managed to get a lot of hands-on experience by working in several companies that operate in the field of Computer Science. As a native German speaker who was living in Canada for a year, my level in English is proficient/fluent. Further, I am in the preferred Freelancer program, making me part of the top 3% on the platform. Many successful past projects can be seen on my profile page. I am looking forward to working with you and if you have any further questions I would be happy to answer them. Best regards, Simon

$550 USD 5일에

5.0

(26 건의 리뷰)

6.1

@DevBench

Greetings. I am an experienced web scraping developer with over 7 years of expertise in Python and database management, and I'm confident in creating the sophisticated review aggregation tool you're seeking. To clarify the requirements, I kindly request the following information: 1. Which online review platforms will be the primary focus for scraping? 2. Are there any specific NLP models you prefer for content validation? 3. Do you have a preferred interval for automated scraping and updating? 4. Are there any particular search engine APIs you'd like to integrate for site discovery? Let's connect for a quick chat session to discuss the project's intricacies. Looking forward to connecting with you to craft an effective review aggregation tool.

$500 USD 7일에

5.0

(15 건의 리뷰)

5.7

@nisthark

Hi Sir, My name is Nisthar. I am a fullstack developer. I have more than 8 years of experience in nodejs, react, nextjs, svelte, vue, php, python, wordpress, laravel etc. I can do this very quickly and efficiently. Pls contact me, Thanks.

$300 USD 5일에

4.7

(22 건의 리뷰)

5.8

@NadMax

Hello. Skilled Python web scraping developer with 8+ years of experience, I can adeptly assist you in creating a sophisticated review aggregation tool for professional websites. To proceed, I kindly request the following: 1. Could you provide more insights into the specific review platforms you intend to aggregate from? 2. Are there any particular NLP models you're considering for content validation? 3. Could you elaborate on the expected user experience features you'd like to implement? 4. What specific criteria do you have for determining the weighted coefficients for different review sources? Let's engage in a detailed discussion to ensure a seamless and efficient aggregation process. I'm excited to collaborate on crafting this powerful review tool that meets your requirements and enhances user experience.

$500 USD 7일에

5.0

(11 건의 리뷰)

5.3

@Demenntor

Dear Client, I am excited to propose my expertise as a highly-skilled Python web scraping developer for your sophisticated review aggregation tool project. With a strong background in web scraping, database management, and automation, I am confident in my ability to deliver a comprehensive solution that meets your requirements. To move forward, I would appreciate the opportunity to discuss your project in more detail and gain a deeper understanding of your specific requirements. Kindly share any additional project specifics or considerations, and I'll be happy to provide further insights and propose a detailed plan of action. Thank you for considering my proposal. I am looking forward to the possibility of working together to develop an advanced review aggregation tool that will elevate your objectives. Best regards, Lalit

$720 USD 7일에

4.9

(38 건의 리뷰)

5.1

@dataspro

Nice to meet you bangalter,The requirements of your project match my areas of work and skills, to introduce myself. My name is Anthony Muñoz and i am the lead engineer for DS Pro IT agency. I have worked for over 10 years as a Full-Stack and software development engineer and have successfully done multiple jobs. It will be a pleasure to work together to make your project. Feel free to discuss about the project with me, greetings.

$908 USD 7일에

4.4

(5 건의 리뷰)

5.5

@PowerRapto

Hello David, Going through your job posting, it seems like you are looking for a Web scraping developer who can scrape date from online review platforms. I'm an expert in Python scraping using Selenium library and have strong background in scraping prominent websites with several high-stand technologies like bypassing Captcha, avoid IP blocking system using Proxy services and delay strategies changing user agents, etc. As I have worked on similar projects previously, I am excited to apply on this project. If we together, I'm 100% sure that It will exceed your expectation. We can talk through chat for further discussion. Regards, Jörg-Peter Ewald

$500 USD 5일에

5.0

(4 건의 리뷰)

4.9

@syedazainab339

Hi, nice to meet you I am skilled mobile engineer with skills including Web Scraping, PHP, Software Architecture, MySQL and Python. I have read your task description carefully and I understood what you want. I can provide the best result. I am free to start work right now. I would also like to discuss your project in more detail via chat. Warm Regards, Syeda

$450 USD 5일에