Build a Scraper and integrate into new MongoDB Database within OctoberCMS
$250-750 USD
Suljettu
Julkaistu yli 8 vuotta sitten
$250-750 USD
Maksettu toimituksen yhteydessä
Looking for a freelancer to develop a web scraper tool and integrate it with our current OctoberCMS installation.
For this project, prospective freelancers must understand MongoDB and have the ability to write a web scraper that can pull up to 2000 points of data from a HTML page into multiple seperate documents in the MongoDB database, which the developer will create according to our spec.
It is expected that administrators will be able to enter a URL to be scraped in the backend, at which point the profile linked to the URL will be either updated, or created if none exists, in the databases, and an entry will be added to the administration module. If updated, it is expected that specific data points will be added to track over-time progress of specific statistics. Full details on all data that we wish to scrape and include will be provided.
The URL scraped is from a single domain, and the overall design and HTML will remain similar, however specific details such as the order or availability of any number of the 2000 points of data may change on any particular page load.
This project is for the scraper, admin module, and database only, and no front-end or graphic design is required. Please place the words "Ironic Tortilla" in your response or it will be removed for "not matching brief". Responses without direct reference to MongoDB and freelancer's ability with MongoDB will be removed for "not matching brief".
This is a single part of a 9-part project, and prospective freelancers may have future opportunities available based on their abilities and success. Freelancers are expected to potentially work with other freelancers or employees assigned to the project to produce expected integration between projects. The entire project is expected to complete within June, and may have up to 5 different freelancer's working on it at any given time.
All work is done via GitHub, and freelancers are expected to follow proper workflow (staging to live) and properly comment their changes as necessary.
My name is Mike and I’m from UK. I work with individual clients and also provide outsourcing services for a number of UK and USA based agencies. Your project description sounds interesting to me and I do have skills & experience that is required to complete this project. I can show you some examples of my work. Please contact me to discuss your project.
Hello,
Can we discuss in detail ?.Do you have any timeframe?
We welcome you to join hands with us as we think we are capable enough to start this project and we give you surety of answering all your queries.
We are waiting to discuss in detail about the project so that we can provide you quality oriented results.
We are a team of 100+ developers doing mobile and web application development.
We have more than 5 years of experience
Thanks
Genius Team
This is exactly what I already implemented on my current project, where I implemented a scraper to scrap properties listing from several properties website. My scraper basically scrap properties listing from multiple different properties websites, and note that regardless of different website layout and functionalities, my scraper is generic handle to handle it.
Note: My scraper currently using MongoDB as well, so it'll work great here
Just some of the features of my existing scraper:
- scrap list of properties listing urls from several properties websites
- from properties listing urls, scrap list of properties urls and its details
- post to our property listing website once scrap
- update/delete properties status in DB when the properties are sold/rented
- flexible enough to be enough when to run for each job
- generic enough that it can be configured for other projects
Note that, depending on the complexity of the website to be scraped, time it takes can differ.
If it's just a simple website, then it shouldn't take long, since I already have the framework implemented :) Since some websites implemented some security features to prevent others from scraping their website
Hope to hear from you soon! If you want to preview of the scraper that I've implemented, just let me know and itll be my please to share. Thanks!
Dear You,
I am really interested to work on this project,
I can start the work now , and can provide the best services from my end.
Please come on chat to discuss more about the project.
Thanks!
Ironic Tortilla
I would love to do this great project. I am about to finish the online course "Data Wrangling in MongoDB".
I have no professional experience as a freelancer about these topics, but want to build a successul frelancing business. That's why I take the course started to freelance about data wrangling.
I am a full time software engineer for over 10 years so I am able to do your work professionally.
Please sent me a sample document. I would love to do it before choosing my bid.
See my work than decide.
Regards,