API Scraper Needed

$250-750 USD

Fullført

Lagt ut

omtrent 2 måneder siden

$250-750 USD

Betalt ved levering

I'm looking for a proficient web scraper to collect data from two e-groceries. Key Requirements: Create an executable scraper in Python that can extract the SKU (product) list from the following Indonesian online grocery mobile apps: - Alfagift ([login to view URL]) - Klikindomaret ([login to view URL]) The scraper must be IP-block tolerant, meaning if the scraper gets blocked, execution can be continued from the point where the last row of data was successfully extracted. Scraper Functions Function: list_category_tree Input: competitor_name (alfagift | klikindomaret), existing_csv (optional - CSV file to append to for continuing execution, if the scraper stopped or got IP blocked) Output: CSV of category tree with the following columns (scrape_date, URL, category name, position index, parent_category (null if top level)) Function: extract_sku Input: competitor_name, location (latitude and longitude, mandatory), category (optional - extract SKUs from all categories if null), sub_category (optional - extract all SKUs from all subcategories if null), existing_csv (optional - CSV file to append to for continuing execution, if the scraper stopped or got IP blocked) Output: CSV of SKUs with the following columns (category, sub_category, brand, sku_name, description, competitor_name, datetime, location, url, nsp, psp, stock_count (if available), popularity_index (if available), weight (if available). If one SKU is contained in more than one category/sub_category, present the same SKU in different rows. Unit tests The scraper must include unit tests that cover main and error test cases, including optional parameters Submission The following must be submitted: - Scraper code and unit tests - README file for running and deploying the scraper - Output of all SKUs from the 2 online grocery apps specified above Project Acceptance Testing The scraper output will be manually verified within 24 hours of submission and SKU list must be 95% accurate. The SKU list is the most important aspect of the scraper output. Variations in other data for each SKU like stock count, popularity index and prices may vary and higher levels of discrepancy (~25%) during manual verification is acceptable. (edited) Ideal Skills: - Proficient in web scraping tools and techniques - Experience in data extraction and processing - Able to deliver data in a well-structured CSV format Please provide examples of previous similar projects you've completed.

Prosjekt-ID: 38757652

Om prosjektet

77 forslag

Eksternt prosjekt

Aktiv 2 måneder siden

Ønsker du å tjene penger?

E-postadresse

Fordeler med budgivning på Freelancer

Angi budsjettet og tidsrammen

Få betalt for arbeidet ditt

Skisser forslaget ditt

Det er gratis å registrere seg og by på jobber

Tildelt til:

@MohammadKassem01

Leveraging over 4 years of experience in data mining and web scraping, your project will be in safe hands with me. I am passionate about programming and automation, this means my proficiency in Python, is top-notch. I guarantee you high quality and accurate work because of my keen eye for detail. Let's have a discussion, I can start right away!

$250 USD om 5 dager

5,0

(34 omtaler)

5,0

77 frilansere byr i gjennomsnitt $554 USD for denne jobben

@ahmadayaz

With over a decade of experience, I have a proven track record in successfully handling challenging projects in Full Stack Development, WordPress, Mobile App Design, SAAS Development, and Brand Identity. I have completed over a thousand projects for organizations in the enterprise and medium tiers. I am skilled in web scraping tools and techniques, data extraction, and can provide structured CSV outputs. I am eager to collaborate on your project to create a Python web scraper for extracting SKU lists from Alfagift and Klikindomaret e-grocery apps. Please provide specific project details, timeline, and budget considerations to tailor my approach accordingly and ensure a successful outcome. Let's discuss further to bring your ideas to life efficiently.

$675 USD om 5 dager

4,9

(311 omtaler)

8,7

@MashoodurRehman1

I am Python web scraper and data extraction expert, proficient in web scraping tools and techniques. I have experience in building executable scrapers in Python to collect data from various e-commerce platforms. I can deliver well-structured CSV outputs and have a track record of completing similar projects successfully.

$637 USD om 5 dager

4,9

(87 omtaler)

7,6

@uumarkhalid31

General proposal 1: I specialize in web scraping jobs like this one. You can see on my profile that I have completed many similar jobs. I am a professional with a rock solid reputation and over 10 years of experience specifically in writing scraping code like this. I know what I can and cannot do, and I won't waste your time by saying I can do something that I can't. Because I exclusively write scraping code I know what to expect and I will always live up to what I tell you, and will also deliver a higher quality final product than my competitors. Regards Umar

$600 USD om 4 dager

4,8

(300 omtaler)

7,4

@ZohaibRoy

⭐⭐⭐⭐⭐ Design Efficient API Scraper for E-Groceries Task ❇️ Hi My Friend, I hope you're doing well. I've reviewed your project requirements, and I see you're looking for a proficient web scraper to collect data from Indonesian e-groceries. Look no further; Zohaib is here to assist you! My team has already completed 50+ similar projects in building complex scrapers using Python and other technologies. I will create an executable scraper in Python capable of extracting SKU lists from Alfagift and Klikindomaret with IP-block tolerance. Additionally, I'll include unit tests, a README for deployment, and ensure the scraper meets your requirements. ➡️ Why Me? I can effortlessly handle your scraper project as I have 5 years of experience in web scraping, data extraction, and processing. Not only this, but I also have a strong grip on handling IP-block scenarios, ensuring continuity from the last successful extraction point. ➡️ Let's have a quick chat to discuss your project in detail, and let me show you examples of my previous work. Looking forward to discussing this with you in chat. ➡️ Skills & Experience: ✅ Python ✅ Web Scraping ✅ Data Processing ✅ CSV File Handling ✅ Error Handling ✅ Unit Testing ✅ Data Extraction Techniques ✅ Python Libraries (e.g., BeautifulSoup, Scrapy) ✅ Proxy Management ✅ Task Scripting ✅ IP-Block Mitigation ✅ Real-time Data Collection Waiting for your response! Best Regards, Zohaib

$1 000 USD om 7 dager

5,0

(83 omtaler)

7,0

@Rekhathakur

Hello, With over a decade of experience in web and mobile app development, I am confident I can deliver an exceptional solution for your API scraping needs. As a proficient Python developer, I have extensive knowledge and experience in data extraction techniques and packaging them into well-structured CSV formats. This aligns perfectly with your requirements for an API scraper. In addition to my technical skills, I am accustomed to running unit tests as part of the project development process, ensuring that all conceivable cases are considered, and the end product is impeccably functional. I've worked on numerous projects involving web scraping, showcasing my meticulousness and ability to maintain scraper functionality despite IP blocks - another critical point in your project layout. Much like the scraper functionalities you've outlined, I have built applications that enhance inferior speed performance in resource degradation by executing from where the last parameter was captured successfully. Providing A1 deliverables with quality SKU lists falls directly in line with my core values as indicated by my numerous previous project completions to total client satisfaction. Look forward attention to detail, open lines of communication and on-time delivery when we work together on this important task ahead! With Regards! Rekha

$750 USD om 7 dager

4,5

(94 omtaler)

7,7

@hakimfrelancer

Hello There, I read your project carefully. I have 8+ years’ of Experience on Development of Website, AI ChatBot & Plugin. Also I am Expert on Integration of AI, ChatGPT, API, Payment Gateway, Mail Setup & Management. I am also specialize in Server Setup, Security, Performance Optimization, and Experienced in Error Debugging & Fixing, Site Migration, Malware Cleaning Related to WordPress, Shopify, WooCommerce, HTML, PHP, Laravel, Node.js, Python, Linux Tasks. I'm a Linux-Based VPS and Hosting Manager with Expertise in Managing Cloud Services such as AWS, Contabo, DigitalOcean, Hostinger, GoDaddy, Namecheap and More… ✅ I am VERIFIED Freelancer. You Can Always Trust Me. Best Regards Hakim

$500 USD om 7 dager

4,7

(163 omtaler)

7,2

@priyank086

Hello, Greetings of the day!! I have understood your requirement in the project that **you need a Python scraper to collect SKU data from two Indonesian online grocery mobile apps, Alfagift and Klikindomaret, with the ability to handle IP blocks and generate structured CSV outputs.** I have 10+ years of experience in Nodejs, Python, AI/ML, Web Designing/Development, HTML5, PHP, E-commerce, and Mobile Application development using Ionic, React Native, and Flutter, etc. I am able to do this work as I have the necessary knowledge. So I think I can handle your requirement decently. If you need any clarifications on my bid, please let's talk about it. I hope to hear from you soon so we can get things done. Thanking You, Regards, Priyank Bhargava

$250 USD om 7 dager

4,8

(71 omtaler)

6,4

@dahani9091

Greetings, I'm Adel, a seasoned web scraping specialist with extensive experience in data extraction and processing. I'm excited about this project and confident in delivering accurate and well-structured data. With my proficiency in Python and web scraping techniques, I can create an efficient and IP-block tolerant scraper that seamlessly extracts the required SKU data from Alfagift and Klikindomaret. I'll implement robust error handling and unit tests to ensure the scraper's reliability and maintainability. My approach involves leveraging advanced scraping techniques to navigate the mobile app interfaces and extract the category tree and SKU data, including all relevant details like brand, description, pricing, and stock information. I'll ensure the data is delivered in a clean CSV format, adhering to the specified column structure. To demonstrate my expertise, I've successfully completed similar projects, including scraping e-commerce platforms and mobile apps for product data. I'm well-versed in handling dynamic content, pagination, and anti-scraping measures. May I inquire about any specific challenges or requirements you anticipate with these mobile apps? I'd be happy to discuss my strategies

$750 USD om 7 dager

4,7

(21 omtaler)

6,5

@RMMcontractors

Hello Yves S., I read you project description carefully and I am excited to express my interest in this project. With several years of experience in full-stack development, I am confident in my ability to bring substantial value to your project. And also I have various experiences in PHP, Data Entry, Excel, Web Scraping, Data Mining. I am a passionate developer and always strive to implement best practices on all my projects. I can start the work for you immediately and will do my best for you and your project. Please message me if you are interested in me. Thank You

$450 USD om 7 dager

5,0

(6 omtaler)

5,4

@TechizeBuilder

Hi, We would like to grab this opportunity and will work till you get 100% satisfied with our work. We are an expert team which have many years of experience on PHP, Data Entry, Excel, Web Scraping, Data Mining Please come over chat and discuss your requirement in a detailed way. Regards

$400 USD om 7 dager

4,7

(24 omtaler)

5,4

@birukGetnet17

Hello Sir, I'm an Android reverse engineer, skilled in working around app security to extract APIs, even when there are protections like encryption or ID checks. I can also set up rotating proxies and use multiple requests at once to avoid hitting IP limits, which keeps the process smooth and fast. Recently, I created a scraper for grocery apps in Saudi Arabia, and I can use the same techniques for your project. Let me know if you're interested in discussing this further. Looking forward to chatting. Best regards, Biruk G.

$500 USD om 1 dag

5,0

(21 omtaler)

5,1

@manpreetkaur991

With my broad skillset and extensive experience in web and mobile development, I feel tailor-made for your API scraping project. While Python is not explicitly listed as one of my primary languages, it's a language I am comfortable working in and have used in various projects. Being well-versed with tools and techniques for web scraping, I can create an executable scraper that proficiently extracts the SKU data you need from Indonesian online grocery mobile apps - Alfagift and Klikindomaret. Moreover, my profound understanding of data extraction and processing will ensure that your scraper is thorough and efficient. As seen from my previous work, I'm highly proficient when it comes to producing structured output in CSV format. For example, I have recently developed a scraping system for a large e-commerce retailer where accuracy was of paramount importance - a challenge that I overcame successfully. In addition to this, my natural propensity for clean coding enabled me to deliver effective unit tests for past projects – covering both main and error test cases just as you've requested. Rounding it all up, trust me to provide what you need – high-quality scraper code with full unit tests, clear instructions on running and deploying the scraper (in the README file) and eventually correct output of all SKUs from Alfagift and Klikindomaret.

$500 USD om 7 dager

4,6

(17 omtaler)

5,5

@sarbtech123

With over four years immersed in Python development and web technologies, I am both proficient and highly experienced in the exact skills you require for this project. My extensive knowledge of effective web scraping tools and techniques coupled with a deep understanding of data extraction and processing will enable me to create a reliable executable scraper that will successfully extract the SKU list you need from the Alfagift and Klikindomaret online grocery apps. I understand the importance of accuracy within the scraper output and this is something I take very seriously. As such, I thoroughly test all my work, providing unit tests that cover main and error test cases - including those of optional parameters - to ensure a high level of integrity within the scraper functions. You should know that I have hands-on experience with tools like Zapier which demonstrates both my ability to automate workflows as well as my attention to delivering structured data in CSV format. Moreover, accepting projects and delivering value has been the hallmark of my career so far because I believe in precision, innovation, collaboration and client satisfaction. Your project requires a focused and meticulous approach to ensure not just its success but maintainability. I'm confident in my ability to provide not just exceptional quality but also an apt README file for easy running and deploying of the scraper.

$500 USD om 7 dager

4,9

(32 omtaler)

5,0

@larsenm2020

Having completed previous projects in data extraction, processing and web scraping successfully and within the agreed timelines PLEASE HAVE A CHAT IT WILL BE WORTH YOUR TIME, I am confident that I can deliver exceptional results for your API Scraper project. My proficiency in Python especially when it comes to building scrapers will ensure that the scraper I develop is not only functional but executable as per your requirements. Notably, a key requirement you stated in the project description was an IP-block tolerant scraper, able to continue execution from the point where the last row of data was successfully extracted. This is an area where I excel because I not only anticipate potential challenges like IP blocking but also take preventative measures to overcome them. In addition to my skills, my approach to problem-solving and meticulousness in work ensures that I cover all main and error test cases during unit testing. The importance of a well-documented README file for running and deploying the scraper has always been a top of mind for me.

$350 USD om 4 dager

5,0

(21 omtaler)

4,4

@dataspro

Nice to talk you yvessy, After reading in detail the requirements of your project and concluding that they match my areas of knowledge and skills, I would like to introduce myself. My name is Anthony Muñoz and I am the lead engineer for DS Pro IT agency. I have worked for over 10 years in Backend and software development and have successfully done multiple jobs. It will be a pleasure to work together to make your project a reality. Please feel free to contact me. I´m looking forward to working with you. I really appreciate your time and remain attentive to any request or question. Greetings

$966 USD om 7 dager

5,0

(6 omtaler)

4,6

@djsmoothy

As an experienced full-stack developer versed in various frameworks and proficient in languages including C# and Python, I am well-equipped to meet your needs for a scraper for theAnfagift and Klikindomaret apps. With over a decade in software engineering, I have become intimately familiar with techniques for web scraping, data extraction, and processing - all critical skills for this project. In addition, my understanding of IP-blocking tolerance means I will create a solution that allows progress to continue smoothly, regardless of potential blocks. Furthermore, my extensive experience with databases such as MSSQL and MySQL adds value to your project's needs to create structured CSVs. These skills will allow me to deliver on each aspect of this project, including the creation of unit tests to cover main and error test cases - guaranteeing the performance reliability essential for long-term use. Note: Unfortunately, I do not have any previous similar projects exactly like this one. However, this should not be seen as a disadvantage as I am fully confident in my abilities to meet and exceed your project's requirements with deep attention to detail gained from similar database management tasks throughout my career.

$750 USD om 7 dager

5,0

(3 omtaler)

4,5

@Uness33

Hi there! I'm Unes, a Python expert with a knack for web scraping and data extraction. Your project to scrape SKU data from Indonesian e-grocery apps is right up my alley. Here's how I plan to tackle it: I'll develop a robust Python scraper that navigates IP blocks seamlessly, ensuring uninterrupted data collection. By leveraging libraries like BeautifulSoup and Selenium, I'll extract the category tree and SKU details efficiently. My approach includes implementing a checkpoint system to resume scraping from the last successful point, minimizing data loss. To ensure accuracy, I'll incorporate unit tests covering all scenarios, including edge cases. The scraper will output well-structured CSV files, ready for analysis. Could you share more about the specific data points you prioritize? This will help tailor the scraper to your needs. Looking forward to discussing how I can bring this project to fruition!

$750 USD om 7 dager

4,9

(7 omtaler)

4,4

@derekjlaporte

As a seasoned Full-Stack Developer and skilled Data Scientist, I bring a well-rounded expertise to your API scraping project. Over the past 5 years, I've crafted numerous robust web applications and developed strong acumen in data-driven decisions. In particular, my Web Scraping knowledge combined with proficient use of Python sets me up perfectly to tackle your Indonesian e-grocery task. My experience with RESTful APIs aligns perfectly with your requirement for a well-structured CSV format for the scraped data. I can ensure that the data from both the Alfagift and Klikindomaret apps is efficiently extracted into structured CSV files, preserving columns as specified. My in-depth knowledge of Python allows me to create an executable scraper that not only scrapes the data but also tolerates IP-blocks with the option of continuing execution from the point of interruption. Pitch continued: Moreover, my Full-Stack background will prove extremely useful when it comes to creating your scraper's unit tests, guaranteeing robustness and meticulous coverage for all main and error test cases. My track record in creating machine learning models will also be beneficial as we can set up systems to verify scraped data accuracy in real-time for the SKU list you deem so important. Don't hesitate to reach out and let's embark on this journey together!

$500 USD om 7 dager

5,0

(4 omtaler)

4,5

@LionsDenGaming

Hello! I’m excited about the opportunity to build a proficient web scraper for your project. I have extensive experience with web scraping tools and techniques, and I can create an executable Python scraper to extract SKU lists from Alfagift and Klikindomaret effectively. The scraper will include a list_category_tree function to build a comprehensive CSV of the category hierarchy, and the extract_sku function will enable detailed SKU extraction based on specified criteria. Both functions will support resuming from the last successful extraction point, ensuring that interruptions due to IP blocks won’t compromise your data collection. I will also implement unit tests to cover main and error cases, ensuring reliability and accuracy in the scraping process. You can expect well-structured CSV outputs with the required data fields. Let’s chat further to discuss your requirements and how I can best meet your needs for this project! Vernon.

$500 USD om 7 dager

5,0

(3 omtaler)

4,6

@ManzoorH

Hello I am full time freelancer with more than 8 years of experience in .NET, Winforms, WPF, SQL Server, .NET Core, WEB APIs, MAUI, MAUI Android and iOS, WCF, Entity Framework, Blazor, React and NODE.JS. I have very solid experience with POS, Banking systems, Inventory Systems, Sales Systems. Please come on chat and lets discuss the project scope and flow in details. Thank you for reading my proposal. Regards Manzoor

$700 USD om 7 dager