product data scraoed

I have 2 websites:  [login to view URL] [login to view URL] These stores dropship products from a variety of suppliers. The suppliers are  [login to view URL] [login to view URL] [login to view URL] And more will be added after this project.  Thus far we have had this script scrape the products from the supplier websites (jbmetro/sweetcraft), and upload them to the shopify stores automatically. All snacks were uploaded to snackland, and all products were uploaded to grocery.  The current scraper I'm using is available here. Most of the functionality is available but I want to expand on it and ensure it is working, hence the need for an update. You can use this code as a base and expand on it.  [login to view URL] The aim of the scraper is to keep accurate pricing (prices may change due to sales/inflation) and to keep accurate product history. I want the inventory/pricing updated everyday at 7am, 11pm, and 4pm, in the GMT+10 Time Zone. Products may also be added, or deleted from the suppliers website. These changes will need to be reflected on our end. So if a product is added to jb metro it is added to our store at the next update, and if a product that is removed from jb metro, it is taken off the store.  A lot of the suppliers sell cases of products (12/24 packs). The example product on the side by jb metro is a protein bar that comes in a box of 16. This is because it has (x16) in the title. The price is also for a 16 pack. For my stores, I want to sell the 16 pack, but I also want to sell an individual bar.  So for this product, when it is uploaded to my store, it should have the option to buy 1 bar or 16, which can be done by using a variant. The price should also reflect it, but i will talk more about that later.  Here is an example of a product with a variant.  [login to view URL] Product Title step by step:  Product titles must have the case amount removed. This will be put as a variant. Example before: 34G LIFESAVER FRUIT TINGLES (x36) Example after: 34G LIFESAVER FRUIT TINGLES Product titles must have the weight (often in grams/kilograms) at the end of the title. Example before: 34G LIFESAVER FRUIT TINGLES Example after: LIFESAVER FRUIT TINGLES 34G They should be changed from all in capital letters or lower case letters to having the first letter of each world capitalized.  Example before: LIFESAVER FRUIT TINGLES 34G Example after: Lifesaver Fruit Tingles 34G Now the PRODUCT TITLE is complete Product description:  Product description should have the product name in it and use chatgpt.  They must be at least 50 characters.  The prompt you want to give it is:  Please write an SEO rich product description for **insert product name** in 50-100 characters.  Product Image.  All products need images. Images are generally fine, regardless of whether it shows one product or the full case.  The only issue is watermarks. All jb metro product images have watermarks, and some bonefect and sweetcraft have watermarks. Watermarks is any product image that has ANYTHING on top of it.  Jb metro has a faint overlay  Bonefect often has box of ___ on top of it. Solution: the solution in the code is to use an api with a watermark remover. You can use whatever solution like searching google for an alternative similar image or taking the first listings image when you look up the product name on google.  Vendor field: The vendor field in shopify is the Brand name. You can use the method in the current code or use chatgpt api.  Tags by collections: The product should be tagged with all categories it came from. E.g. if you take 1KG AURORA ESPRESSO 100% ARABICA BEANS (x1) From JbMetro, it comes from GROCERIES - COFFEE - and then BEANS. All of these should be tagged as individual tags. So the product will have one tag called groceries, one called coffee and one called beans. This applies for all websites.  Upc/barcode: All products need a barcode, that is specific to that product/variant. On JBmetro it provides an inner and outer barcode where available. The inner barcode is the individual product and outer is the case. For products without barcodes, the script currently gets the top search from google by searching the product name + barcode and takes the first number result. You can look at the script for more detail. Still however some products are missing there value  SKU: Should be a 6-9 digit number which is NOT shared with any other variant/product.  Variants: As what i said on page 1, most products should have an option to buy 1 or the full case. The case variant should show the case amount as the name which depends on the product. If the product is just 1 being sold individually, the the price needs to be divided by the case value, 0.50 is added and then multiplied by 1.3.  For case products they are just sold for 50c more and then multiplied by 1.3 The listing for 85G EQ LEAN PROTEIN COOKIE COOKIES N' CREME (x12) with the price of $33.49 will have 2 options, one for one cookie and one for the box of 12.  The individual cookie variant will have its own barcode, and sku. The price is 33.49, so we divide that by the box amount for the per product cost price of an individual cookie. 33.49/12 = 2.79.  We then add 0.50  2.79+0.50 = $3.29  Then we multiply by 1.3.  3.29*1.3 = 4.27.  That is the selling price for an individual cookie.  For the case variant, it is just 50c added,   33.49+0.5= 33.99  And then multiplied by 1.3 33.99*1.3 = 44.187 Sales channel - You need to include all sales channels in the options.  Product locations - Each supplier has there own location on shopify. Stock in shopify should show up in its designated stock location Troubleshooting - overriding ability  While this is all good, the supplier's system we get products from have errors themselves and nuances we have to fix. For example, bonefect sells [login to view URL] which doesnt mention the case value in the title.  [login to view URL] has the case value, but we cant sell indivudal sour straps,  Therefore, we need a way to override the product titles, price, image ect. In other words, we need a way to make sure a product does NOT update for specific products. Im not sure how this can be done… Maybe a dashboard that when a product gets updated there, it updates both snackland and grocery go?