A parser needs to be developed to collect products from the manufacturer's website https://come-for.com/ua/, which gathers products from the online store and exports them to Google Sheets in a specified structure, so that the content manager can further work with this data without manually copying products from the site and then importing these products into our online store made on the CMS Khoroshop.
What needs to be parsed:
- Product name
- Brand
- Category on the manufacturer's website
- Type of product, if it can be determined
- Link to the product
- Article or product code, if available on the site
- Price
- Currency
- Availability, if available on the site
- Main product photo
- Product photo gallery
- Product description
- Short description, if available
- Product specifications
- Sizes, modifications, options, if they exist
- Data on height, firmness, filling, materials, load, and other parameters, if they are available in the product card
- SEO data, if available in the code or on the page
- Date of parsing
- Product processing status
Exporting to Google Sheets:
Data needs to be recorded in a Google spreadsheet according to a ready-made structure that we will provide to the executor.
The table already has a large number of columns for products, including:
article, parent article, product name, modification name, brand, section, price, old price, currency, display, availability, photo, gallery, link, date of addition, HTML title, META keywords, META description, H1, supplier, product description, short description, description for marketplaces, specifications, sizes, AI status, AI generation date, AI error, and other fields.
Not all fields need to be filled in at the first stage. The main task of the executor is to correctly collect the available data from the manufacturer's website and accurately place it in the corresponding columns of the table.
Important:
The parser should not directly add products to the site, but should only collect data from the manufacturer's website and export it to Google Sheets.
Further editing of products, adaptation of categories, rewriting of descriptions, AI processing, and import into Khoroshop will most likely be carried out separately by the content manager.
Logic of the parser's operation:
- Launching the parser manually or through a simple command
- The parser goes to the specified site
- Collects a list of products
- Goes into each product card
- Collects all available data
- Forms a correct data structure
- Records the result in Google Sheets
- Does not duplicate products upon repeated launch
- Has the ability to update existing products by link or article
- Should log errors if a certain product could not be processed
Desired update logic:
Two modes need to be anticipated:
- Full parsing
Collect all products from the site and record them in the table. - Updating existing products
Upon repeated launch, the parser should check if the product is already in the table. If the product exists, only the variable fields need to be updated: price, availability, photo, specifications, or other data that has changed.
If the product is new, it should be added as a new row.
Categories:
On the manufacturer's website, categories may differ from the categories on our site. For example, the manufacturer may have categories "topper", "futon", "thin mattress", while on our site this may be one category "Thin Mattresses".
At the first stage, it is necessary to export the category as it is indicated on the manufacturer's website.
It is also desirable to add a separate column for the future category of our site, but it can be filled in manually by the content manager.
Photos:
It is necessary to collect:
- Main product photo
- All gallery photos
- Links to photos in separate columns or in one field through a separator
- It is desirable to foresee the possibility of uploading photos to Google Drive or another storage in the future, but at the first stage, links to photos are sufficient if they are stable
Specifications:
If the specifications are displayed on the site in a table or blocks, they need to be saved as structured as possible.
It is desirable to implement one of the options:
- Separate columns for key specifications
- A general column "Specifications" in the format parameter: value
- A combined option, if it is technically feasible
Specifications need to be stored in such a way that they can later be adapted to the structure of Khoroshop.
Technical requirements:
- The parser must be stable and suitable for repeated use
- The code must be clear and structured
- The source code must be handed over after the work is completed
- A short instruction for launching must be provided
- It must be described what dependencies need to be installed
- Error logging must be anticipated
- Possible site limitations, pagination, lazy-load, dynamic loading of photos or specifications must be taken into account
- The solution must work correctly with the Google Sheets API
- Authorization to Google Sheets must be implemented securely
- It is desirable that the tool can be run locally or on a server
Possible stack:
The stack can be proposed by the executor. If it can be done through N8N (it is already on our server), then that would be ideal.
Or Python, Node.js, or another reliable solution would also be suitable.
Preferred options:
Python + requests / BeautifulSoup / Selenium / Playwright + Google Sheets API
or
Node.js + Puppeteer / Playwright + Google Sheets API
The final stack can be agreed upon with the executor if they explain why their option is better for this task.
What needs to be obtained as a result:
- A working parser for the site (we will provide the URL to the executor). For now, we are doing it for one specific site.
- Automatic export of products to Google Sheets
- Correct filling of the main columns of the table
- Protection against product duplication
- The ability to restart
- Error log or a separate sheet in Google Sheets with processing statuses
- A short instruction for launching
- Transfer of the source code
- Comments in the code or a short technical explanation of the logic
- Recommendations from the executor on scaling for other sites
Acceptance criteria:
The work is considered completed if:
- The parser successfully collects products from the competitor's site that we will provide to the executor.
- The table includes names, prices, photos, links, categories, descriptions, and specifications of the products
- The data is recorded in the required columns of Google Sheets
- Upon repeated launch, products are not duplicated
- New products are added as separate rows
- Existing products can be updated
- There is a clear instruction on how to launch the parser
- There is source code
- There is an explanation of what to do if the site changes its structure
- The executor showed a test result on real products
It is important to think through the architecture of the solution not only for a one-time script but as a tool that can be adapted for other sites and scaled.
In the future, the following functions may be needed:
- Parsing other manufacturers
- Regular price updates
- Monitoring new products
- Monitoring changes in the assortment
- Price comparison
- Automatic launch on schedule
- AI processing of descriptions
- Preparing the file for import into Khoroshop
Requirements for the executor:
We are looking for a specialist who has real experience in:
- Parsing online stores or manufacturers' websites
- Working with Google Sheets API
- Processing product catalogs
- Working with photos, specifications, prices, and product modifications
- Creating stable parsers that can be run repeatedly
- Setting up logging and error handling
It will be an advantage if you have experience with e-commerce products, furniture, mattresses, Khoroshop, or preparing product feeds.
What to write in the response to the task:
Please, in your response, immediately indicate:
- What stack you propose for implementation
- Whether you have experience parsing online stores
- Whether you have worked with Google Sheets API
- How you plan to avoid product duplication
- How you will process photos and galleries
- How you will log errors
- How much time is needed for the first working version
- What is the estimated cost of implementation
- Whether you will be able to adapt the solution for other sites later
- Examples of similar works, if any
The structure of the Google Sheets file where the products need to be parsed is here - https://docs.google.com/spreadsheets/d/1NaEaFHfMIVW_YUg7JOYbm4SuMcIcxd7xIdkaZZ-UHZg/edit?gid=2050477948#gid=2050477948
The manufacturer's website Come-For - https://come-for.com/ua/
Initially, a test should be done on several products. After checking the structure and quality of the data, a full parsing of all products can be launched.
Expected result:
Not just a one-time script is needed, but a quality internal tool for automating the collection of products from the manufacturer's website into Google Sheets, which can later be used to prepare products for import to the site.
