Data exchange via JSON from 1C to WordPress

Technical task for data exchange via JSON on WordPress

Goal:

Ensure correct synchronization of product stock between 1C and WordPress through a JSON file. Currently, there is a problem with a large volume of data — the file hangs and does not update information on the site when there are many products.

Task:

Implement a mechanism on the WordPress side that will allow processing large JSON files, breaking the processing into smaller parts (batches) and ensuring stable data synchronization without overloads.

Main requirements:

1. Trimming data in the JSON file:

   • Implement splitting a large JSON file into parts (batches) for step-by-step processing.
   • Create a mechanism that allows processing files in parts, for example, 100–200 products at a time, to avoid timeouts or server overloads.

2. Background execution of data exchange:

   • Set up background processing for product batches
   • Each batch must be processed separately, and after processing, a signal must be sent to process the next batch until all products are processed.
   • Ensure that data is correctly updated even with large volumes of products.

3. Logging and error handling:

   • Implement logging of the data exchange process: record the date and time of the start/end of each batch, the number of products that were successfully updated, and any possible errors.
   • In case of errors, add the ability to reprocess only those batches that were not processed correctly.

4. Updating products on the site:

   • For each product, the SKU must be checked. If the product already exists on the site, the stock quantity and, if necessary, the price must be updated.
   • If the product is not found in the WooCommerce database, it must be created as a new entry (if necessary).

5. Data consistency between JSON and WooCommerce:

   • Ensure that the format of the JSON file received from 1C corresponds to the fields used in WooCommerce (SKU, quantity, price).
   • If necessary, perform data validation before processing each batch to avoid issues with incorrect data.

6. Server technical limitations:

   • Set PHP limits on the server to ensure sufficient time for processing each product batch:
     • max_execution_time = 300 (or more). - here at the developer's initiative
     • memory_limit = 256M (or more). - here at the developer's initiative
   • If necessary, develop a mechanism for distributing large volumes of data across multiple sessions to avoid overloads and timeouts.

7. Periodic synchronization:

   • Set up automatic launch of the synchronization process  (for example, every 2 hours).
   • Provide the ability for site administrators to manually start the data exchange process with 1C.

8. Interface for displaying synchronization status:

   • Create an interface on the WordPress admin page (or in a special plugin) to view the exchange status:
     • Time of the last successful import.
     • Number of processed batches.
     • Logs of successful updates and errors.

Input data:

• JSON file transmitted from 1C (structure: SKU, stock quantity, price).
• API or FTP access for transferring the JSON file to the WordPress server (provided by 1C).

Deadline:

[Specify a specific deadline for task completion]

Result:

• Stable data synchronization with 1C, even with a large number of products.
• No hangs or interruptions in the exchange process.
• Complete log of the process for tracking the status of data import.
• Ability to manually start the exchange and view the synchronization status.