Integration of KeyCRM with BigQuery for e-commerce
Automated system for importing, processing, and analyzing orders from an online store with integration of KeyCRM and Google BigQuery.
Full synchronization of orders, products, payments, custom fields, and marketing data has been implemented.
Data from KeyCRM undergoes cleansing, normalization, linking with additional entities, and is exported to BigQuery for report generation, analytics, and BI.
Key features:
Import of orders, products, customers, managers, payments, and marketing data from KeyCRM via OpenAPI.
Processing of custom fields of orders (custom_fields) with support for values for each order.
Storage and updating of data in MySQL (Django ORM) with support for migrations and idempotent logic.
Export of orders to Google BigQuery with automatic creation and updating of the table schema.
MERGE operations for UPSERT in BigQuery: new and updated orders are synchronized without duplication.
Local mirror of orders for quick analytics and data reconciliation.
Support for complex relationships: products in an order, custom reasons for cancellation/exchange/return, manager comments.
Django admin panel with convenient viewing and editing of all entities.
Technologies:
Python 3.12, Django 5.x, MySQL, Google BigQuery, REST API (KeyCRM), Celery, Docker.
Full support for migrations, idempotent updates, transactions.
Logging, error handling, automated tests.
Result:
The system allows the business to obtain up-to-date, clean, and structured data about orders for analytics, reporting, and BI, automate export to the cloud, track reasons for returns/cancellations, and generate reports based on custom fields and marketing channels.
Full synchronization of orders, products, payments, custom fields, and marketing data has been implemented.
Data from KeyCRM undergoes cleansing, normalization, linking with additional entities, and is exported to BigQuery for report generation, analytics, and BI.
Key features:
Import of orders, products, customers, managers, payments, and marketing data from KeyCRM via OpenAPI.
Processing of custom fields of orders (custom_fields) with support for values for each order.
Storage and updating of data in MySQL (Django ORM) with support for migrations and idempotent logic.
Export of orders to Google BigQuery with automatic creation and updating of the table schema.
MERGE operations for UPSERT in BigQuery: new and updated orders are synchronized without duplication.
Local mirror of orders for quick analytics and data reconciliation.
Support for complex relationships: products in an order, custom reasons for cancellation/exchange/return, manager comments.
Django admin panel with convenient viewing and editing of all entities.
Technologies:
Python 3.12, Django 5.x, MySQL, Google BigQuery, REST API (KeyCRM), Celery, Docker.
Full support for migrations, idempotent updates, transactions.
Logging, error handling, automated tests.
Result:
The system allows the business to obtain up-to-date, clean, and structured data about orders for analytics, reporting, and BI, automate export to the cloud, track reasons for returns/cancellations, and generate reports based on custom fields and marketing channels.
Mukachevo 
Available for hire
Offer work