Web Scraping

There is an active production platform with a catalog and automatic updates of external offers and prices. Stack: — Node.js / TypeScript; — PostgreSQL; — existing price refresh service and cron; — separate ready Python module for validation and selection of offers; — staging and production. It is necessary to make targeted improvements to the existing price refresh pipeline without completely rewriting the backend. MANDATORY SCOPE 1. Integration of the Python module — The Python module remains a separate component; — returns a structured result: offers, selected offer, statuses, and risk flags; — Node.js validates the result and performs a write to the database; — provide for error handling and partial/failed runs; — the legacy pipeline is not turned off until QA is completed. 2. Launch refresh by list Add launch: — by one slug/id; — by the provided list of slug/id. Assume CLI or existing service API. A new user interface is not required. 3. Shadow Mode New results must be recorded separately and not affect production until QA. Shadow fields required: — price; — selected offer ID; — direct URL; — offer status; — risk/QA flags; — checkedAt; — engineVersion. 4. Expanding the existing offers table Add: — source; — external_offer_id; — last_seen_at; — last_checked_at; — engine_version; — risk flags or storage in existing JSON; — unique constraint to protect against duplicates. It is not required to create a new parallel offer system if the existing table can be safely expanded. 5. UPSERT, STALE, and DB transaction Replace the current DELETE → CREATE scheme: — UPSERT existing and new offers; — offers missing in the full successful snapshot are translated to STALE; — in case of API error, partial result, or incomplete snapshot, active offers should not become STALE; — updating offers, selected offer metadata, and shadow fields for one model is performed within one DB transaction; — in case of an error, a full rollback is performed. 6. Canonical-safe refresh Price refresh should not change: — brand; — reference; — model; — collection; — name/title; — slug; — descriptions; — images; — SEO fields. Only offer, price, and shadow data are updated. 7. Preserving current cron logic Preserve: — existing cron; — rolling batches; — cooldown; — checking PRICE_REFRESH_MIN_DAYS before calling the external API; — legacy production pipeline until Shadow QA is completed. 8. Audit output One option is sufficient: — shadow columns in the existing admin table; or — CSV export. Minimum data: — model/reference; — production price; — shadow price; — delta; — production/shadow URL; — status; — risk flags; — checkedAt; — engineVersion. A new complex dashboard is not required. 9. Staging and QA — DB migrations; — staging deployment; — smoke test on 5 provided models; — then Shadow Mode on approximately 50 models; — fixing technical errors identified during these runs; — brief documentation of the Python → Node.js contract and rollback procedure. OPTIONALLY ASSESS SEPARATELY Simple technical promotion without a new UI: — promotion of one model by slug; — promotion of a list of slugs; — transferring confirmed shadow values to production; — technical check after rollout. RESULT — Pull Request; — DB migrations; — working integration Python → Node.js; — Shadow Mode; — UPSERT, STALE, and transactional update; — launch by slug/id; — staging deployment; — smoke-test results; — brief documentation; — at least 7 days of bug fixes for the implemented scope after acceptance. IN RESPONSE, INDICATE 1. Fixed price for the mandatory scope. 2. Separate cost for the promotion mechanism. 3. Timeline. 4. Hourly estimate. 5. When you are ready to start. 6. Experience with PostgreSQL transactions, migrations, and ingestion pipelines. 7. What questions need to be clarified before starting. 8. Whether staging, QA, migrations, and bug-fix period are included. Template responses without specific estimates will not be considered. Access to production is not provided at the first stage. Work begins with limited code review and staging.

A centralized server system for collecting and storing data from Planfix, 1C, Meta Ads, and Google Ads is needed, as well as a web dashboard for displaying and analyzing this data. All data, change history, calculations, and aggregated metrics must be stored exclusively in the server database. The dashboard should not store or duplicate business data. It must retrieve the necessary information from the server database via API according to user requests and display it in the form of KPIs, charts, tables, and detailed reports.

We are looking for support for a project based on Yii , we need to make edits and improvements to the database, there is partially a connection with the previous contractor .....................

It is necessary to migrate the database from CRM G-PLUS to MyChatBot Database volume - 26 thousand leads 2 funnels - Call center and Sales department with their own funnels Lead cards (besides name and number) have many different fields Leads also have voice recordings of calls. These also need to be transferred I expect an approximate amount and implementation timeline from the candidate

Create a dashboard for monitoring and analyzing the performance of the company's location network (branches) in Google Business Profile (GBP) through the official Google Business Profile API. Process via a script based on Google Apps Script (link to Google Sheets). Record data in Google Sheets (which serves as a database for Looker Studio). Update: Daily (with an indication of the last update date). Create a Google Cloud service account. The script runs once a day (trigger at 03:00 AM) and sends a request to the GBP API. It retrieves metrics for the previous day for each location (locationId). Records data in a flat format (row = unique combination of Date + Branch ID + Metrics). Key Performance Indicator CardsCard NameGBP MetricDynamic FormatProfile ViewsImpressions (Search + Maps)Percentage %, Sparkline (blue)CallsLocal Services Phone CallsPercentage %, Sparkline (green)Website ClicksWebsite ClicksPercentage %, Sparkline (purple)Direction RequestsDirection RequestsPercentage %, Sparkline (orange)Average RatingAverage Review RatingAbsolute change (e.g., +0.1), Sparkline (yellow)New ReviewsNew Reviews CountPercentage %, Sparkline (turquoise)