Scraping data from websites in Python

Good day! Two tasks need to be completed: 1. Develop a product parser from an external website (10–40 thousand items, marketplace) with structured data saved in MySQL for subsequent output in WordPress. 2. Install and configure n8n on VPS, as well as organize AI content processing: prompt setup, text rewriting, image processing, SEO optimization, and text checking for AI detection. You can estimate the cost of completing both the entire project and each task separately. .

It is necessary to perform parsing from Viber channels (Total number - 49 channels, about 80 thousand subscribers).

An independent service handler for Excel files is required for the existing microservices system. The task involves creating a reliable pipeline for receiving, validating, and transforming data from tables into a structured database format. Functional tasks: Development of an API based on gRPC for receiving processing commands and returning execution statuses. Implementation of file parsing logic: reading large volumes of data (XLSX), cleaning, type checking, and mapping to business models. Implementation of a data access layer (Repository/Unit of Work) for saving results in PostgreSQL via Entity Framework Core. Ensuring thread safety and efficient resource usage (especially when processing large files). Technical requirements: Platform: .NET 10. Architectural patterns: Dependency Injection, CQRS, modular project architecture. Communication: Strictly gRPC. Working with Excel: Use of efficient libraries (e.g., EPPlus, OpenXML, or similar of your choice). Modularity: Code should be organized so that the service is easily scalable and testable. Expected results: A fully functional microservice ready for deployment in a containerized environment. A clean codebase adhering to SOLID principles. Documented .proto files. Basic unit tests for critical data processing nodes. Candidate requirements: In your response, please specify: Your experience with .NET in microservices architecture. Examples of how you organize DI and modularity in your projects. Experience with Excel libraries in .NET. Willingness to work with gRPC contracts.

Good afternoon. I need a keyword parser that outputs results through a Telegram bot. How it should work: Automatic search on 4 websites for keywords that change from time to time. Search queries are sent every few minutes. The words are uploaded in the form of a .txt file. The Telegram bot should have buttons: start bot, stop bot, download file (downloads a file with active keywords), upload file (uploads an edited file with new words). The bot should ignore previously found results, i.e., it should not indicate the same ad twice. The result comes to the bot in the form of a link with a photo, but just a link is sufficient. P.S. searching websites without API, VPS with 6TB and 50 IPs are already available. For detailed information, please contact me via private message.

Task: one dashboard with all business metrics — advertising, funnel, payments, manager performance, revenue planning. Data is pulled automatically via API. Scope: only the YCL direction (employment in Europe). Kommo has other directions — only YCL funnel deals will be included in the repository (filter by funnel/tag to be agreed upon).1. Data Sources (Integrations) Kommo CRM — leads, deals, funnel stages, responsible persons, sources, dates of transitions between stages (must keep history), reasons for refusals, custom deal fields (see point 2). Stripe — payments, amounts, statuses (success/failure/refund), linked to deals. Meta Ads — expenses, impressions, clicks, CPL, leads by campaigns (currently operational). Google Ads, Reddit Ads, LinkedIn Ads — planned; architecture — extensible connectors without core rework. SEO/organic— Google Search Console + GA4. Cross-link: traffic source → lead in Kommo → payment in Stripe (UTM, deal ID in Stripe metadata — propose the mechanism). 2. Mandatory Cuts (Deal Fields in Kommo) Each metric must be filtered/grouped by: Client Citizenship (Kenya, Nigeria, India, etc.). Residence Status: lives in their country / expat (already in Europe). These are two different segments with different cycles, conversion rates, and checks. Country of Placement / Service: Poland, Serbia, Slovakia, Germany (ZAV). Manager, team, traffic channel, period. If any fields are missing in Kommo — the executor indicates which fields need to be added, the client adds them.3. Funnel and Leading Indicators Data by funnel, for each stage — summary and leading metrics: Traffic → lead: leads, CPL by channels + day-to-day expense/click dynamics. Lead → qualification: conversion + first response speed, touches/calls to the manager per day, unanswered leads. Qualification → contract/invoice: conversion + sent offers, stalled deals (days in stage above norm). Invoice → payment: payments, average check + unpaid invoices, failed payments. Summary: revenue, ROMI by channels, run rate to monthly plan. 4. Deal Cycle Average and median lead → payment cycle (business benchmark ~4 weeks), cycle trend over time. Breakdown of cycle by stages (how many days a deal sits at each stage) — to see which stage is dragging. List of deals that have stalled at a stage longer than normal. Cycle breakdown by segments: citizenship, residence status, country of placement, manager. 5. Early Warning of Decline (Key Block) Since the cycle is ~4 weeks, today's leads = payments in a month. The system must: Compare leads/qualifications of the current week with the moving average (4 weeks) and issue an alert if there is a downward deviation: “leads -X%, with a 4-week cycle expect a payment decline in the week [date].” Build payment forecast for 4 weeks ahead from the current pipeline: deals at each stage × historical conversion of the stage × remaining cycle. Highlight in red weeks where the forecast is below plan — with time to react. 6. Additional Payments and Sales Planning In the Kommo deal card, the date and amount of the planned additional payment are stored. The system must: Collect a calendar of upcoming additional payments: total expected, by weeks/months. Highlight overdue additional payments (date passed, no payments in Stripe) — a separate list for follow-up. Calculate the monthly plan as: plan − already paid − scheduled additional payments = how many new sales are needed (in money and in deal units at average check). Weekly schedule: additional payments + forecast of new payments against the weekly plan. 7. Manager Performance Daily snapshot for each manager: touches/calls, conversations, sent offers, payments — for each day separately, with a chart over the period. Progress on personal plan compared to monthly pace (ahead / on pace / behind). Benchmarking with colleagues. 8. Visualization and Roles “Traffic lights” (green/yellow/red) for key metrics relative to norms/plans; progress scales; trend graphs; mobile adaptive. Roles: CEO — everything; COO — entire funnel and managers; team lead — their team; manager — their metrics and position relative to colleagues. 9. Reports and AI Automated reports on schedule (daily summary, weekly report) in the dashboard and/or messenger. Free-form queries (“how has CPL from Meta changed over 2 weeks?”) — LLM over the repository. Alerts in the red zone and according to the rules from points 5–6. 10. Technical Expectations and Staging Repository (PostgreSQL/BigQuery or equivalent) + ETL: Kommo webhooks + periodic synchronization (15–60 min). Frontend: custom or BI tool — propose with justification; requirements for roles, traffic lights, forecasts, and AI queries must be implementable. Stages: (1) audit and metrics map → (2) MVP: Kommo + Stripe + Meta, funnel, traffic lights, roles → (3) deal cycle, early warning, additional payments and plan → (4) SEO, AI reports, alerts → (5) new advertising channels. Payment is staged, with a demo for each stage. In the response, indicate: similar projects (end-to-end analytics), stack with justification, timeline and cost estimates by stages, monthly ownership cost (hosting, tokens, licenses).