Hybrid Workflow (Scraping + Claude 3.5 API) for mass generation of product cards

Доброго дня! 3-тє місце соло на AI Agent Olympics Hackathon (AI Week 2026 Milan, найбільша AI-подія Європи, 731 команда). Production multi-agent + RAG системи на n8n + Claude - основний стек, MSc Strategic PM, PRINCE2.

По питаннях:

Cloudflare bypass: Scrapfly або ZenRows як основний шлюз через HTTP Request ноду (residential IP rotation + browser fingerprinting, ~65% domain coverage в 2026). Bright Data як fallback на складних доменах. ScrapeOps теж робочий варіант, але дешевший Scrapfly зазвичай дає кращу success rate на Cloudflare/DataDome.

Стабільний JSON від Claude: використовую нативний Structured Outputs (output_config.format з JSON schema) - це офіційна фіча Claude API, не prompt engineering. Гарантує валідний JSON під schema, наступні ноди в n8n не падають. Плюс fallback: JSON Parse нода з try/catch + retry з temperature 0. До речі, для production рекомендую Sonnet 4.6 або Opus 4.7 замість 3.5 - кращий JSON compliance та дешевше за токен на 4.6.

Орієнтовно: 7-10 днів, 15 000 грн.
…
Портфоліо у профілі

Hello, Georgiy, I implemented my last project using scrapling and LLM integration.
Let's go step by step:
1. For integration with Google Sheets, you can use the node with the official API.
2. Bypassing CAPTCHA, there are currently two approaches that work:
1) Use scrapling with their built-in DynamicFetcher with stealthy settings.
2) Use CDP with CloakBrowser — the most reliable option.
3. You can connect in three ways:
1) Write a lightweight backend and call it (reliable and stable)
2) Write your own node (time-consuming)
… 3) Use the Execute Command node (but this is less stable than the first option)
4. For LLM responses, there are built-in validations for structure output. You can also connect via openrouter (they have additional protection for this).
So, my recommended option is to create your small backend and call it using n8n. This fits well with the logic you described.
I would be happy to work with you!

Show original

Translate

Good day, Georgiy!

I specialize in developing automated AI/workflow systems, integrations, and building pipelines on n8n using LLM models, parsers, and third-party APIs. I work on tasks where data processing stability, scalability, and structure control between nodes are important. Your case looks like a well-thought-out production pipeline, and this is exactly the type of task where n8n fits very well as an orchestration layer.

To provide you with an objective estimate on timelines and costs, I would like to clarify a few points:

— What is the estimated processing volume planned (products/donors per day)?
— Are all donors uniform in structure, or will separate mapping/parsing be needed for different sites?
— Is the XML/YML feed formed for a specific marketplace or under your own standard?
… — Is caching/saving of intermediate results needed (for example, parsed raw data or AI output)?
— Is the VPS already set up for n8n, or does this also need to be configured?

Regarding your questions:

To bypass Cloudflare in n8n, I would use ScrapeOps or ZenRows with proxy rotation and browser rendering. If the donors are more complex — Playwright/Puppeteer through a separate microservice that n8n calls via webhook. This provides a significantly more stable parsing flow in production.

For the Claude API, I would use strict JSON mode under schema validation: clear system prompt + predefined JSON schema + an additional validation layer in n8n (IF/Function node), so that no incorrect response breaks subsequent nodes. For critical structures (categories, tags) — also fallback/retry logic.

After your responses, I will be able to accurately assess the scope of work, propose an optimal architecture, and provide realistic timelines and budget.

Georgiy, I suggest discussing the details in private messages — I can help assemble not just a workflow, but a stable production-ready pipeline that will scale without constant manual adjustments.

Show original

Translate

Доброго дня
Мене звати Дмитро. Маю досвід з n8n, AI automation, Claude API, парсингом, workflow orchestration та e-commerce автоматизаціями.
Ваш pipeline виглядає дуже грамотно побудованим — особливо подобається підхід із JSON-only AI flow та розділенням логіки по етапах.
Можу допомогти з:
— побудовою та стабілізацією n8n workflow
— інтеграцією Google Sheets/Drive API
— Claude API orchestration
— Cloudflare bypass parsing
— XML/YML feed generation
… — error handling / retry logic
— queue processing
— VPS deployment та proxy setup
Відповіді на питання:
1. Для Cloudflare parsing у n8n найкраще використовував би ZenRows або ScrapeOps + fallback strategy через rotating proxies/browser rendering. Для складних кейсів — headless browser node (Playwright/Puppeteer через external worker).
2. Для Claude API використовував би strict JSON schema approach:
— system prompt із жорсткою структурою
— JSON mode / prefilled response template
— validation node після Claude
— fallback parser + auto-retry при invalid JSON
— enums/allowed values для категорій і тегів
Так workflow не буде ламати наступні вузли.
По строках: орієнтовно 1–3 тижні залежно від складності парсерів та кількості donor sources.
Бюджет: після перегляду повного ТЗ.

Hello! I have relevant experience with similar tasks, and I can help with your project. I can quickly familiarize myself with the requirements, propose the best solution, and execute the work thoroughly. I am ready to discuss the details and get started.

Show original

Translate

Good day! 👋

Regarding your questions:

To bypass Cloudflare — ZenRows or ScraperAPI via HTTP Request node with proxy rotation.
Stable structure from Claude — system prompt with a strict JSON schema + temperature: 0 + validation in Function node with a retry on invalid JSON.
We will implement everything — all 4 stages from reading Google Sheets to the finished XML/YML feed. We are ready to start immediately. We will discuss the details in private 🤝

Show original

Translate

Hello! We are a team of developers with 4 years of experience in bot development, process automation, and professional data parsing in Python. Hybrid workflows based on Claude 3.5 are one of the best ways to effectively extract and structure frequently changing information. We will take on the full cycle: from writing a reliable scraper (scripts to bypass blocks) to integrating a neural network for intelligent analytics of the collected content. Since we are a team, you will receive a tested and fault-tolerant turnkey solution with a well-thought-out architecture. Let's discuss the data volumes and target sites in private messages for precise system configuration!

Show original

Translate

Hello! Our team has 4 years of experience in automation, data parsing, and developing intelligent bots. We specialize in Python and building complex information gathering systems, so integrating Claude 3.5 for processing unstructured data in a hybrid workflow is our core task. We understand how to optimize prompts, bypass website protections using proxies, and set up flexible content filtering logic through the Antropic API. We will provide clean code, high parsing speed, data protection, and stable architecture operation regardless of changes in the structure of target resources. We are ready to discuss the pipeline logic and output data format right now. When would it be convenient for you to chat?

Show original

Translate

Good day, I can implement everything turnkey without any problems. I have over 5 years of experience in automating business solutions using Python, have worked with large brands, and developed more than 50 automations, with and without AI. I would like to take a closer look at the technical task; if it is written using AI, 70% will need to be redone since they often do not select the right technologies or solutions. I will guide you through each stage and answer any questions. Feel free to reach out!

Show original

Translate

Hello.

I can set up a workflow for you. Write to me, and we will discuss.

Show original

Translate

We already have a nearly ready similar solution for AI generation of cards and feeds, which can be quickly adapted to your tables, donors, and marketplace rules. ))

In terms of timing, I would estimate about 5-7 days for the first stable phase.
Regarding the budget - starting from 18,000 UAH for a working conveyor in n8n with logs, structured JSON, and export to XML or YML.
6,500 UAH will likely only be enough for a prototype without a proper error handling buffer for parsing.

I won't be able to switch to Telegram within the first contact - the marketplace may ban both parties.
We can agree on everything here.

… For Cloudflare, I would use ZenRows, ScrapeOps, or Apify with browser rendering, proxy rotation, retries, and result caching.

Regarding the Claude API - due to the strict data schema.
I would set up separate validation nodes after each AI step, JSON schema, a limited list of categories from dictionaries, a retry request for invalid structures, and a fallback scenario when a field is not defined.
This way, the subsequent n8n nodes won't break due to an extra comma or unexpected format.

> Question 1 - Is the final feed format for a specific marketplace already defined in your specifications, or will we need to verify the mandatory fields during implementation?
> Question 2 - How many donors and approximately how many products need to be processed in one run?

Similar cases from Ingello:
> https://business.ingello.com/vorfahr - automation with AI and data processing
> https://business.ingello.com/prime-eva - e-commerce and systematic work with product data
> https://business.ingello.com/fractal - multi-agent processes and complex automations

Main page for FLH:
> https://systems-fl.ingello.com/ua

I can take the first phase - to assemble a working scenario in n8n, connect the table, parser, Claude, JSON validation, error logging, and test feed export.
In touch ))

Show original

Translate

Hello! I have reviewed your project and am ready to start working immediately. I guarantee excellent quality and promptness.

Show original

Translate

Hello. Four stages with Cloudflare bypass and three consecutive Claude nodes in JSON mode is not a typical n8n flow; here you need to think about retry logic and edge case handling at the data level, not just connecting nodes. Over the past year, I have built two similar pipelines for e-commerce: one using ScrapeOps + n8n for 800+ SKUs daily, and the other with ZenRows + OpenRouter for multi-core parsing. Both produce YML feeds without manual intervention. For Cloudflare - ZenRows with JS rendering, because ScrapeOps often fails on complex challenges. For stable JSON from Claude - a separate system prompt with an example (few-shot) plus a validation node using a zod schema before passing it on; if Claude returns broken JSON - the node catches it, logs it, and retries with a temperature of 0.1. Regarding timelines - 3-5 working days, as the technical specification is already ready, but Cloudflare bypass always requires testing on real donors. Question: how many donor sites are approximately in the pool and are there any among them that use PerimeterX or DataDome? This will affect the choice of gateway and the number of test iterations.

Show original

Translate