Situation:
The client needs a tool for automatic information gathering from an online store: name, photo, price, description, stock, category.
What I did:
- Wrote a parser that traverses the entire catalog
- Automatic photo upload
- Retrieved price, specifications, availability, SKU
- Handled pagination and complex HTML structures
- Bypassed anti-bot protection (timings, headers, dynamic delays)
- Exported data to PostgreSQL and Google Sheets
- Implemented a launch schedule via cron + reports
Result:
- The client automatically updates prices and products without human intervention
- Savings of ~60 hours of work per month
- Ability to import data into their website/CRM
- Stable operation of the parser 24/7
Technologies:
Python, Selenium, BeautifulSoup, lxml, Asyncio, PostgreSQL, GSheets API.
The client needs a tool for automatic information gathering from an online store: name, photo, price, description, stock, category.
What I did:
- Wrote a parser that traverses the entire catalog
- Automatic photo upload
- Retrieved price, specifications, availability, SKU
- Handled pagination and complex HTML structures
- Bypassed anti-bot protection (timings, headers, dynamic delays)
- Exported data to PostgreSQL and Google Sheets
- Implemented a launch schedule via cron + reports
Result:
- The client automatically updates prices and products without human intervention
- Savings of ~60 hours of work per month
- Ability to import data into their website/CRM
- Stable operation of the parser 24/7
Technologies:
Python, Selenium, BeautifulSoup, lxml, Asyncio, PostgreSQL, GSheets API.