eBay scraping
Asynchronous scraper for laptops on eBay, built on Python, Playwright, SQLite, and Google Sheets API.
This project automatically collects data about laptops from eBay, using dynamic page rendering through the proxy API Scrape.do. The scraper extracts detailed product information such as title, price, condition, shipping cost, seller location, number of units sold, seller reviews, refurbished product status, and product URL.
Features:
- Asynchronous scraping using Playwright
- Dynamic JavaScript rendering
- Proxy API integration
- Automatic pagination handling
- SQLite database for tracking page progress
- Automation of export to Google Sheets
- Continuation of scraping from saved page
- Extraction of structured product data
Technology stack:
- Python
- Playwright
- AsyncIO
- Requests
- SQLite
- Google Sheets API
- gspread
- dotenv
Workflow:
1. Load environment variables
2. Initialize SQLite database
3. Load rendered eBay pages via Scrape.do
4. Analyze product cards
5. Extract product data
6. Save products to Google Sheets
7. Save current page progress
8. Continue until all pages are fully scraped
The project is designed for scalable scraping and automation tasks in e-commerce.
This project automatically collects data about laptops from eBay, using dynamic page rendering through the proxy API Scrape.do. The scraper extracts detailed product information such as title, price, condition, shipping cost, seller location, number of units sold, seller reviews, refurbished product status, and product URL.
Features:
- Asynchronous scraping using Playwright
- Dynamic JavaScript rendering
- Proxy API integration
- Automatic pagination handling
- SQLite database for tracking page progress
- Automation of export to Google Sheets
- Continuation of scraping from saved page
- Extraction of structured product data
Technology stack:
- Python
- Playwright
- AsyncIO
- Requests
- SQLite
- Google Sheets API
- gspread
- dotenv
Workflow:
1. Load environment variables
2. Initialize SQLite database
3. Load rendered eBay pages via Scrape.do
4. Analyze product cards
5. Extract product data
6. Save products to Google Sheets
7. Save current page progress
8. Continue until all pages are fully scraped
The project is designed for scalable scraping and automation tasks in e-commerce.