Development of a high-performance automotive marketplace parser
Task: Create a stable and fast tool for monitoring the car market with automatic data export in Excel (CSV) format for further price analysis.
Implemented solution and technical features:
Asynchronous architecture: By switching to httpx.AsyncClient and asyncio, the script performs network requests without blocking the thread, ensuring high data collection speed (50+ ad pages in less than a minute).
Data shift protection: Isolated parsing "from the object" (card-wise) has been implemented. The script dynamically checks for the presence of characteristics before writing, so the absence of price, mileage, or fuel type in individual ads does not cause errors and does not shift columns.
Memory optimization: Data collection is optimized through fast lists of dictionaries, and the Pandas DataFrame is initialized only once at the end of the process. This eliminates system load when parsing large volumes.
Ready business result: Data is exported to CSV with the correct utf-8-sig encoding, ensuring proper display of Cyrillic characters in Microsoft Excel without additional settings.
Technology stack: Python, HTTPX (AsyncClient), BeautifulSoup4, Pandas, Asyncio.
Implemented solution and technical features:
Asynchronous architecture: By switching to httpx.AsyncClient and asyncio, the script performs network requests without blocking the thread, ensuring high data collection speed (50+ ad pages in less than a minute).
Data shift protection: Isolated parsing "from the object" (card-wise) has been implemented. The script dynamically checks for the presence of characteristics before writing, so the absence of price, mileage, or fuel type in individual ads does not cause errors and does not shift columns.
Memory optimization: Data collection is optimized through fast lists of dictionaries, and the Pandas DataFrame is initialized only once at the end of the process. This eliminates system load when parsing large volumes.
Ready business result: Data is exported to CSV with the correct utf-8-sig encoding, ensuring proper display of Cyrillic characters in Microsoft Excel without additional settings.
Technology stack: Python, HTTPX (AsyncClient), BeautifulSoup4, Pandas, Asyncio.
Kyiv 
Available for hire
Offer work