Goal:
Develop an automatic parser that will extract coin addresses from (https://dexscreener.com/new-pairs?dexIds=raydium&maxAge=24&minLiq=1000&order=desc&profile=1&rankBy=activeBoosts) Dexscreener and save them in a convenient format for further use.
Functional requirements:
1. Automatic data collection:
• The parser should automatically access the specified page and extract coin addresses from the list of new token pairs that meet the specified parameters (dexIds=raydium, maxAge=24, minLiq=1000, order=desc, profile=1, rankBy=activeBoosts).
2. Update frequency:
• The parser should update the data at a specified frequency (for example, every 15 minutes or hour) to ensure the information is current.
3. Data storage:
• Save to the API we have already created, or directly connect to the database. The connection code and functions to avoid duplicates are already available.
4. Error handling:
• Implement mechanisms for error handling and retries in case of a failed request to the site.
• Log errors and notify responsible parties about any issues that arise.
Technology requirements:
• Programming language: Python, Node.js, or another suitable for web scraping.
• Use of libraries for web scraping (for example, BeautifulSoup, Scrapy for Python)
Example - https://dexscreener-ai-data-scraper-crypto-dex-analytics.en.softonic.com/chrome/extension