Technical task (TT) for creating a program for data collection and processing
1. Project description: Develop a program for automatic data collection from the websites https://www.flashscore.com/ and https://www.transfermarkt.world/, which inputs data into Google Sheets.
2. Table format:
Date/number | Team1 | Team2 | OddsForFavoriteWin | OddsTotalOver2.5 | OddsTotalOver0.5 1H | ValueTeam1 | ValueTeam2
3. Data sources:
Flashscore – for collecting information about matches and odds.
From here we take the team name and league for Transfermarkt.
Transfermarkt – for collecting information about team values.
4. Match selection criteria:
- Select matches where there is a team with odds for winning less than 1.75.
- Collect only matches from certain leagues (definition of the league list).
- Authorization on Flashscore for access to extended data.
5. Functional requirements:
- Automatic web scraping for data collection.
- Inputting results into Google Sheets via API.
- Collecting data on odds and team values according to selection criteria.
Setting up automatic data collection at a specified time:
- First data collection at 20:00 for the next day.
- Data update at 10:00 AM on the same day.
- Ability to set up a schedule through a task scheduler or the program's built-in function.
6. Technical requirements:
- Access to Google Sheets via API (authorization via OAuth 2.0).
- Logging the process to track operation and errors.
- Support for scheduling task execution (via cron or built-in scheduler).
7. Testing: Verification on several matches to confirm the correctness of data collection.
8. Deadline: 2 days.
9. Budget: $250.
