Market Data Ops: collection and caching of Forex quote history
Data Parsing
Any backtest starts not with the strategy, but with the data. To ensure that the backtester is not dependent on manual uploads and random files, I have moved the quote history to a separate service: it collects Forex data from Dukascopy, keeps it fresh, and caches it locally. Now the verification always relies on one verified database.
What's inside:
- Downloading quotes from Dukascopy by pair, dates, and timeframes, in the background with progress.
- A single local history database in one structure for all backtests and exports.
- Re-downloading with overlap from the last candle: history is not lost, duplicates are removed.
- Auto-updating once a week on schedule.
- Viewing the database composition (pairs, timeframes, ranges, number of candles), M1-W1 charts, and export.
On top of the data, there is a REST API, so the backtester accesses the history directly.
#Python #Forex #FastAPI #DataEngineering #Trading #API #Automation #Dukascopy #LightweightCharts #RestAPI #TradingView #PineScript
What's inside:
- Downloading quotes from Dukascopy by pair, dates, and timeframes, in the background with progress.
- A single local history database in one structure for all backtests and exports.
- Re-downloading with overlap from the last candle: history is not lost, duplicates are removed.
- Auto-updating once a week on schedule.
- Viewing the database composition (pairs, timeframes, ranges, number of candles), M1-W1 charts, and export.
On top of the data, there is a REST API, so the backtester accesses the history directly.
#Python #Forex #FastAPI #DataEngineering #Trading #API #Automation #Dukascopy #LightweightCharts #RestAPI #TradingView #PineScript