Dmitro A.
Rating
Language proficiency level
Skills and abilities
Portfolio
-
23 USD Etsy parser with asynchronous data collection and visualization in Excel
Data ParsingUsed libraries:
● playwright: for browser initialization, obtaining cookies, loading Etsy pages, and bypassing protection mechanisms.
● BeautifulSoup (bs4): for parsing HTML pages, finding necessary elements (e.g., product links) and filtering them by required parameters.
● openpyxl: for creating tables, formatting cells, inserting text data and images into .xlsx files.
… ● Pillow (PIL): for image processing, including resizing images before inserting them into Excel.
● httpx: for asynchronous loading of product images from high-quality links.
Main tasks:
● Data collection automation – Searching for products by keywords, filtering by store name, bypassing CAPTCHA.
● Information processing – Parsing HTML to obtain product names, IDs, images, resizing images.
● Saving results – Creating an Excel table, inserting text and images, formatting the table.
● Asynchronicity – Simultaneous processing of requests and loading images.
● Flexibility of settings – Configuring input parameters, scan depth, and pauses.
Implementation process:
1. Data collection:
Data is collected using the EtsyClient class, which encapsulates functions for interacting with the Etsy platform, collecting keywords, loading pages, and processing results. BeautifulSoup libraries are used for parsing and httpx for loading product images. Data is organized into a structure ready for saving to a file.
2. Data processing and saving:
The openpyxl library is used to save the collected data. An Excel table is created, into which both text data about products and product images are recorded. For each product, the image size is automatically adjusted before insertion to ensure correct display in the table.
3. Asynchronicity and efficiency:
The data collection and processing process is implemented asynchronously, allowing multiple requests to be processed and images to be loaded simultaneously. Thanks to the asynchronous approach, the data collection process is significantly accelerated, reducing the program's execution time.
4. Flexibility of settings:
The program is easily configurable to work with different stores on Etsy due to the use of class variables, allowing parameters to be changed without the need to modify the code.
Tags:
#python #parsers #Parsing #playwright #webscraping #Parsers #scrape #beautifulsoup #beautifulsoup4 #bs4 #pillow #openpyxl
-
Forecasting cryptocurrency prices using neural networks
PythonUsed libraries:
● aiohttp: for collecting historical data from the Binance API
● PyTorch and PyTorch Lightning: for building and training the model
● pandas, numpy: for working with time series, preparing and analyzing data
… ● seaborn, matplotlib: for data visualization
● DeepSpeed: for optimizing training on large datasets
Main tasks:
● Data collection and normalization, preparation of time series for cryptocurrencies
● Building and training the optimized DeepAR model
● Forecasting data using the created model
● Visualizing results to assess model quality
Implementation process:
1. Data preparation:
The function get_klines_for_train retrieves historical price change data at time intervals and calculates technical indicators such as RSI, EMA, and others. After that, the data is saved to a CSV file for further use. The currency pair JUP/USDT was chosen for model training with a 1-hour interval, and for testing — the pairs BTC/USDT, SOL/USDT, and XRP/USDT. Although the amount of data was limited, it was sufficient for effective training. In the main module, data is loaded from the CSV file, and the dataset is normalized using a specially implemented MinMax method.
2. Creating and training the DeepAR model:
The model is based on a customized version of DeepAREnhanced, which is an optimized variant of the classic DeepAR. To improve accuracy on large time series, a special version of ScaledNormalLoss was introduced, which enhances the loss function. Training was conducted using DeepSpeed to accelerate the process and EarlyStopping to avoid overfitting.
3. Forecasting and visualization:
After training, the model was used to forecast prices on the test dataset. The forecasting results were compared with actual values using the plot_comparison() function, which visually demonstrates the model's accuracy and its effectiveness in predicting future trends.
Tags:
#python #pytorch #binance #binance.com #datascience #cryptocurrency #криптовалюты #ai
Reviews and compliments on completed projects 5
20 December 2024
23 USD
Data parsing
Dmitry was able to complete the project that the three previous contractors could not. I recommend him as a person who is excellent at both Python and writing parsers. I will definitely reach out again.
21 November 2024
27 USD
Write a TG bot for TradingView that receives commands from the indicator, refinement of the indicator.
Excellent specialist, I recommend, does everything on time, correctly, I am very satisfied, I am contacting for the second time and will definitely contact again!! I RECOMMEND TO EVERYONE!
![]()
3 November 2024
27 USD
Set up automatic updates of stock and prices on Khoroshop.
everything is done, great job
1 November 2024
23 USD
Combine 3 open-source tradingview indicators into one
Great specialist, I recommend, improved the treadingview indicator.
10 June 2024
16 USD
Parser
Thank you for the completed work
![]()
Activity
| Latest proposals 10 | Budget | Added | Deadlines | Proposal | |
|---|---|---|---|---|---|
|
Improvements to the Telegram bot
27 USD
|
|||||
|
Website parsing
27 USD
|
|||||
|
Development of software for registering hotmail/outlook accounts in Python using post/get requests.
349 USD
|
|||||
|
Data parsing
23 USD
|
|||||
|
OLX Parser 24/7
45 USD
|
|||||
|
Futures parser
226 USD
|
|||||
|
Object and text recognition in photos
45 USD
|
|||||
|
Python script for data parsing and sending the result to Telegram
41 USD
|
|||||
|
Parser from solscan or gmgn
72 USD
|
|||||
|
Parsing product cards from the website to Excel
23 USD
|