Python developer, scraping expert
Hello, We are looking for a person who will solve the following tasks:
Refactoring of the parser system (10 sources)
There is a system of 10 parsers from different sources.
Refactoring is needed:
- Unify the architecture of all parsers
- Extract common logic into separate modules
- Prepare for integration with AirFlow
- Set up weekly execution
An analysis of the current code and suggestions for improving the structure are required.
Code: https://drive.google.com/file/d/1RMZsWYDLTYV3c4zkHrrUT0-tdTo7Cua4/view?usp=sharing
Access to the database can also be shared, but if you can set it up locally, that would be great. This is not a one-time task; it can be considered as a paid test, and as a result, we are looking for a person for part-time work.
Requirements:
Familiarize yourself with the code
Propose and justify changes to the code.
Make changes
Set everything up on the server and run it on a scheduler
Please write your hourly rate and estimate for this task right away!
-
10 days226 USD
1825 52 0 10 days226 USD
Hello. I specialize in automation and information parsing. I am interested in long-term cooperation. I looked at the code - everything is clear, but the imports are currently incorrect. Regarding the database - only data from the "urls_sitemap" table will be needed - to check if the parsers are working.
I haven't worked with AirFlow yet - but I'm ready to figure it out quickly.
As a result of this task - is it required to minimally integrate with AirFlow or just to prepare and run the old code on the server?
Rate: $25 / day
Update: I have already figured out AirFlow - but now the source files are no longer available.
-
1 day23 USD
69 1 day23 USD
Hello!
I am a professional Python developer with extensive experience in automation, parsing, bot creation, and web tools.
I have completed over 500 projects — from simple competitor price parsers to complex systems that gather millions of products from Amazon, bypassing Cloudflare, captchas, IP blocks, and authorizations.
I have worked with websites such as:
- Amazon
- Instagram
… - Facebook
- Google
- Twitter
- LinkedIn
- Walmart
and many others.
Technologies I work with:
- Python
- Requests
- BeautifulSoup
- Selenium
- Playwright
- Scrapy
- Undetected Chromedriver
- IP rotation
- Django + PostgreSQL.
For large projects, I create admin panels and databases for convenient work with the collected data.
I also have my own lead database with emails, phone numbers, and social media, which will help speed up your marketing or sales.
I am ready to show a test example before starting work so you can assess the quality. Write to me, tell me about the task — and we will find the best solution for your business.
-
1 day20 USD
414 1 0 1 day20 USD
Good afternoon, I am ready to consider, I often work with parsers in different programming languages. And write me a private message, we will discuss everything in more detail there.
-
10 days158 USD
320 10 days158 USD
Good day!
I have experience in Python development.
I reviewed the code and can suggest changing (improving) the architecture to Clean Architecture, as well as fixing the import errors. I can set up the database locally by myself, but for the parsers to work correctly, the data from the urls_sitemap table is very much needed.
I haven't worked with Airflow yet, but I learn quickly and am confident that I can prepare the system for integration with Airflow. I also have experience in setting up a scheduler for automatic task execution.
Rate — $7/hour.
Write to me, I will be happy to collaborate!
-
3 days180 USD
1412 14 3 1 3 days180 USD
Hello, I reviewed your task and would like to take on the implementation. I can work even 16 hours a day at an hourly rate of $10. I would be happy to collaborate.
-
11 days20 USD
512 3 1 11 days20 USD
Hello, interested in part-time work, indicating the rate and deadline (when working 4 hours a day)
Current freelance projects in the category Data Parsing
OpenCart — rental catalog of special equipment
135 USD
OpenCart — Equipment Rental Catalog Need to launch an equipment rental catalog on OpenCart. Theme: excavators cherry pickers forklifts generators cranes scaffolding other construction equipment. It is preferable that you already have a ready-made template or developments… Web Programming, Data Parsing ∙ 16 hours 7 minutes back ∙ 42 proposals |
Transfer the program - the server where the program was located has crashed (officially permitted parsing of government data)
47 USD
Hello! My client has encountered the case described below. We need help transferring to a new server and testing the program. It would be better to have a programmer who understands parsing. Software & Server Configuration, Data Parsing ∙ 20 hours 5 seconds back ∙ 25 proposals |
Parsing and classification of a large array of imagesIt is necessary to implement a project for collecting and structuring a large array of architectural images from open web sources.The task includes: automated collection of images; uploading files in the highest available quality; classification of images by categories:… Python, Data Parsing ∙ 20 hours 28 minutes back ∙ 30 proposals |
Parsing Resumes from Work.ua and Rabota.uaIt is necessary to download a database of current resumes from Work.ua and Rabota.ua in Excel format. It is preferable that the Excel file contains the following columns: Name Phone number (in the format +380...) Email Position (profession) Positions considered Category of… Information Gathering, Data Parsing ∙ 1 day 20 hours back ∙ 18 proposals |
Website parsingImplementation of 4 parsers (directory websites) is required. There is a technical specification, and there is a code example as a reference. The tasks include: Writing a parser Integrating a proxy Deduplication logic (transfer the logic from the example) Hashing logic based… Data Parsing ∙ 2 days 12 hours back ∙ 42 proposals |
Ukraine