Thank you, everything is fine, we will continue to cooperate.
Budget: 12500 UAH Deadline: 4 days
😄 Hello,
I understand your need to set up an automated parsing system to extract new articles/news from specified websites and save them in your database, avoiding blocks from Cloudflare or similar services. The system should operate on a schedule (cron) and be deployed on your server. The choice of programming language is flexible, but Python is a logical option if a headless browser is needed.
According to the description, this project is similar to one I worked on recently. I was responsible for developing a content extraction system from various blogs, news portals, and online publications. As in your case, the main task was parsing and saving the textual content of articles.
For this project, I used Python and the BeautifulSoup library for parsing HTML. It is great for navigating, searching, and modifying the parse tree. To automatically run the script, I applied the built-in sched module in Python for task scheduling, similar to a cron job.
Additionally, for making HTTP requests and receiving responses, I used the requests module, and to bypass Cloudflare protection and similar services – cfscrape.
For data storage, I used SQLAlchemy – an ORM system for Python that provides powerful tools for efficient database management.
Regarding processing websites that use JavaScript for rendering content, I used Selenium WebDriver along with PhantomJS, which allowed simulating user behavior in the browser and obtaining dynamic content.
To avoid blocking and comply with robots.txt rules, I used the robotexclusionrulesparser module, and for random intervals between requests – the time module.
🕒 I am open to both part-time and full-time employment, and I am flexible regarding time zones. I am confident that I can successfully implement your project, and I assure you that you will not be disappointed if we collaborate.
I look forward to further discussion. Thank you for considering my proposal!
Budget: 10000 UAH Deadline: 15 days
I am interested in this project, I do parsing in Python, payment is negotiable, I would be happy to collaborate.
9
0
Budget: 13000 UAH Deadline: 6 days
I am ready to help with setting up an automatic parsing system to collect content from the resources you specified. I plan to use Python with libraries such as BeautifulSoup or lxml for parsing HTML pages, as well as Selenium or Playwright to bypass Cloudflare-type protection. This will allow us to ensure reliable parsing, even if the site uses complex protection methods.
The parser will run via cron, executing once an hour, checking for new articles on each of the resources, and collecting only textual content. The results will be stored in your database. I will also anticipate the use of proxies or random User-Agents to avoid blocks.
The estimated cost of the project is 13,000 UAH, and the completion time is 6 days, depending on complexity. I am also ready to assist with designing the database structure if needed.
Please clarify what type of database you are using and whether there is already a ready structure for storing data? If my approach and estimate suit you, let's discuss the details.
3
0
Budget: 5000 UAH Deadline: 5 days
Hello, I have similar experience, passwords for 8 news portals for a group of Telegram channels. I have also worked with various databases and there will be no problems, if necessary, to write SQL code to work with your database, I have also worked with MongoDB. I quickly went through the links and cannot say for sure if there is serious protection, but I can bypass Cloudflare with the availability of proxies and other resources. I write in C#. Have a good day!
Yaroslav Zhomiruk
Winning proposal
4
0
Budget: 5000 UAH Deadline: 4 days
I looked, everything is parsing fine, no problems arose, so I am ready to execute the project.
Regarding the server – as I understand, you have some database where everything needs to be recorded? That is, there are two options: either create a microservice in Docker that your API will connect to, or write data directly to the database.
One more thing – if the source websites change their structure, the parser may start retrieving data incorrectly. We need to discuss how to act in such cases. Overall, the project is not complicated, but I still do not fully understand the architecture. If you provide details regarding deployment, we can do everything quickly.
2
1
Budget: 6000 UAH Deadline: 3 days
Vladimir, hello!
I have sufficient experience in parsing and will complete your task in 3 working days.
I look forward to collaborating!
19
0
Budget: 9000 UAH Deadline: 5 days
Hello! I am ready to create a Python script for automatic parsing for you, taking into account all the listed points.
I suggest we discuss the details in private, and I will guide you on the terms.
Proposals concealed
The list does not show proposals concealed by the client or freelancer with a Plus profile, as well as proposals violating rules
