Correction of errors in the script

Python — incorrectly specified categories?

Project translated automatically. Log in or register, to view the original We need an experienced programmer.There is a script that using Selenium enters the site looking for the keyword there, gathers the headlines of the answer, and analytics, the time of the nodes, waiting for the answer, etc (as in the fiberbag).In case of error, it makes a screen and saves the HAR file.If 1 copy is launched in 4-5 seconds, everything works normally.Problems start when several tasks are launched simultaneously.(For example, 30 tasks in the 1st minute).The script begins to work unstable.The data in the BD on some bags are stored partially or not stored entirely.The most common problems I saw in the logs: 1 .ProtocolException("Error in HTTP connection: Http2ZombieException('Connection or stream already dead: 1682387806.5554209, False')") 2. r, w_, e_ = select.select([self.socket], [], [], poll_interval) ValueError: filedescriptor out of range in select() What to do: 1 .Fix problems with simultaneous launch that would work properly.2ndOptimize the code.Increase the productivity.At least 60 bags per minute should be processed, preferably more.ThreeGet rid of the extra request with which the headlines are collected.Do it through the selenium.4 .Find out why some processes of chromium do not close and continue to hang in the processes.The script consists of two parts.1 .Managers - who start tasks on schedule.2ndThe accomplished script.All tasks are launched in a separate stream.Technology: Python 3.10, Mysql and Selenium

Proposals 2 Rejected 1

Mar'yan S.

15 0

Projects 17
Rating -
Rating 866

Budget: 500 UAH Deadline: 3 days

Hello to you.
Do you run the code on a PC or other car? You may just lack the operating memory to process all processes.
Also, I am interested in whether chromium is launched in headless mode?
In the rest, you need to look at the code to understand what the main problem is.

The list does not show proposals concealed by the client or freelancer with a Plus profile, as well as proposals violating rules

Kirilo Skripnichenko

0 0

Projects -
Rating -
Rating 194

Budget: 500 UAH Deadline: 1 day

Good day !
Full-stack developer with more than 6 years of work experience, your project is very interesting, I have the necessary experience to implement it, I offer such technologies Node.js + Vue.js. Let us make a call for a detailed discussion of the task, share our vision and discuss cooperation, write

Current freelance projects in the category Python

Refinement of the Python program for the new Twitter (X) chat interface.

Desktop Apps 52 proposals 31 July

Not specified
Signal bot for trading

Bot Development 57 proposals 31 July

67 USD
Website parsing, bypassing Akamai protection

Data Parsing 39 proposals 30 July

Not specified
Development of an AI bot for receiving and processing orders from restaurants in Telegram with integration into 1C.

72 proposals 29 July

446 USD
Need to fix a bug in the current version of the bot.

Bot Development 87 proposals 27 July

Not specified

Ivan Y.
Ukraine

Projects 12
Rating -
Rating 489

Mar'yan S.

Proposals are currently absent

Kirilo Skripnichenko

Current freelance projects in the category Python

Refinement of the Python program for the new Twitter (X) chat interface.

Signal bot for trading

Website parsing, bypassing Akamai protection

Development of an AI bot for receiving and processing orders from restaurants in Telegram with integration into 1C.

Need to fix a bug in the current version of the bot.