Python + Selenium. Anti-captcha. Parsing Google search.
Hello. I will provide a detailed log of the problem to anyone who writes in the bid. The task is for someone who has experience with bots/parsers that worked on Selenium and made requests to Google search and used anti-captcha service. I have a parser in Python that makes requests through aiohttp. It has a local host. It makes a request to Google search. It uses Selenium for this.
Example of the problem. The parser opened Selenium and sent a search query to Google search. A captcha appeared. The anti-captcha service solved the captcha. The bot continues to search. The problem is that sometimes Selenium passes the solved captcha key. But then it happens that Google accepts the key sometimes and sometimes it does not. Even if the captcha is 100% solved. Until I manually solve the captcha in ANOTHER TAB of Selenium, the bot cannot continue working. I note in ANOTHER TAB. Because in this TAB where the captcha was, it dies manually, and even the captcha cannot be passed. You pass the captcha twice, and Google redirects the page: google.com/sorry/index
And the link with the yellow tail disappears. Instead, it becomes the short one that I wrote.
So the issue is not with the service or Google itself. But with Selenium itself. Something is wrong with it. Because! When I manually solve the captcha in a new tab, Google immediately and always allows me to continue searching. In my Selenium, there is a chrome data folder. So that you don't think that the session is not saved. Two different developers looked at the code. And they said that only someone who understands Selenium well can solve the problem. Because Python cannot control everything. Sometimes to fix something, you need to dive into the actual code of Selenium itself, and that is already binary code.
For someone who has developed something on Selenium, figuring out such a problem will be easy. It is necessary to eliminate the errors that are in the parser console; possibly the problem is due to them, I think there shouldn't be any.
All this is strange to me. Because on the captcha solving service, there is a detailed instruction. Api Python + Selenium. We followed it in detail. Whoever helps will always be given permanent work. I have many different projects. Thank you for your attention!
-
Невірний підхід до проблеми. Треба робити так, що б Гугл не викидав капчу)
-
У вас проблема с самим селениумом, который детектится, просмотрите фингерпринты, попробуйте использовать стелс, или поднятие реального браузера с подключением CDP сессии, рассмотрите варианты с антидектектами по типу адспавер, гологин и т.д.
Также немаловажную роль играют прокси, обязательно резидентские, никаких датацентров (которые дешевле).
-
Олексанр, у вас все ще відкрите питання?
-
Current freelance projects in the category Data Parsing
Creation of automatic video shorts
45 USD
What exactly do I need I need to create a light AI agent or content factory 1. This is idea creation Reference https://youtube.com/shorts/omgVX00yaD8?is=1Gz_TUUlftZnkxQv 2. This is text creation 3. This is background creation + text 3. Animating the background while keeping the… AI & Machine Learning, Data Parsing ∙ 15 hours 1 minute back ∙ 12 proposals |
Parsing company emails from a specific industry
113 USD
Find contacts (e-mail, phone number) of companies engaged in the specified business mentioned in this Google document https://docs.google.com/document/d/1_d-iLxLSL3gjRvo1GmUpWNm0Pm5YoKGcEvaiDc_2I0g/edit?usp=sharing The result should be in the form of a table with the following… Data Parsing ∙ 1 day back ∙ 28 proposals |
Need a parser for the online store https://www.lcsc.com/It is necessary to regularly (once a month, or upon script launch) obtain up-to-date information about the products available in the store. https://www.lcsc.com/ from the catalog of all sections.… Data Parsing ∙ 2 days 5 hours back ∙ 43 proposals |
OpenCart — rental catalog of special equipment
135 USD
OpenCart — Equipment Rental Catalog Need to launch an equipment rental catalog on OpenCart. Theme: excavators cherry pickers forklifts generators cranes scaffolding other construction equipment. It is preferable that you already have a ready-made template or developments… Web Programming, Data Parsing ∙ 2 days 21 hours back ∙ 58 proposals |
Transfer the program - the server where the program was located has crashed (officially permitted parsing of government data)
46 USD
Hello! My client has encountered the case described below. We need help transferring to a new server and testing the program. It would be better to have a programmer who understands parsing. Software & Server Configuration, Data Parsing ∙ 3 days 1 hour back ∙ 29 proposals |