We have a ready-made website on WordPress, where profiles of business owners of a certain category will be placed. The task is to collect, clean, store, and display these profiles on the site.
It is necessary to parse data from LinkedIn — approximately five thousand business owners of a specific category, including profile pictures.
There are also other data sources (donors) from which parsing can be done; details can be discussed. During the data collection process, the information should be properly structured (name, company name, business category, contact details, photos, etc.), while checking records for duplicates and removing them before adding to the database and displaying them on the site. The data should be as clean and standardized as possible in format.
To store the parsing results, a separate table should be created in the database to avoid overloading the main WordPress database. Part of the site interface and admin panel should be adapted to work specifically with this table. After cleaning and verification, the data should be imported to the site and displayed in the "Business Owners" section according to the structure and design of the page.
From you, we need a brief description of how you will parse data from LinkedIn, as well as your cost for the work and the estimated time frame for completion — from data collection to publication of profiles on the site.
Experience:
The contractor must have experience in creating parsers for Google Maps, social networks, Google search results, and aggregator sites. This is not a one-time task — we need a person who can regularly parse data from various sources as needed.
Scraping → Temporary storage → Cleaning → Final database → Connecting data to the website and displaying on WP
Python — Playwright (browser automation, bypassing protection), BeautifulSoup, asyncio, API integration. Libraries requests, BeautifulSoup, Selenium, Scrapy or Selenium.
For Google Maps — SerpAPI/Outscraper as a reliable data source.
Text cleaning, deduplication, classification through AI, filtering by keywords.
Validation of phones, emails, standardization of working hours, deduplication of businesses before recording in the database
MySQL, WordPress REST API, Custom Post Types + ACF. Scheme: collection → temporary storage → cleaning → MySQL → WP via REST API or WPAllImport.
Data is small business, data will be taken from Google Maps, Google Search, Websites, Yelp, Public government sites
There is a budget for API/proxy to bypass Google restrictions
We consider people only from Ukraine
If there is an active automated scraper in production, that will be a plus.