We are looking for an experienced developer to enhance and optimize an advanced web page scanning system. The project focuses on cybersecurity threat analysis, automation, and optimal server resource utilization.
Optimization tasks (to improve):
1. Browser pool management (Puppeteer):
Currently, each user is assigned a separate browser instance. The system needs to be optimized so that a maximum of 20 instances are active, and subsequent requests are queued. The user should receive a message with the estimated waiting time for the scan in case of overload.
2. Cleaning Puppeteer temporary directories:
Scans generate a large number of temporary files. Previous attempts to clean them caused active sessions to be interrupted. A flexible and safe mechanism needs to be implemented that removes old data without disrupting current processes.
Implementation tasks:
1. Content Security Policy (CSP):
Implementing CSP in the scanner environment to protect against XSS, SQLi, and other attacks while processing unknown URLs. CSP must be designed not to affect the quality of scanning and not to generate false positive results. Process isolation will be required.
2. Support for the TOR network:
Adding the ability to scan resources available through TOR. This requires the installation of appropriate dependencies and configuration of proxies and ports to connect to the hidden network.
3. Vulnerability recognition (based on CVE):
Integration with public CVE databases. During scanning, the system will match known vulnerabilities to recognized technologies (provided that the scan does not analyze internal infrastructure, only publicly visible elements).
4. Generating favicon hash:
Adding functionality to generate a hash of the favicon for quick identification of cloned sites or phishing.
Planned functionalities (product development):
1. One-click Threat Alerting:
An automatic threat report generation system (Twitter, email, Telegram). Ready-to-publish templates with metadata – compliant with standards.
2. Detection of testing environments (staging):
Automatically identifying domain names characteristic of testing environments used by criminals (e.g., dev., staging., panel.).
3. Intelligent redirect tracking:
Visualization of redirect chains. Helps quickly identify phishing, malware, and other hidden links.
5. OCR for screenshots:
Built-in OCR for extracting text from phishing screenshots (email addresses, account numbers, cryptocurrency wallets). Ability to quickly copy detected data.
6. Copy-Paste Analyzer:
A module that allows pasting the content of a suspicious message and automatically extracting IOC: IP, URL, hash, etc.
7. JS Crawler:
Automatic analysis of JS files in search of URLs and endpoints. Generating a simple dependency tree.
8. Recursive URL Hunter:
Recursive link tracking from HTML and JS, with depth limitation (e.g., up to 3 levels), with a visual graph of connections.
9. Dynamic Redirect Tracker:
Tracking HTTP statuses (301, 302, JS redirect) and building a redirect path in a clear format.
10. Payload Hunter:
Detecting embedded executable files (.exe, .apk, .zip, etc.) in HTML/JS and the path to them.
11. JS Deobfuscator:
A tool for deobfuscating JavaScript – detects base64, eval, etc., with highlighting of detected URLs.
12. URL Extractor:
Exports all links from HTML and JS files with one click – useful for manual analysis or export.
If you have experience with Node.js, Puppeteer, web application security, and enjoy working on high-impact projects, get in touch. We are looking for someone who not only knows code but also understands the logic of offensive and defensive tools in cyberspace.
