An intelligent system for collecting, processing, and analyzing news content from hundreds of sources in real-time. It aggregates news, performs semantic content analysis, considers user behavior (comments, social reactions), and generates insights about trends, anomalies, and overall sentiment.
Main functions:
* Collecting news from RSS, API, and websites using custom parsers,
* Content analysis of news: classification of topics, keyword detection, tone determination (positive, negative, neutral),
* Interaction with social networks: accounting for reactions, comments, and news sharing,
* Visualization of the dynamics of topic popularity, mentions, emotions, and behavioral patterns,
* Detection of atypical activities and anomalies in news streams or user behavior.
Technological stack:
* Python — data processing, NLP analysis, anomaly detection,
* Node.js — request processing, API interfaces, data stream handling,
* MySQL — storing news, user activity, and analysis results,
* RabbitMQ — asynchronous data processing,
* Docker — containerization of services for stable deployment,
* GitHub Actions — automation of CI/CD, testing, image building,
* OpenAI Platform — using GPT models for improved content analysis and structured data generation.
Role in the project:
"Turnkey" project — sole full-stack developer. System architecture, integration of NLP models, building data processing pipelines and CI/CD automation, UI dashboards for visualization of analysis results.
#python #NodeJS #javascript #MySQL #rabbitmq #openai #nlp #ml #Parsing #bigdata
Main functions:
* Collecting news from RSS, API, and websites using custom parsers,
* Content analysis of news: classification of topics, keyword detection, tone determination (positive, negative, neutral),
* Interaction with social networks: accounting for reactions, comments, and news sharing,
* Visualization of the dynamics of topic popularity, mentions, emotions, and behavioral patterns,
* Detection of atypical activities and anomalies in news streams or user behavior.
Technological stack:
* Python — data processing, NLP analysis, anomaly detection,
* Node.js — request processing, API interfaces, data stream handling,
* MySQL — storing news, user activity, and analysis results,
* RabbitMQ — asynchronous data processing,
* Docker — containerization of services for stable deployment,
* GitHub Actions — automation of CI/CD, testing, image building,
* OpenAI Platform — using GPT models for improved content analysis and structured data generation.
Role in the project:
"Turnkey" project — sole full-stack developer. System architecture, integration of NLP models, building data processing pipelines and CI/CD automation, UI dashboards for visualization of analysis results.
#python #NodeJS #javascript #MySQL #rabbitmq #openai #nlp #ml #Parsing #bigdata