DLE Module News Parsing and OpenAI Rewriting

• Parse data from websites. (Universal parser) 

• Generates texts and images based on specified parameters.

• Automatically publishes news in selected DLE categories.

• Configured through the admin panel and operates on a schedule (via CRON).

Functional Requirements

1. Basic settings (default parameters)

   • Field for specifying the general OpenAI prompt   (Writing style, author information, if the prompt was written in a certain style) (used for generating texts for all categories by default, as a supplement to the individual  prompt).

   • Settings for the OpenAI model version (e.g., GPT-4 or GPT-3.5).

2. **Categories Section

   • Ability to select one or more DLE categories for data parsing.

   • For each category, the following can be configured:

     • Prompt for text generation (individual for each category).

     • Prompt for image generation (used to create 2-4 images for one news item).

     • Limits for parsing (number of news items per run: from 1 to 5).

     • Links to sources for parsing (ability to add multiple URLs).

     • Parser settings from the source, tags from which to take the title, tags from which to take the news text (for the universality of the parser and the ability to parse from any resource)

3. Parsing functionality

   • Saving the list of news headlines in the database to avoid duplication.

   • Parsing includes:

     • News headline.

     • Short text of the news.

   • Checking for new materials with each run (comparing headlines with the database).

4. Content generation via OpenAI

   • After parsing, the news is processed through the OpenAI API:

     • A complete text of the news is generated based on the prompt tied to the category.

     • 2-4 images are generated based on the prompt for the category.

5. Automatic posting

   • Generated news is automatically added to the selected DLE category.

   • Ability to edit generated materials before publication (optional).

6. Admin panel of the module

   • Interface for:

     • Adding/editing categories.

     • Adding/editing parsing sources.

     • Specifying prompts for texts and images.

     • Choosing the version of the OpenAI model.

   • Displaying the parsing history (which headlines have been processed).

   • Setting the frequency of operation via CRON (e.g., every 15 minutes, 1 hour, etc.).

7. Operation via CRON

   • The module performs the following tasks on schedule:

     • Checking for new materials on parsing sources.

     • Generating texts and images via OpenAI.

     • Automatically adding generated news to the corresponding categories.