StreamSets DataCollector or NIFI
33 USD 2) We select data streaming (for example here: https://www.pubnub.com/developers/realtime-data-streams/ or your version)
3) Installed SDC (StreamSets DataCollector) or NiFi (who wants his option, make his own.SDC and NiFi are perfectly installed through Docker.4) We set up PayPal to process data at our own discretion.The main thing to be:
- receiving information from the source (directly from the service or from Kafka)
- validation of receiving messages (e.g. the name should not exceed 16 characters, otherwise we will cut a line with a name up to 16 characters)
- Field processing (for example, in streaming you get 2 separate fields: name and surname, in processing you make one separate field fullname.The condition of choice (for example, if you meet a name { name }, then you perform certain actions (for example, send a message to Telegram.- Placing the processed data in the BD (you can choose any, but I highly recommend to get to know with Druid at least something to understand).Each work ends with a presentation of what was done by the team and a report on the work done.The content of the report:
- Title page with the theme name and team list
Establishment of task
- Approach to the decision (where the data comes from, what technology has been chosen for the decision, etc.)
Description of the decision
The conclusions
The laboratory is also attached to the code + export of the SDC/NiFi file.Attention !Reports do not need to be printed, enough DOCX/PDF
ТОЛЬКО StreamSets DataCollector или NiFi. Не своя разработка
Client's review of cooperation with Dmytro Ohorodnik
StreamSets DataCollector or NIFIThe work was done in time. The executive is always in contact. I recommend it.
Freelancer's review of cooperation with Marina Shshshshsh
StreamSets DataCollector or NIFIThe ideal customer. No calls.
-
Winning proposal7 days33 USD
1125 43 0 1 Winning proposal7 days33 USD
Hello to you. I propose a solution on Python3, with a subsequent preprocessing of data and their storage. If you can choose any database, it is better to use PostgreSQL or MySQL. I will make the code with comments, I will also write the report on the implementation of the parser. I am pleased to take on the performance.
-
3 days34 USD
2233 82 0 3 days34 USD
Good day ,
I propose this laboratory solution: we parsim data from any source (e.g. by API) and the script on Python 3 preprocesses data and fills the BD. There is such a job solution - I can rewrite to your needs.
I will describe the whole process in the report.
I will also review your option of decision, interested in cooperation.
I will be glad to help you!
Current freelance projects in the category Data Parsing
Collection of Telegram channels (English-speaking)Need to collect a database of Telegram channels with the following criteria (2000-3000) channels Criteria for TG channels: Language: English Geography of the audience: USA (US)/United Kingdom/Canada, etc. English-speaking channels under tier 1-2 geo Audience:… Information Gathering, Data Parsing ∙ 6 hours 35 minutes back ∙ 3 proposals |
Database of websites on WooCommerceIt is necessary to compile a database of Ukrainian online store websites on WooCommerce with the contact information provided on the sites. Only active websites (indicator: updated catalog/content, working domain) Table format - website address, phone number, e-mail. Data Parsing ∙ 21 hours 44 minutes back ∙ 18 proposals |
Create a dashboard in https://airtable.com/ for the performance of advertising creatives from Facebook ads.Full specification https://docs.google.com/document/d/1_n_oYRNZWYxalUA---DM5AD1b5ZSrtePw5J4G42svGw/edit?usp=sharing Databases & SQL, Data Parsing ∙ 2 days 12 hours back ∙ 17 proposals |
Creation of an Excel file for uploading products to the websites of other partners.I am interested in creating an Excel table with all parameters. Here is the website - https://heiztechnik.com.ua/ And the positions I am interested in to be transferred: Manual boilers: 1) TIS UNI 15-95 kW (10) pcs 2)TIS HARD 150-500 kW (7) pcs Pellet boilers: 1)TIS PELLET… Data Parsing ∙ 2 days 16 hours back ∙ 34 proposals |
A developer is required for parsing the catalog and automating data import.Detailed technical specifications in the attached document Please indicate the estimated cost and timeline in your response Do you have experience working with parsing large catalogs What possible difficulties or limitations do you see in this task Databases & SQL, Data Parsing ∙ 2 days 19 hours back ∙ 39 proposals |
Kyiv