I have a text file with 2,624,694 URLS for words.
Task is to extract data from https://www.classicthesaurus.com/
sample content page with word relations: https://www.classicthesaurus.com/apple
sample content antonym: https://www.classicthesaurus.com/robot/antonyms
For all ~2.6 million terms, I want ALL related word information. This may include synonyms, antonyms, broader, narrower, related. You will need a proxy intermediary.
Output should be a text file:
I want the counts for all terms. e.g. for "apple", "fruit" is "40; "iphone" is "12"
Expected output is a syntax like this:
#headword=
synonyms: a, b, c,
synonyms-score: 84, 32, 1
antonyms: t, u, v
antonyms-score: 4, 3, 1
broader: m, n
broader-score: 2, 1
narrower:
narrower-score:
related: z
related-score: 2
;
Max 100 terms per line. Truncate excess. Most words only have a little data.
