I'm trying to get URLs for about 30,000 site names. Google, decades ago, used to have a free API to do web searches. Now, you have to pay for them. Duck Duck Go got rid of theirs, too. Maybe Yandex, et all has one?
I ended up scraping
https://html.duckduckgo.com/html/ with a headless browser (because if you make a direct request without figuring out how to beat their browser sniffer, you get a captcha) to extract the URLs.
I understand why we can't have free APIs now. Things were way nicer when various actors on the web could trust each other to a degree!