I've built an app which would benefit immensely from integrating ML webcrawling.
But I'd like be able to intelligently navigate of webpages toward an objective. This'd unlock a lot of useful applications. And it's feasible.
Background: I've done a lot of procedural webcrawling (puppeteer, selenium, straight up js - particularly on sites that seek to be crawl-proof). And I've solved captchas autonomously within a webcrawling app using w/ pytesseract. I'm also aquatinted w/ pytorch + brain.js + kaggle.
Any data scientists/devs who'd fit in collaborating to solve this problem (or a subset of it), hit me up. Looking to work with those with a track record of executing/getting things done (rare). :)
-Blake
[email protected]
discord: deepb#4778
Not related, but what are you thoughts on scraping linkedin? feasable at scale?