Davriellelouna@lemmy.world to Technology@lemmy.worldEnglish · 1 year agoThe AI company Perplexity is complaining their bots can't bypass Cloudflare's firewallwww.searchenginejournal.comexternal-linkmessage-square209linkfedilinkarrow-up1226arrow-down10
arrow-up1226arrow-down1external-linkThe AI company Perplexity is complaining their bots can't bypass Cloudflare's firewallwww.searchenginejournal.comDavriellelouna@lemmy.world to Technology@lemmy.worldEnglish · 1 year agomessage-square209linkfedilink
minus-squarethreeganzi@sh.itjust.workslinkfedilinkEnglisharrow-up2·1 year agoDoes it not need to be scraped to be indexed, assuming it’s semi-typical RAG stuff?
minus-squareElectricd@lemmybefree.netlinkfedilinkEnglisharrow-up1·1 year agoI assume their script does some search engine stuff like query google or bing and then “scrap” the links they go on Some selenium stuff
Does it not need to be scraped to be indexed, assuming it’s semi-typical RAG stuff?
I assume their script does some search engine stuff like query google or bing and then “scrap” the links they go on
Some selenium stuff