Tech bros just actively making the internet worse for everyone.
Tech bros just actively making
the internetsociety worse for everyone.FTFY.
I mean, tech bros of the past invented the internet
Those are not the tech bros. The tech bros are the ones who move fast and break things. The internet was built by engineers and developers
Nah, that was DARPA
Those were tech nerds. “Tech bros” are jabronis who see the tech sector as a way to increase the value of the money their daddies gave them.
We had a trust based system for so long. No one is forced to honor robots.txt, but most big players did. Almost restores my faith in humanity a little bit. And then AI companies came and destroyed everything. This is why we can’t have nice things.
Big players are the ones behind most AIs though.
There once was a dream of the semantic web, also known as web2. The semantic web could have enabled easy to ingest information of webpages, removing soo much of the computation required to get the information. Thus preventing much of the AI crawling cpu overhead.
What we got as web2 instead was social media. Destroying facts and making people depressed at a newer before seen rate.
Web3 was about enabling us to securely transfer value between people digitally and without middlemen.
What crypto gave us was fraud, expensive jpgs and scams. The term web is now even so eroded that it has lost much of its meaning. The information age gave way for the misinformation age, where everything is fake.
Capitalism is grand, innit. Wait, not grand, I meant to say cancer
I feel like half of the blame capitalism gets is valid, but the other half is just society. I don’t care what kind of system you’re under, you’re going to have to deal with other people.
Oh, and if you try the system where you don’t have to deal with people, that just means other people end up handling you.
In this case it is purely fault of the money incentive though. Noone would spend so much effort and computation power on AI if they didn’t think it could make them money.
The funniest part is though that it’s only theoretical anyway, everyone is only losing on it and they’re most likely never gonna make it back.
It matters a lot though what kind of goal the system incentivises. Imagine if it was people’s happiness and freedom instead of quarterly profits.
That’s the part people never really seem to understand. It makes sense though because we’re subjected to the system from birth and it’s all a lot of people know so they can’t grasp the idea of a world outside of that so it can sometimes be difficult to get through to people on that
Imagine if it was people’s happiness and freedom instead of quarterly profits
- Whose happiness and freedom?
- How is it to be measured?
- Capitalists honestly believe that free trade is the best albeit flawed way to do both of the above
It’s definitely valid to disagree about point #3, but then you need to give a better model for #1 and #2
- Optimise for maximum happiness and freedom and minimum suffering on a societal level, while making sure that a certain threshold of freedom-from-suffering is met for every single individual. (So that happiness for the majority is not reached via the suffering of minorities.) Obviously there can be arguments about where we draw the lines for the specifics of these.
- Quite a few ways have been developed to measure happiness, wellbeing, flourishing, quality of life, whatever nuance you want to pick.
- We have ample proof that it’s a system that fundamentally undermines societal happiness by incentivising the accumulation of wealth at the expense of the wellbeing of people and ecosystems. It’s decades overdue to radically change the way the world’s economy works. (Which is of course not something those with the most power want, so unlikely to happen any time soon.)
Could you clarify on what you mean with “dealing with people”? I’m not really sure the point you’re trying to make with that
The complaint that got blamed on capitalism was:
The information age gave way for the misinformation age, where everything is fake.
and if there’s one entity/person most responsible for that, it’s Putin or the GOP. Most of it is political, and very little to do with capitalism itself. Except that capitalism surrounds and is intertwined with everything.
Still, if you get rid of capitalism, it doesn’t get rid of politics. I’d argue that the root of the issue is the GOP trying to hoard power (money and otherwise), and power is going to exist with or without capitalism. Is North Korea capitalist? Do they have issues with disinfo?
This Christian Sharia Law movement doesn’t exist for money.
Capitalism itself is political. But the point I was making was that capitalism was the driving force of enshittenfication of all our technology that could be used for helping us all but instead it’s only about profit. Which is capitalism…
Two of the largest drivers are religion, christians wanting their Sharia Law, and Russia taking political control of the US.
Capitalism is in the top three, sure. It’s also part of the driver of that technology.
I don’t think we should worship capitalism as we have, but I don’t think getting rid of capitalism as a whole solves more problems than it creates.
Give me capitalism with heavy socialist controls and political separation please, thanks. The general idea of using money as a measure of what society owes you isn’t terrible. It’s allowing that measure to get so out of whack and have such inordinate control of everything that is the problem.
The neat part is that anything bad that happens under capitalism is capitalism’s fault, but anything good that happens is actually socialism happening in spite of capitalism, somehow.
Could you give some examples?
Socialized healthcare
And in what way does capitalism socialize healthcare in the United States?
Socialized healthcare exists there – at least until the current administration finishes ripping it away
I would give this reddit gold
Instant easy complaints help-i’m-oppressed-by-Capitalism today sound an awful lot like the instant easy complaints help-i’m-oppressed-by-Communism I used to hear from rednecks
Ask someone who starved & died under either system how obviously superior it is, you will find millions on either side
Also consider that Socialism is totally legal under Capitalism. Want to start a co-op? Go for it. Want to legislate and implement socialized healthcare? Many Capitalist countries have.
Under Communism, Capitalism must be illegal and stamped out by force. Want to start a business making shoes and hire someone to work for an agreed upon wage? Illegal.
When the goal involves guaranteeing positive rights, I’m not sure how it can be achieved without coercion. Which is how any socialist policies get implemented under capitalism anyways.
Web3 was about enabling us to securely transfer value between people digitally and without middlemen.
It’s ironic that the middlemen showed up anyway and busted all the security of those transfers
You want some bipcoin to buy weed drugs on the slip road? Don’t bother figuring out how to set up that wallet shit, come to our nifty token exchange where you can buy and sell all kinds of bipcoins
oh btw every government on the planet showed up and dug through our insecure records. hope you weren’t actually buying shroom drugs on the slip rod
also we got hacked, you lost all your bipcoins sorry
At least, that’s my recollection of events. I was getting my illegal narcotics the old fashioned way.
the old fashioned way.
A whole swath of trained toads using a special made tube network?
getting into a car with a stranger who said he was 15 minutes away two hours ago
You were there too? 😜
Holy shit do I not miss that lifestyle
also we got hacked, you lost all your bipcoins sorry
aaaaaaaaand - it’s gone!
You want some bipcoin to buy weed drugs on the slip road? Don’t bother figuring out how to set up that wallet shit, come to our nifty token exchange where you can buy and sell all kinds of bipcoins
Maybe I’m slow today, but what is this referencing? Most dark web sites use Monero. Is there some centralized token that people used instead?
Edit: Oh, I guess you’re referring to Mt.Gox? I mean yeah, people were pretty stupid for keeping their bitcoin in exchange wallets (and sending it right to the drug dealers directly from there? Real dumb). That’s always a bad idea. I don’t think they transferred it there instead of something else, they just never took custody of the coins after buying them on the exchange.
Monero
Satoshi was right and Crypto absolutely has valid use cases. What if your government doesn’t want you accessing meds you need at prices you can afford? What if your government doesn’t like your sexual orientation, but you want a subscription to a dating site? What if your government throws up unjust export controls or tariffs that suddenly make you and your business impossible?
Crypto’s best killer use case is uncensorable, untraceable money
Bitcoin is neither of those things. There is a reason people buy heroin with Monero. It actually does what crypto is supposed to do, which means it could safeguard your Grindr XTRA subscription.
Yeah Monero is just cool, tech-wise
Mr. Internet, tear down these walls! (for all these walled gardens)
Return the internet to the wild. Let it run feral like dinosaurs on an island.
Let the grannies and idiots stick themselves in the reservations and asylums run by billionaires.
Let’s all make Neocities pages about our hobbies and dirtiest, innermost thoughts. With gifs all over.
I’m down with that. Web 1.5? Let’s do it. I’ll get my Geocities page up and then we can rev up that hit counter.
https://homestarrunner.com/toons/backtoawebsite
“Lemme get that hit counter!”
Sound like it went the same way everything else went. The less money is involved the more trustworthy it is.
I use Anubis on my personal website, not because I think anything I’ve written is important enough that companies would want to scrape it, but as a “fuck you” to those companies regardless
That the bots are learning to get around it is disheartening, Anubis was a pain to setup and get running
I know this is the most ridiculous idea, but we need to pack our bags and make a new internet protocol, to separate us from the rest, at least for a while. Either way, most “modern” internet things (looking at you, JavaScript) are not modern at all, and starting over might help more than any of us could imagine.
Like Gemini?
From official Website:
Gemini is a new internet technology supporting an electronic library of interconnected text documents. That’s not a new idea, but it’s not old fashioned either. It’s timeless, and deserves tools which treat it as a first class concept, not a vestigial corner case. Gemini isn’t about innovation or disruption, it’s about providing some respite for those who feel the internet has been disrupted enough already. We’re not out to change the world or destroy other technologies. We are out to build a lightweight online space where documents are just documents, in the interests of every reader’s privacy, attention and bandwidth.
It’s not the most well thought-out, from a technical perspective, but it’s pretty damn cool. Gemini pods are a freakin’ rabbi hole.
Yep! That was exactly the protocol on my mind. One thing, though, is that the Fediverse would need to be ported to Gemini, or at least for a new protocol to be created for Gemini.
If it becomes popular enough that it’s used by a lot of people then the bots will move over there too.
They are after data, so they will go where it is.
One of the reasons that all of the bots are suddenly interested in this site is that everyone’s moving away from GitHub, suddenly there’s lots of appealing tasty data for them to gobble up.
This is how you get bots, Lana
I’ve personally played with Gemini a few months ago, and now want a new Internet as opposed to a new Web.
Replace IP protocols with something better. With some kind of relative addressing, and delay-tolerant synchronization being preferred to real-time connections between two computers. So that there were no permanent global addresses at all, and no centralized DNS.
With the main “Web” over that being just replicated posts with tags hyperlinked by IDs, with IDs determined by content. Structured, like semantic web, so that a program could easily use such a post as directory of other posts or a source of text or retrieve binary content.
With user identities being a kind of post content, and post authorship being too a kind of post content or maybe tag content, cryptographically signed.
Except that would require to resolve post dependencies and retrieve them too with some depth limit, not just the post one currently opens, because, if it’d be like with bittorrent, half the hyperlinks in found posts would soon become dead, and also user identities would possibly soon become dead, making authorship check impossible.
And posts (suppose even sites of that flatweb) being found by tags, maybe by author tag, maybe by some “channel” tag, maybe by “name” tag, one can imagine plenty of things.
The main thing is to replace “clients connecting to a service” with “persons operating on messages replicated on the network”, with networked computers sharing data like echo or ripples on the water. In what would be the general application layer for such a system.
OK, this is very complex to do and probably stupid.
It’s also not exactly the same level as IP protocols, so this can work over the Internet, just like the Internet worked just fine, for some people, over packet radio and UUCP or FTN email gates and copper landlines. Just for the Internet to be the main layer in terms of which we find services, on the IP protocols, TCP, UDP, ICMP, all that, and various ones and DNS on application layer, - that I consider wrong, it’s too hierarchical. So it’s not a “replacement”.
IP is the most robust and best protocol humanity ever invented. No other protocol survived the test of time this well. How would you even go about replacing it with decentralization? Something needs to route the PC to the server
Something needs to route the PC to the server
I don’t want client-server model. I want sharing model. Like with Briar.
The only kind of “servers” might be relays, like in NOSTR, or machines running 24/7 like Briar mailbox.
IP. How would I go about replacing it? I don’t know, I think Yggdrasil authors have written something about their routing model, but 1) it’s represented as ipv6, so IP, 2) it’s far over my head, 3) read the previous, I don’t really want to replace it as much as not to make it the main common layer.
client-server model. I want sharing model. Like with Briar
Guess what
Briar itself, and every pure P2P decentralized network where all nodes are identical… are built on Internet Sockets which inherently require one party (“server”) to start listening on a port, and another party (“client”) to start the conversation.
Briar uses TCP/IP, but it uses Tor routing, which is IMO a smart thing to do
I’m talking about Briar used over BT.
Even
AF_BLUETOOTHsockets are… sockets, where one machine ("server’) opens to listen, and the other (“client”) initiates the stream
Anubis isn’t supposed to be hard to avoid, but expensive to avoid. Not really surprised that a big company might be willing to throw a bunch of cash at it.
This is what I’ve kept saying about POW being a shit bot management tactic. Its a flat tax across all users, real or fake. The fake users are making money to access your site and will just eat the added expense. You can raise the tax to cost more than what your data is worth to them, but that also affects your real users. Nothing about Anubis even attempts to differentiate between bots and real users.
If the bots take the time, they can set up a pipeline to solve Anubis tokens outside of the browser more efficiently than real users.
Yeah but ai companies are losing money so in the long run Anubis seems like it should eventually return to working.
It’s the usual enshittification tactic. Make AI cheap so companies fire tech workers. Keep it cheap long enough that we all have established careers as McDonald’s branch managers, then whack up the prices once they’re locked in.
Costs of solving PoW for Anubis is absolutely not a factor in any AI companies budget. Just the costs of answering one question is millions of times more expensive than running sha256sum for Anubis.
Just in case you’re being glib and mean the businesses will go under regardless of Anubis: most of these are coming from China. China absolutely will keep running these companies at a loss for the sake of strategic development.
Thanks for the info 👍 would not have thought Anubis would be so irrelevant
What the alternative?
Not much for open source solutions. A simple captcha however would cost scrapers more to crack than Anubis.
But when it comes to “real” bot management solutions: The least invasive solutions will try to match User-Agent and other headers against the TLS fingerprint and block if they don’t match. More invasive solutions will fingerprint your browser and even your GPU, then either block you or issue you a tracking cookie which is often pinned to your IP and user-agent. Both of those solutions require a large base of data to know what real and fake traffic actually looks like. Only large hosting providers like CloudFlare and Akamai have that data and can provide those sorts of solutions.
reminder to donate to codeberg and forgejo :)
Question: those artificial stupidity bots want to steal the issues or want to steal the code? Because why they’re wasting a lot of resources scraping millions of pages when they can steal everything via SSH (once a month, not 120 times a second)
I feel like at some point it needs to be active response. Phase 1 is a teergrube type of slowness to muck up the crawlers, with warnings in the headers and response body, and then phase 2 is a DDOS in response or maybe just a drone strike and cut out the middleman. Once you’ve actively evading Anubis, fuckin’ game on.
I think the best thing to do is to not block them when they’re detected but poison them instead. Feed them tons of text generated by tiny old language models, it’s harder to detect and also messes up their training and makes the models less reliable. Of course you would want to do that on a separate server so it doesn’t slow down real users, but you probably don’t need much power since the scrapers probably don’t really care about the speed
I love catching bots in tarpits, it’s actually quite fun
Yeah that was my thought. Don’t reject them, that’s obvious and they’ll work around it. Feed them shit data - but not too obviously shit - and they’ll not only swallow it but eventually build up to levels where it compromises them.
I’ve suggested the same for plain old non-AI data stealing. Make the data useless to them and cost more work to separate good from bad, and they’ll eventually either sod off or die.
A low power AI actually seems like a good way to generate a ton of believable - but bad - data that can be used to fight the bad AI’s. It doesn’t need to be done real-time either as datasets can be generated in advance
A low power AI actually seems like a good way to generate a ton of believable - but bad - data that can be used to fight the bad AI’s.
Even “high power” AIs would produce bad data. It’s currently well known that feeding AI data to an AI model decreases model quality and if repeated, it just becomes worse and worse. So yea, this is definitely viable.
The problem is primarily the resource drain on the server and tarpitting tactics usually increase that resource burden by maintaining the open connections.
The idea is that eventually they would stop scraping you cause the data is bad or huge. But it’s a long term thing, it doesn’t help in the moment.
The promise of money — even diminishing returns — is too great. There’s a new scraper spending big on resources every day while websites are under assault.
In the paraphrased words of the finance industry: AI can stay stupid longer than most websites can stay solvent.
Some guy also used zip bombs against AI crawlers, don’t know if it still works. Link to the lemmy post
Wasn’t this called black ice in Neuromancer? Security systems that actively tried to harm the hacker?
These crawlers come from random people’s devices via shady apps. Each request comes from a different IP
Most of these AI crawlers are from major corporations operating out of datacenters with known IP ranges, which is why they do IP range blocks. That’s why in Codeberg’s response, they mention that after they fixed the configuration issue that only blocked those IP ranges on non-Anubis routes, the crawling stopped.
For example, OpenAI publishes a list of IP ranges that their crawlers can come from, and also displays user agents for each bot.
Perplexity also publishes IP ranges, but Cloudflare later found them bypassing no-crawl directives with undeclared crawlers. They did use different IPs, but not from “shady apps.” Instead, they would simply rotate ASNs, and request a new IP.
The reason they do this is because it is still legal for them to do so. Rotating ASNs and IPs within that ASN is not a crime. However, maliciously utilizing apps installed on people’s devices to route network traffic they’re unaware of is. It also carries much higher latency, and could even allow for man-in-the-middle attacks, which they clearly don’t want.
Honestly, man, I get what you’re saying, but also at some point all that stuff just becomes someone else’s problem.
This is what people forget about the social contract: It goes both ways, it was an agreement for the benefit of all. The old way was that if you had a problem with someone, you showed up at their house with a bat / with some friends. That wasn’t really the way, and so we arrived at this deal where no one had to do that, but then people always start to fuck over other people involved in the system thinking that that “no one will show up at my place with a bat, whatever I do” arrangement is a law of nature. It’s not.
Yep
Or your TV or IOT devices. Residential proxies are extremely shady businesses.
Yes. A nonprofit organization in Germany is going to be launching drone strikes globally. That is totally a better world.
Its also important to understand that a significant chunk of these botnets are just normal people with viruses/compromised machines. And the fastest way to launch a DDOS attack is to… rent the same botnet from the same blackhat org to attack itself. And while that would be funny, I would also rather orgs I donate to not giving that money to blackhat orgs. But that is just me.
Gosh. Corporations are rampantly attempting to access resources so they can perform copyright infringement en-masse. I wonder if there is a legal mechanism to stop them? Oh, no there isn’t because our government is fully corrupted.
I think, in this particular case, it’s aggressive apathy/incompetence and not malice. Remember, Trump didn’t even know what Nvidia was.
AI’s don’t have a skin color or use the bathroom so you can’t whip your cult into a frenzy by Othering it. You can’t solidify your fascism by getting bogged down in the details of IP law.
Okay what about…what about uhhh… Static site builders that render the whole page out as an image map, making it visible for humans but useless for crawlers 🤔🤔🤔
Accessibility gets throw out the window?
I wasn’t being totally serious, but also, I do think that while accessibility concerns come from a good place, there is some practical limitation that must be accepted when building fringe and counter-cultural things. Like, my hidden rebel base can’t have a wheelchair accessible ramp at the entrance, because then my base isn’t hidden anymore. It sucks that some solutions can’t work for everyone, but if we just throw them out because it won’t work for 5% of people, we end up with nothing. I’d rather have a solution that works for 95% of people than no solution at all. I’m not saying that people who use screen readers are second-class citizens. If crawlers were vision-based then I might suggest matching text to background colors so that only screen readers work to understand the site. Because something that works for 5% of people is also better than no solution at all. We need to tolerate having imperfect first attempts and understand that more sophisticated infrastructure comes later.
But yes my image map idea is pretty much a joke nonetheless
Don’t worry, we were never going to make anything 100% accessible anyway, that would be impossible.
AI these days reads text from images better than humans can
AI is pretty good at OCR now. I think that would just make it worse for humans while making very little difference to the AI.
The crawlers are likely not AI though, but yes OCR could be done effectively without AI anyways. This idea ultimately boils down to the same hope Anubis had of making the processing costs large enough to not be worth it.
OCR could be done effectively without AI
OCR has been neural nets even before convolutional networks emerged in the 2010s
Yeah you’re right, I was using AI in the colloquial modern sense. My mistake. It actually drives me nuts when people do that. I should have said “without compute-heavy AI”.
My mistake
hold on I am still somewhat new to Fedi & not fully used to people being polite
Computer vision models can read/parse pixel geometry.
Is there nightshade but for text and code? Maybe my source headers should include a bunch of special characters that then give a prompt injection. And sprinkle some nonsensical code comments before the real code comment.
Maybe like a bunch of white text at 2pt?
Not visible to the user, but fully readable by crawlers.
If a bot can’t read it, nor can a visually impaired user.
Well if it’s a prompt injection to fuck with llms you don’t want any users having to read it anyway, vision impaired or no.
You missed my point. A prompt injection to fuck with LLMs would be read by a visually impaired user’s screen reader.
It’s always a cat-n-mouse game.
Except previously bombarding another person’s server for personal gain was illegal.
Is there a migration tool? If not would be awesome to migrate everything including issues and stuff. Bet even more people would move.
Codeberg has very good migration tools built in. You need to do one repo at a time, but it can move issues, releases, and everything.
Can there be a challenge that actually does some maliciously useful compute? Like make their crawlers mine bitcoin or something.
Did you just say use the words “useful” and “bitcoin” in the same sentence? o_O
The saddest part is, we thought crypto was the biggest waste of energy ever and then the LLMs entered the chat.
At least LLMs produce something, even if it’s slop, all crypto does is… What does crypto even do again?
Crypto does drug sales and fraud!
Bro couldn’t even bring himself to mention protein folding because that’s too socialist I guess.
LLMs can’t do protein folding. A specifically-trained Machine Learning model called AlphaFold did. Here’s the paper.
Developing, training and fine tuning that model was a research effort led by two guys who got a Nobel for it. Alphafold can’t do conversation or give you hummus recipes, it knows shit about the structure of human language but can identify patterns in the domain where it has been specifically and painstakingly trained.
It wasn’t “hey chatGPT, show me how to fold a protein” is all I’m saying and the “superhuman reasoning capabilities” of current LLMs are still falling ridiculously short of much simpler problems.
The crawlers for LLM are not themselves LLMs.
