The Cognitive Dark Forest
38 points by refaktor
38 points by refaktor
I think this presupposes that people build things for credit or reward. Often (I would argue mostly) I build something just to have the thing. A big company snipes me and builds the thing into their own product? Great, all the better, now I have options. Maybe more versions will show up.
Inasmuch as open source is "building things so that they exist", this will be positive for open source.
Yeah, I find arguments that "I shouldn't put work out there since it might be exploited" surprising, because the joy of code and the internet generally is that someone can "exploit" my work without it costing me anything at all - copies are free.
I have to remind myself that different people have different value systems.
Similar to why I always use Apache 2 or MIT or BSD for my open source work rather than GPL or similar. I want people to be able to use my work with as few restrictions as possible.
I think that, in practice, with (a)gpl vast majority of people will be able to use your work. It’s the corporations that have the biggest issue with (a)gpl.
I want people who work at companies with an allergic reaction to the GPL to be able to use my work without having to fight their legal department first.
Corporations can and even do use copyleft code all the time, it literally just means that they have to share any enhancements if they’re modifying or adding to that code, which seems to make sense to me.
Or not! Since most corporations are happy to just ignore the requirements of the GPL and do whatever they like anyway.
I find arguments that "I shouldn't put work out there since it might be exploited" surprising, because the joy of code and the internet generally is that someone can "exploit" my work
God, the passive voice here. Your joy. You get joy out of it, the joy is not inherent in companies exploiting work, and there are plenty of people who do not get joy out of being exploited. The difference is, that not only are LLMs forcing your view of exploitation upon the rest of the industry, but here you are in comments constantly defending it, too.
Perhaps a more coherent version of that position is "has frequently been exploited to achieve goals which I fundamentally oppose".
Someone who believes e.g. "FAANG are a net negative to the world" is going to have a hard time feeling good about their OSS work being easy to use at those places.
Both Instagram and Threads are built on Django so I guess I've come to terms with that one already.
That doesn't follow to me. I think Facebook is probably net negative, but I wouldn't prefer the world if Facebook was built with crappier libraries and tools and had to reinvent more wheels. Sure, maybe the company would be slightly less successful, but that isn't the only effect.
Yeah, I find arguments that "I shouldn't put work out there since it might be exploited" surprising, because the joy of code and the internet generally is that someone can "exploit" my work without it costing me anything at all
I think what you're missing is the implicit value that building open projects had before: you could become famous. Your project could become a default choice within some technological niche, attracted contributors and grow. In the new era, if you implement an interesting idea, it's just gets trivially AI-washed (as in, the AI re-implementation is called "from scratch" and isn't linked to the original) and gets sold without even giving you any bragging rights. There's no incentive to contribute to the established original.
Like, if AI were around when you guys built Django, in a year's time a large company would just produce something similar and push it onto the Python webdev community through a modicum of marketing. You wouldn't have any benefit of a head-start that consolidated the effort around it.
To clarify, I mostly hear this idea of AI being a natural evolution of opens source sharing ideas from people who got their fame before it happened: you, mitsuhiko, antirez. You guys enjoyed this implied value of building in the open without having to explicitly pursue it: it just how it happened to work. You telling younger people how to enjoy this brave new world sounds a little tone-deaf, to be honest.
Most people still build most things for some sort of reward. It's interesting dynamics where this doesn't affect you, but this doesn't challenge the points of the blog post directly.
Either way, as written at the top, this post is a thought experiment and of course there are edge cases it doesn't cover. It explores possible general principles.
I do not think all is over, and in fact writing this helped me find some existential optimism again. By laying out the fears, I was able to see their weaknesses (which might become another post at some point).
It is also a somewhat privileged position to be able to make something just to see it exists. I too have done that and have been similarly privileged. But the Luddite in me has to ask: if the job market keeps worsening and "the forest" keeps growing, will any of us have that privilege or will we be desperately trying to survive?
Outside of capitalism, this kind of force from AI might really be a boon. Inside it, from my perspective, it's only another tool to widen all the socioeconomic gaps.
So-called "AI" in all its incarnations is solving one problem above all(for a certain class of people): wages.
"Often"? Speak for yourself I guess. I do things because I want to be the one doing it. Otherwise why do anything? Every single skill I have is done better by someone else, somewhere.
I have to agree, given how freely AI companies have been able to (legally and illegally) index our hand-crafted "training data" from Github, Reddit, et al, without our permission, and sell it back to us (arguably with increasingly better LLMs) on a subscription. It is eerily similar to stealing the labor of many and keeping the fruits of it for the ones who stole the labor.
Increasingly I feel like the more I share in the open, the more that will train a future LLM that will eventually get used, not just by evil monopolizing corporations, but also by governments or military against me in a direct or indirect form.
By this logic though, shouldn't it be easier than ever to snipe the big companies shamelessly broadcasting their signals across the universe? A solo developer is more empowered than ever to disrupt the status quo, all the while maintaining a very minimal broadcast signal that is virtually undetectable
A piece of feedback for the author: give people a reason to care about what you write very, very early in your writing. Because you submitted this here yourself, I'm assuming you want other people to read it.
Two paragraphs in and I have no idea what this is about - I am always going to close the page. Maybe it's awesome but I will never know.
On the contrary, I'm perfectly happy reading a piece like this that doesn't start with a complete summary of itself or some grandiose exaggerated claim. I'm usually only here on lobsters when I've got time to kill, and personally I don't mind trusting the author to deliver their point in their own style by the end (especially for a piece as short as this one). This isn't a criticism on your reading style (not everybody has the time I do or would choose to dedicate it the way I do), it's just a reminder that other reading styles exist and the author might not be catering to yours specifically.
I'm not asking for a complete summary or a grandiose claim. I'm looking for literally any indication of "the point". If there is no point, sure, I agree that different people may engage differently. But usually there is one. Don't bury the lede.
For sure I would not have commented this on a post not submitted by the author BTW. It's no commentary on the writing itself. If you're sharing your writing somewhere, you're doing it for a reason.
I'm not trying to be smart or ironic, really asking - but besides title, subtitle and text I'm not sure in this case how I should do that? Please provide more info.
the text is ok, but the reader needs to know the reference (liu cixin). i don't think that a long, deep dive into the dark forest concept would improve the text; it will probably make it a case of TLDR.
I disagree that they need to know the text. The key idea being referenced in those works is mentioned in this text directly. The reference serves more as a citation for the analogy. It's just a way to introduce a kind of alternate "Great Filter Hypothesis" / Fermi Paradox and the. Applies that idea to ideas instead of civilisations.
The 3-body problem series as a whole is not a load bearing concept in this piece, by any means.
Well, I honestly can't really help you since I don't know the point. Maybe the other thread is the answer - I can't be sure.
This really presupposes that execution solely consists in writing code.
It's a thought experiment so it presupposes A LOT. You are correct, execution is not just writing code, and even when it is, it's not this simple.
So I won't repeat myself too much: https://lobste.rs/s/bpotqb/cognitive_dark_forest#c_f4lwum
maybe the question here is which use are authors of gratis work willing to accept, and how can they enforce it. not publishing is an extreme but effective way of enforcement.
We will again build and innovate in private, hide, not share knowledge, mistakes, ideas.
I keep almost most of my code private and local now. With few incentives to share or learn in depth, I see the future as stifled or constrained innovation; a result of a negative trickle up as companies will have less novel open source and a reduced common knowledge and standards footprint on which to build products.
As an ex-Googler with a privacy focus who's paid close attention to this concern since long before ML got to its current level of plausible output... I don't think companies are actually very good at using the data available to them to generate product ideas. They react a lot more than they act.
So I don't share this author's concern about megacorps figuring out that an idea "is pregnant" because I simply don't think the people who would do that are good enough at asking the kinds of questions that would lead to such insights.
The thing about critique of capital being subsumed into capital, though, that's a real and very serious problem. That doesn't come from these new technologies, it's always been the case. Think about how rock music was, when it was new, a direct challenge to the status quo.... now try to name a mainstream song from the last 30 years that is.
May I ask if this piece was written with AI assistance? (Not trying to pick an argument or be judgemental, genuinely curious, if only to train my personal "AI detector".)
These snippets made me think so.
But in the cognitive dark forest, the most dangerous actor is not your peer. It’s the forest itself.
You think of something new and express it - (...) - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger.
Resistance isn’t suppressed. It’s absorbed.