Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.
That’s not what an algorithms researcher means when we talk about “understanding”. Obviously we know the mechanism by which it operates, it’s not an unknown alien technology that dropped into our laps.
Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?
Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.
It cannot possibly develop agency.
I don’t believe I’ve suggested anywhere that I think it will, but I’ll play around with this concern anyway… There’s a lot of discussion going on about having models feed back on themselves to learn from their own output. I don’t find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it’s not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don’t think it’s all that far fetched to consider the possibility of it happening.
Sure, but this outcome is not at all surprising. There are plenty of smart AI people that have nuanced views of what kind of threat could be posed by recklessly unleashing tools that we don’t fully understand into the hands of people who are likely to do harmful things with them.
It’s not surprising that those valid nuanced concerns get translated into overly simplistic misrepresentations entangled with pop sci fi panic around rogue AI as they try to move into public discourse.
AI person reporting in. Without saying whether or not I personally believe that the current tools will lead to the end of humanity, I’ll point out a few possibilities that I find concerning about what’s going on:
The hype around AI is being used to justify mass layoffs, where humans are being replaced by tools that do a questionable job and can’t really understand the things those humans could understand. Whether or not the AI can do as good of a job according to some statical measurement is less relevant than the fact that a human is less likely to make an extremely grave mistake and more likely to be able to recognize when that does happen. I’m concerned this will lead to cross-industry enshitification on an unprecedented scale.
The foundation models consume a huge amount of energy. The more impressive you want it to be, the more energy it needs. As long as the data centers which run them are dependent on fossil fuels, they’ll be pumping a huge amount of carbon in the air just to do replace jobs that we didn’t need to have replaced.
As these tools are used more and more, they’re going to end up “learning” from content created by themselves instead of something that’s closer to a ground truth. It’s hard to predict what kind of degradation of service will come from this, but the more we create systems that rely on these tools, the more harm it will do to us.
Given the cost and nature of these tools, they’re likely to yield the most benefit to moneyed interests that want to automate the systems that maintain their power and wealth. E.g. generating large amounts of convincing disinformation to manipulate the public into supporting politicians or policies that benefit a small number of wealthy people in the short term while locking humanity into a path towards destruction.
And none of this accounts for possible future iterations of AI tools that may be far more capable than what exists today. That future technology will most likely be controlled by powerful people who are primarily interested in using it to bolster the systems that keep them in power, to the detriment of humanity as a whole.
Personally I’m far less concerned about a malicious AI intentionally doing harm to humanity than AI being used as a weapon by unscrupulous people.
Yeah the only way it would be that high is if it lumps C and C++ together. But at that point it may be an underestimate.
A better language wouldn’t have any need to use POSIX signals in this way.
I really don’t think that there is any perfect programming language.
You’d be wrong 🦀🦀🦀🦀🦀
I think a long time ago a vicious cycle began in the advertising space where predatory ads had more incentive to pay for ad space, so sensible people start to perceive ads in general as predatory. Now no sensible advertiser that’s trying to promote a legitimate product for legitimate reasons will do so by buying ad space, thus reinforcing the increasingly accurate perception that all ads are predatory.
Still much better than C++ templates, and I say that as someone who used to genuinely love C++ template metaprogramming. Admittedly Rust traits+generics are far more limited than C++ templates, but very often I find that to be a positive. The list of things that I feel traits+generics are missing is small and rapidly shrinking.
A good enough compiler would prevent them from happening 🦀
Sorry to be pedantic but Rust only guarantees no data races can happen. It does not prevent race conditions more generally.
Don’t get me wrong, I absolutely love the language for sparing me from the hell that is data races, but the language alone won’t solve race conditions for you.
I use thread sanitizer and address sanitizer in my CI, and they have certainly helped in some cases, but they don’t catch everything. In fact it’s the cases that they miss which are by far the most subtle instances of undefined behavior of all.
They also slow down execution so severely that I can’t use them when trying to recreate issues that occur in production.
I’m not sure if you’re genuinely asking what a test suite is or if this is a sarcistic joke about how no one bothers to test their C++ code.
And even if you do get to use pure modern C++ you’ll still get burned by subtle cases of undefined behavior (e.g. you probably haven’t memorized every iterator invalidation rule for every container type) that force you to spend weeks debugging an inexplicable crash that happened in production but can only be recreated in 1/10000 runs of your test suite, but vanishes entirely if you compile in debug mode and try to use gdb.
And don’t even get me started on multi-threading and concurrency.
There’s a difference between “You have to decide when to synchronize your state” and “If you make any very small mistake that appears to be perfectly fine in the absence of extremely rigorous scrutiny then this code block will cause a crash or some other incomprehensible undefined behavior 1/10000 times that it gets run, leaving you with no indication of what went wrong or where the problem is.”
I’m not saying you can’t do multi-threading or concurrency in C++. The problem is that it’s far too easy to get data races or deadlocks by making subtle syntactical mistakes that the compiler doesn’t catch. pthreads does nothing to help with that.
If you don’t need to share any data across threads then sure, everything is easy, but I’ve never seen such a simple use case in my entire professional career.
All these people talking about “C++ is easy, just don’t use pointers!” must be writing the easiest applications of all time and also producing code that’s so inefficient they’d probably get performance gains by switching to Python.
Your graph also cuts out early. Eventually you want to get performance gains with multi-threading and concurrency, and then the line drops all the way into hell.
Yeah that’s my big takeaway here: If the people who are rolling out this technology cannot make these assurances then the technology has no right to exist.
Literally nothing you’ve said gives any indication that you actually know the current state of foundation model research. I won’t claim it’s my research specialty, but I work directly with people whose full time job is research and tuning on foundation models, and everything I’m saying is being relayed from conversations that I have with them.
“Cannot ever possibly be used like that”… Like what specifically? To drive a car? That’s being done. To give financial advice? That’s being done. To console people who are suicidal or at risk of harming themselves? That’s being done. To make kill / no kill decisions in an active warzone? It’s being considered (if not already being done in secret).
This technology is being used in extremely consequential positions despite having very weak guarantees around safety. This should give any reasonable person pause. I’m not taking any firm stance on whether this specific regulation is the right approach, but if you think there should be no legal accountability for the outcomes of how this technology gets used then I guess you’re someone who thinks seatbelts should be optional in cars and it’s okay for airplanes to fall out of the sky due to neglect.