And this is why Digit wanted a clarification. Let’s make a quick split between “Tech Bro” and Technology Enthusiast.

I’d maybe label myself a “tech guy”, and forego the “bro”, but I could see other people calling me a “tech bro”. I like following tech trends and innovations, and I’m often a leading adopter of things I’m interested in if not bleeding edge. I like talking about tech trends and will dive into subjects I know. I’ll be quick to point out how machine learning can be used in certain circumstances, but am loudly against “AI”/LLMs being shoved into everything. I’m not the CEO or similar of a startup.

Your specific and linked definition requires low critical thinking skills, big ego and access to “too much” money. That doesn’t describe me and probably doesn’t describe Digit’s network.

Their whole point seemed to be that the tech-aware people in their sphere are antagonistic to the idea of “AI” being added to everything. That doesn’t deserve derision.

I’m happy you provided a few examples. This is good for anyone else reading along.

Equifax in 2017: Penalty was, let’s assume the worst case, 700$M. The company in 2017 made 3.3$B, and I’d assume that was after the penalty, but even if it wasn’t, that was a penalty of 27% of revenue. That actually seems like it would hurt.

TSB in 2022: Fined ~48.6£M by two separate agencies. TSB made 183.5£M in revenue in 2022, still unclear if that was pre- or post- penalty, but this probably actually hurt.

Uber in 2018: your link suggests Uber avoided any legal discovery that might have exposed their wrongdoing. There are no numbers in the linked article and a search suggest the numbers are not public. Fuck that. A woman was killed by an AI driven car and the family deserves respect and privacy, but uber DOES NOT. Because it’s not a public record, I can’t tell how much they paid out for the death of the victim, and since uber is one of those modern venture-capital-loss-leader companies, this is hard to respond to.

I’m out of time – and won’t likely be able to finish before the weekend, so trying to wrap up – and Boeing seems complicated and I’m more familiar with Crowdstrike and I know they fucked up. In both cases, I’m not sure how much of a penalty they paid out relative to income.

I’ll cede the point: There are some companies who have paid a price for making mistakes. When you’re talking companies, though, the only metric is money-paid/money-earned. I would really like there to be criminal penalties for leadership who chase profit over safety, so there’s a bit of ‘wishful thinking’ in my worldview. If you kill someone as a human being (or 300 persons, Boeing), you end up with years in prison, but company just pays 25% of it’s profit that year instead.

I still think Cassandra is right, and that more often than not, software companies are not held responsible for their mistakes. And I think your other premise, that ‘if software is better at something’ carries a lot: Software is good at explicit computation, such as math, but is historically incapable of empathy (a significant part of the original topic… I don’t want to be a number in a cost/benefit calculation). I don’t want software replacing a human in the loop.

Back to my example of a flock camera telling the police that a stolen car was identified… the software was just wrong. The police department didn’t admit any wrongdoing and maaaaybe at some point the victim will be compensated for their suffering, but I expect flock will not be on the hook for that. It will be the police department, which is funded by taxpayers.

Reading your comments outside this thread, I think we would agree on a great many things and have interesting conversations. I didn’t intend to come across as snide, condescending or arrogant. You made the initial point, cassandra challenged you and I agreed with them, so I joined where they seemed not to.

The “bizarre emotion reaction” is probably that I despise AI and want it nowhere near any decision-making capability. I think that as we embed “AI” in software, we will find that real people are put at more risk and that software companies will be able to deflect blame when things go wrong.

The burden of proof is on you. Show me one example of a company being held liable (really liable, not a settlement/fine for a fraction of the money they made) for a software mistake that hurt people.

The reality is that a company can make X dollars with software that makes mistakes, and then pay X/100 dollars when that hurts people and goes to court. That’s not a punishment, that’s a cost of business. And the company pays that fine and the humans who mode those decisions are shielded from further repercussions.

When you said:

the idea that the software vendor could not be held liable is farcical

We need YOU to back that up. The rest of us have seen it never be accurate.

And it gets worse when the software vendor is a step removed: See flock cameras making big mistakes. Software decided that this car was stolen, but it was wrong. The police intimidated an innocent civilian because the software was wrong. Not only were the police not held accountable, Flock was never even in the picture.

It seemed somewhat topical to me. Google’s censorship is the same trend of enshittification that Yar is talking about.

There are tons of other comments talking about the censorship issue. Using this moment to plug open source software is not unreasonable.

Thanks for your reply, and I can still see how it might work.

I’m curious if you have any resources that do some end-to-end examples. This is where I struggle. If I have an atomic piece of code I need and I can maybe get it started with a LLM and finish it by hand, but anything larger seems to just always fail. So far the best video I found to try a start-to-finish demo was this: https://www.youtube.com/watch?v=8AWEPx5cHWQ

He spends plenty of time describing the tools and how to use them, but when we get to the actual work, we spend 20 minutes telling the LLM that it’s doing stuff wrong. There’s eventually a prototype, but to get there he had to alternate between ‘I still can’t jump’ and ‘here’s the new error.’ He eventually modified code himself, so even getting a ‘mario clone’ running requires an actual developer and the final result was underwhelming at best.

For me, a ‘game’ is this tiny product that could be a viable unit. It doesn’t need to talk to other services, it just needs to react to user input. I want to see a speed-run of someone using LLMs to make a game that is playable. It doesn’t need to be “fun”, but the video above only got to the ‘player can jump and gets game over if hitting enemy’ stage. How much extra effort would it take to make the background not flat blue? Is there a win condition? How to refactor this so that the level is not hard-coded? Multiple enemy types? Shoot a fireball that bounces? Power Ups? And does doing any of those break jump functionality again? How much time do I have to spend telling the LLM that the fireball still goes through the floor and doesn’t kill an enemy when it hits them?

I could imagine that if the LLM was handed a well described design document and technical spec that it could do better, but I have yet to see that demonstrated. Given what it produces for people publishing tutorials online, I would never let it handle anything business critical.

The video is an hour long, and spends about 20 minutes in the middle actually working on the project. I probably couldn’t do better, but I’ve mostly forgotten my javascript and HTML canvas. If kaboom.js was my focus, though, I imagine I could knock out what he did in well under 20 minutes and have a better architected design that handled the above questions.

I’ve, luckily, not yet been mandated that I embed AI into my pseudo-developer role, but they are asking.

I think this is what will kill vibe coding, but not before there’s significant damage done. Junior developers will be let go and senior devs will be told they have to use these tools instead and to be twice as efficient. At some point enough major companies will have had data breaches through AI-generated code that they all go back to using people, but there will be tons of vulnerable code everywhere. And letting Cursor touch your codebase for a year, even with oversight, will make it really tricky to find all the places it subtly fucked up.

I have 3 questions, and I’m coming from a heavily AI-skeptic position, but am open:

1. Do you believe that providing all that context, describing the existing patterns, creating an implementation plan, etc, allows the AI to both write better code and faster than if you just did it yourself? To me, this just seems like you have to re-write your technical documentation in prose each time you want to do something. You are saying this is better than ‘Do XYZ’, but how much twiddling of your existing codebase do you need to do before an AI can understand the business context of it? I don’t currently do development on an existing codebase, but every time I try to get these tools to do something fairly simple from scratch, they just flail. Maybe I’m just not spending the hours to build my AI-parsable functional spec. Every time I’ve tried this, asking something as simple as (and paraphrased for brevity) “write an Asteroids clone using JavaScript and HTML 5 Canvas” results in a full failure, even with multiple retries chasing errors. I wrote something like that a few years ago to learn Javascript and it took me a day-ish to get something that mostly worked.

2. Speaking of that context. Are you running your models locally, or do you have some cloud service? If you give your entire codebase to a 3rd party as context, how much of your company’s secret sauce have you disclosed? I’d imagine most sane companies are doing something to make their models local, but we see regular news articles about how ChatGPT is training on user input and leaking sensitive data if you ask it nicely and I can’t imagine all the pro-AI CEOs are aware of the risks here.

3. How much pen-testing time are you spending on this code, error handling, edge cases, race conditions, data sanitation? An experienced dev understands these things innately, having fixed these kinds of issues in the past and knows the anti-patterns and how to avoid them. In all seriousness, I think this is going to be the thing that actually kills AI vibe coding, but it won’t be fast enough. There will be tons of new exploits in what used to be solidly safe places. Your new web front-end? It has a really simple SQL injection attack. Your phone app? You can tell it your username is admin’joe@google.com and it’ll let you order stuff for free since you’re an admin.

I see a place for AI-generated code, for instant functions that do something blending simple and complex. “Hey claude, write a function to take a string and split it at the end of every sentence containing an uppercase A”. I had to write weird functions like that constantly as a sysadmin, and transforming data seems like a thing an AI could help me accelerate. I just don’t see that working on a larger scale, though, or trusting an AI enough to allow it to integrate a new function like that into an existing codebase.

I’d wager that the votes are irrelevant. Stock overflow is generously <50% good code and is mostly people saying ‘this code doesn’t work – why?’ and that is the corpus these models were trained on.

I’ve yet to see something like a vibe coding livestream where something got done. I can only find a lot of ‘tutorials’ that tell how to set up tools. Anyone want to provide one?

I could… possibly… imagine a place where someone took quality code from a variety of sources and generate a model that was specific to a single language, and that model was able to generate good code, but I don’t think we have that.

Vibe coders: Even if your code works and seems to be a success, do you know why it works, how it works? Does it handle edge cases you didn’t include in your prompt? Does it expose the database to someone smarter than the LLM? Does it grant an attacker access to the computer it’s running on, if they are smarter than the LLM? Have you asked your LLM how many 'r’s are in strawberry?

At the very least, we will have a cyber-security crisis due to vibe coding; especially since there seems to be a high likelihood of HR and Finance vibe coders who think they can do the traditional IT/Dev work without understanding what they are doing and how to do it safely.

This is my fear. It’s still possible, barely, to buy a dumb TV. When my current fridge/dishwasher/stove/etc dies in a few years, will there even be a dumb version? Will it cost 5x the price of a spyware version? How about my thermostat. HVAC? Car? And will attempting to disable any of this spyware land me in prison?

Right now, uninformed/unaware/stupid people are affected by this. Pretty soon, everyone will be, or they will have to forego things we consider to be necessities now, like refrigeration and cell phones or be rich enough to buy the privacy-focused models.

I can’t immediately find it, but I just saw another post about a new privacy-focused cellphone with a huge price tag. The established manufacturers have a cost advantage. Samsung et al. can easily make a new fridge with fewer consumer rights, but a new company will have to spend tons of capital to make a factory to put out a comparable product; and they won’t have the advantage of selling your data to subsidize the price.

Privacy is and will become more-so a commodity unless we fight for it.

That new hire might eat resources, but they actually learn from their mistakes and gain experience. If you can’t hold on to them once they have experience, that’s a you problem. Be more capitalist and compete for their supply of talent; if you are not willing to pay for the real human, then you can have a shitty AI that will never grow beyond a ‘new hire.’

The future problem, though, is that without the experience of being a junior dev, where do you think senior devs come from? Can’t fix crappy code if all you know how to do is engineer prompts to a new hire.

“For want of a nail,” no one knew how to do anything in 2030. Doctors were AI, Programmers were AI, Artists were AI, Teachers were AI, Students were AI, Politicians were AI. Humanity suffered and the world suffocated under the energy requirements of doing everything poorly.

I fully agree: Companies and their leadership should be held accountable when they cut corners and disregard customer data security. The ideal solution would be that a company is required to not store any information beyond what is required to provide the service, a la GDPR, but with a much stricter limit. I would put “marketing” outside that boundary. As a youtube user, you need literally nothing, maybe a username and password to retain history and inferred preferences, but trying to collect info about me should be punished. If your company can’t survive without targeted content, your company should not survive.

In bygone days, your car’s manufacturer didn’t know anything about you and we still bought cars. Not to start a whole new thread, but this ties in to right-to-repair and subscriptions for features as well. I did not buy a license to the car, I bought the fucking car; a license to use the car is called a lease.

I understand what you are saying, and what you want… but admitting fault publicly is a huge liability, as they have then stated it was their negligence that caused the issue. (bear with me and read this wall of text – or skip to the last paragraph)

I’ve worked in the Sec Ops space, and it’s an arms race all the time. There are tools to help identify issues and breaches quickly, but the attack surface is just not something that can be managed 100%. Even if you know there is a problem, you probably have to send an issue to a developer team to update their dependency and then they might need to change their code as well and get a code review approved and get a window to promote to production. A Zero-Day vulnerability is not something you can anticipate.

You’ve seen the XKCD of the software stack where a tiny peg is propping up the whole thing? The same idea applies to security, but the tiny peg is a supply chain attack where some dependency is either vulnerable, or attacked by malicious actors and through that gain access to your environment.

Maybe your developers leverage WidgetX1Z library for their app, and the WidgetX1Z library just updated with a change-log that looks reasonable, but the new code has a backdoor that allows an attacker to compromise your developers computer. They now have a foothold in your environment even with rigorous controls. I’ve yet to meet a developer who didn’t need, or at least want, full admin rights on their box. You now have an attacker with local admin inside your network. They might trip alarms, but by then the damage might be done and they were able to harvest the dev database of user accounts and send it back home. That dev database was probably a time-delayed copy of prod, so that the developer could be entirely sure there were no negative impacts of their changes.

I’m not saying this is what happened to Plex, but the idea that modern companies even CAN fully control the data they have is crazy. Unless you are doing full code reviews of all third-party libraries and changes or writing everything in-house (which would be insane), with infallible review, you cannot fully protect against a breach. And even then I’m not sure.

The real threat here is what data do companies collect about us? If all they have is a username, password and company-specific data, then the impact of a breach is not that big – you, as a consumer, should not re-use a password. When they collect tons of other information about us such as age, race, location, gender, sex, orientation, habits, preferences, contacts, partners, politics, etc, then those details become available for anyone willing to pay. We should use breach notifications like this to push for stronger data laws that prevent companies from collecting, storing, buying or selling personal data about their customers. It is literally impossible for a company to fully protect that information, so it should not be allowed.

Full agree. It’s scary. These companies have collected enough data on us all – sometimes (often?) through things we didn’t directly use and thus didn’t need to accept any T&C for, such as surveillance cameras in a business or public street – that they can predict our actions, moods, and make inferences about our lives.

They have been doing this for YEARS, and they are constantly getting better. They don’t even need health data, but I can guarantee they want it. I remember noticing that we had a phase where my wife was being advertised baby products on her streaming service. We were not having another child, but the timing was eerily close to the interval between #1 and #2. I actually just had a hesitation about divulging that I have 2 kids, but then said fuck it, they already know.

Add to all that the ‘for the children’ angle, which I’ve always hated. It’s such a transparent lie that anyone with a lick of common sense can see through it. For anyone even on the fence, this is the foot in the door: Allow them the ability to track you ‘for the children’ and they will track you for the corporation as well, and the government, and your ex-boyfriend who is now a cop.

Fight this shit.

It’s almost like the privacy alarmists, who have been screaming for decades, were on to something.

Some people saw the beginning of Minority Report and thought, ‘that sounds like a good idea.’

We used to be in a world where it was unfeasible to watch everyone, and you could get away with small ‘crimes’ like complaining about the president online because it was impossible to get a warrant for surveillance without any evidence. Now, we have systems like Flock¹ cameras², ChatGPT and others that generate alerts to law enforcement on their own, subverting a need for a warrant in the first place. And more and more frequently, you both can’t opt out and are unable to avoid them.

For now, the crime might be driving a car with a license plate flagged as stolen (or one where OCR mistakes a number), but all it takes is a tiny nudge further towards fascism before you can be auto-SWATted because the sentiment of your draft sms is determined to be negative towards the fuhrer.

Even now, I’m sacrificing myself to warn people. This message is on the internet and federated to multiple instances. There’s no way I can’t be identified by it with enough resources. Once it’s too late, I’ll be already on the list of people to remove.

I really like the description of AI coding as ‘custom stack overflow generator’ because it really sells the flaws as well, to an experienced dev. We go to stack overflow for help with some weird quirk of a language or find an obscure library that solves our specific need.

I think vibe coding is cobbling together a project from a bunch of stack overflow posts – and they only use the question part of the post.

Fully agreed. On the service-provider side, we have ‘safe harbor’ laws: A site isn’t liable for copyrighted user-generated content as long as they have mechanisms to take down items when notified.

Liability-wise: The payment processors should have no fucking insight into what is being sold, only that they handle the transactions. Therefore, they should have no liability, similar to “safe harbor”.

Reputation-wise: I can almost see a history where Visa, for example, used a statement like “we don’t handle transactions for X” as a marketing ploy… but that is way past where we are. There’s no chance of reputational damage to a payment processor for the items for which they handled a payment. Combined with the above, if I say I’m giving $20 to Tim, you give $20 to Tim and take it from me. Done. Not your problem.

As another commenter stated, the payment processor should be a dumb pipe, and anything illegal being sold should be a liability for the seller or buyer. The idea of a moral judgement of the processor is as stupid as a water pipe to your house cutting off the flow if your shower runs too long.

The real problem is the politicians, or lobbyists/influencers, who are sending bribes to each other to gain advantage… but visa doesn’t have a problem handling a venmo transaction for ‘tuition’.

Let me buy horny games until after you block world superpower corruption first. But honestly, don’t even do that. Just handle moving the money when someone send it. That’s your only job.

This look to be amazing. I live by a CLI, but most browsers have some keyboard shortcuts at best. This one seems to have the capability to be fully KB using a VIM-like command line, which is awesome.

Thanks for the recommendation!

Like many things, a tool is only as smart as the wielder. There’s still a ton of critical thinking that needs to happen as you do something as simple as bake bread. Using an AI tool to suggest ingredients can be useful from a creative perspective, but should not be assumed accurate at face value. Raisins and Dill? maybe ¯\(ツ)/¯, haven’t tried that one myself.

I like AI, for being able to add detail to things or act as a muse, but it cannot be trusted for anything important. This is why I’m ‘anti-AI’. Too many people (especially in leadership roles) see this tool as a solution for replacing expensive humans with something that ‘does the thinking’; but as we’ve seen elsewhere in this thread, AI CANT THINK. It only suggests items that are statistically likely to be next/near based on its input.

In the Security Operations space, we have a phrase “trust but verify”. For anything AI, I would use 'doubt, then verify" instead. That all said. AI might very well give you a pointer to the place to ask how much motrin an infant should get. Hopefully, that’s your local pediatrician.

I’m SO mad at this story. There is no reason to charge the parents. As others have stated, helicoptering kiddos is detrimental, and they need to be allowed to roam their environment – That can come at the cost of danger, but we cannot be expected to grow with 0 risk.

Sure, as a parent, you can state: ‘don’t go there’, and ‘always look both ways’, but kids are kids and there’s only so much you can enforce without being overbearing. In this scenario, without video evidence, there’s no clear conclusion about fault for either the driver or the child.

I’m okay with letting the driver off (criminally, let insurance pay the family but don’t put the driver in jail) and acknowledging this as an accidental death, especially since he stuck around and is complying. Charging the parents for negligence, though, is just fucking brutal when they are suffering the loss of a child, not to mention the impact on the older son, who probably is feeling an unreasonable amount unreasonable of guilt: “I could have held his hand; I could have reminded him of the road…” (not his quotes, my presumed internal dialog). Again, as others have stated, this is a city planning problem, not a parental one: If there was a way to walk to a grocery store that didn’t cross a 4-lane road, that’d be a better option, but there are plenty of places where that is not possible.

These parents do NOT need the extra burden of being held legally liable for an accident and anyone blaming them for this without knowing them personally and being able to describe other aspects of their parent as negligent is just an asshole.

As with other responses, I recommend a local model, for a vast number of reasons, including privacy and cost.

Ollama is a front end that lets you run several kinds of models on Windows and Linux. Most will run without a GPU, but the performance will be bad. If your only compute device is a laptop without a GPU, you’re out of luck running things locally with any speed… that said, if you need to process a large file and have time to just let the laptop cook, you can probably still get what you need overnight or over a weekend…

If you really need something faster soon, you can probably buy any cheap($5-800) off-the-shelf gaming pc from your local electronics store like best buy, microcenter, walmart, and get more ‘bang for your buck’ over a longer term running a model locally, assuming this isn’t a one-off need. Aim for >=16GB RAM on the PC itself and >=10GB on the GPU for real-time responses. I have a 10GB RTX 3080 and have success running 8B models on my computer. I’m able to run a 70B model, but it’s a slideshow. The ‘B’ metric here is parameters and context(history). Depending on what your 4k-lines really means (book pages/printed text?, code?) a 7-10B model is probably able to keep it all ‘loaded in memory’ and be able to respond to questions about the file without forgetting parts of it.

From a privacy perspective, I also HIGHLY recommend not using the various online front ends. There’s no guarantee that any info you upload to them stays private and generally their privacy policies have a line like ‘we collect information about your interactions with us including but not limited to user generated content, such as text input and images…’ effectively meaning anything your send them is theirs to keep. If your 4k line file is in any way business related, you shouldn’t send it to a service you don’t operate.

Additionally, as much as I enjoy playing with these tools, I’m an AI skeptic. Ensure you review the response and can sanity check it – AI/LLMs are not actually intelligent and will make shit up.