17 cannot be presented as a word because it’s a number and if it can then that word is “seveen”.
That was an odd one. While I could see the confusion with thinking it couldn’t spell a number, because it’s a number, not a word, I couldn’t figure out the (il)logical leap it was taking to come up with seveen, and even when the person asked what s-e-v-e-n-t-e-e-n spelled, it said 17, which it considered two words, and of course, 17 is a number with no letters, therefore 17 is spelled seveen.
I don’t know enough about these AI bots* to know if it was the ‘how many e’s are in 17’ that was throwing it. To make it worse, IIRC, that AI bot was meant to be a teacher, to educate.
*The most I’ve really ‘interacted’ with AI is getting gemini to help me with some coding (for a hobby, nothing anyone but me will really ever see). But to go back to something I said earlier, this is where I see a problem. Since I asked, and received an answer from gemini, I didn’t have to ask here or on reddit or stackexchange or an arduino forum or wherever. That means the next person with a similar question will never find a thread from me about it, they’ll never see the answers I got, they’ll never see the feedback I gave, and any discussion that became of it while tweaking my code to get it to work the way I want it. They’ll be starting from scratch. As these online discussions happen less and less, AI will have less new information to work with. At some point, I fear, it’ll be pulling coding (or any field) advice from a combination of outdated information (though written by humans) and it’s own code that people copied to to these forums (ie someone asking a question and the replies being ‘I asked AI and it said…’).
I also like to think the AI bubble is bound to pop at some point and this kind of thing will slow down, at least a little.
This is the part I think a lot of people miss. The expansion of AI has now made use of the entire body of digitized information. There’s nothing left to feed that expansion at the same rate.
The next phase of AI expansion will have to be on self correction; that is, the same process humans use to advance knowledge.
I think others have covered my issues, but to summarize:
- I’m very concerned about who owns this technology. Posters here on the SDMB have talked about relative advances in local/open source models, and that makes me feel a little better. But for the frontier models, the ones that really have the potential to destroy millions of jobs, the owners are all private capital who have no incentive to do things in the public interest.
- The technology is enshittifiying things already, but we’ve only seen the beginning. I don’t understand how we’ve all lived through the enshittification of breakthrough disruptive tech like social media, streaming, Uber/AirBnB, online dating, etc, and are still willing to give “big tech” the time of day. They capture our attention and then they try to extract as much out of us as they can while making the products that we fell in love with shittier and shittier. I think there’s a saying in Tennessee about this…
- Nobody is moving fast enough to put up guardrails. AI is rampant in schools, and rather than dropping the hammer while we figure it out, we’re having adult discussions about it. We’re in the honeymoon phase and it sucks already. Students don’t need to “learn how to use AI” because it’s the easiest thing to use – that’s the whole point of it, right? You have a normal human conversation with it and then it does stuff for you. That’s not a transferable skill I’m worried about teaching my students. What it IS is offloading a bunch of opportunities for learning other transferable skills to a machine owned by a billionaire.
I see parallels to this in the architecture field, and general office work too. Computers, CAD drafting software, and general office suite products have mostly eliminated the draftsman, typing pool, and most secretarial jobs. The actual architect, engineer, office executive, manager, etc. has gotten more productive and makes more money, but they don’t make as much more money as the tools have allowed them to, or enough to compensate the loss of those “semi-skilled” jobs. The fruits of those productivity gains have drifted to the owners. Sole proprietorships are something of an exception there.
This is an important point that I think is often missed when discussing the Industrial Revolution, specifically the Second Industrial Revolution starting around 1870. It’s often framed as “growing industrial production drew formerly agricultural workers to the cities looking for better opportunities and wages.” But this is the time of rapid farm mechanization, which put what was formerly the majority of the population out of work. They had no choice but to move to the industrializing cities, because their livelihood in the country evaporated. There were so many of them that there were more workers available than jobs, so wages were extremely low and they could only afford to live in cramped tenements, basically refugees.
We learned the price of pollution and lax regulations from the aforementioned Second Industrial Revolution. At the time belching smokestacks were a symbol of progress and prosperity (some Troglodytes still think that to this day), but you know, it was the best of times, it was the worst of times and all that. Now having been through that, we’re not so eager to just hand the robber barons everything they want, but they’re using their control of the media and unrestrained campaign spending to try to ram through everything they possibly can.
There’s a myth that ecosystems tend to produce and antidotes to the poisons it creates, but I do think it applies to AI. It is something of an arms race, a fight fire with fire situation. For all the AI’s that are being used to find and exploit software vulnerabilities, there’s AI’s being used to find and patch those software vulnerabilities. This gives me SOME hope for the future.
The pushback against AI also gives me some hope. Japan and Korea have significant demographic problems that make robots almost a necessity. However, I was at a restaurant the other day that had a robot bring out drinks after you ordered them from a screen, and while I suppose that might be preferable for those with social anxiety (but then you have to ask why you’re going to a sit-down restaurant in the first place) I found it kind of off-putting. AI-generated menus are enough to get people to not patronize a restaurant. Of course, eventually they’ll get good enough that you won’t be able to tell, but I think social norms will change and we’ll see some greater value put on authenticity.
The loss of entry and junior-level job positions and training/mentoring opportunities is disturbing, and I hope it doesn’t become as dire as it looks like it could be. Keep in mind though that even before AI the olds were always complaining about “kids these days.” They don’t think, they can’t problem-solve, they rely on xyz technology to do the work for them, etc. On the other hand, the C-suite has never been afraid to hire the younger cheaper less experienced kids to replace the older more expensive and experienced employees when they can, so maybe the vibe coding “prompt engineer” kids will end up being ok, I don’t know.
In general, I think the single biggest difference between the current AI boom and previous technologies like the internet, cars, computers, railroads, television, cameras, or the printing press, is that you didn’t have to force most people to use them. They wanted to use them and experiment with them, they saw the potential benefits and wanted in on it. The first uses of AI have all been shoved down our throats, whether as consumers or as employees in things like search engine “summaries”, customer service assistants and chatbots that were already systems everyone hated, software “companions”, and corporate mandates to find ways to integrate AI into your workflow. They do this because we won’t pay for AI as a standalone product, but if they bundle them into existing products and services they can charge more for them and mask some of the additional costs. But then customers get mad because the price of everything is going up to cover something they don’t even want. “But look at the usage numbers!” say the executives. It’s a fictional statistic of a fictional product to puff up fictional stock prices, and we’re all paying for it whether we want it or not. It just didn’t happen like that before.
There’s very real concerns about AI slop poisoning training data in the future. It may make AI self-limiting, though I suspect that will happen only after so much damage is done that the internet as a whole is no longer a reliable source of information for anything of consequence. I hope not. Still, an interesting point as it relates to vibe coding is that it’s only successful if the code actually works. That puts a floor on the amount of junk code that’s out there. Now, that floor may be too low, because the bar is only that it must function, not that it’s efficient or user-friendly.
Vibe-coded Photoshop and other Adobe app replacements are making the rounds right now, and the general consensus is that they look ok, but performance and actual features and functionality are still pretty bad. That’ll no doubt improve, but we’ve been complaining for decades now that human coders are producing apps that are bloated and inefficient and take up all the available resources of new and better computers. We’ve mostly been able to stay ahead of that curve, but so many resources are being diverted away from consumer hardware to AI datacenter hardware, fewer people are able to afford the better computers we’ll need to run the bloated vibe-coded software being generated in those AI data centers. Or who knows, maybe if we just throw more AI resources at the code it’ll be able to make it as clean, efficient, simple, and well-documented as possible, it’s just a matter of time and priorities.
My spouse is a consultant in a (non-technical) professional field, and recently had a meeting with another professional who wanted her input on a piece of writing he had created as a resource.
It was clearly the product of AI. He freely shared that yes, he had fed a tool his notes and such, and it produced this book/booklet for him based on that. Some of her observations:
- The book had some language that paralleled language she has coined and published in the field. She is un-cited/un-credited for what seems likely to be a case of AI lifting her work and “re-creating” it for this other person, in something “he wrote”. (This thread isn’t diving into the intellectual property of it all, but surely ‘AI as implemented steals IP’ is a valid if debatable objection that is not ‘luddite-ism’).
- He is interested in ways he can take this piece and turn it into smaller publishable pieces, but doesn’t know where to begin. It is likely that had he already gone through the work (and time) of refining his ideas and shaping them into this booklet, he would also have a grasp of the sections, the smaller units, how they fit together, and how they might stand on their own. Instead he has pages of writing that he did not write.
This is someone who has good ideas, is educated, and has used AI to ‘skip to the end’ of a process, and thus has missed out on the learning that writing this book would have given him. The process of writing it would have provided him with significant professional growth and development. Instead, he is essentially unchanged by having ‘written’ a book.
What quality will our thought leaders and experts be, when they spent a whole career evaluating AI answers to their needs and simply deciding whether it is “good enough” or “what I meant”? When they never had to exert effort in learning how to do all the small things that go into being able to do the big things? When they never learn to think about how to present an idea or construct sentences in their own voice because they accept what AI gives them as if it was their own?
That’s the danger. That this pushes society across a line (toward which we have been creeping for some time) beyond which we lose the incentive to do things slowly, to spend time learning, to develop skills that are difficult to master, and thus never learn to be the sort of person who has the capability to master difficult things.
“But this is true of every invention ever, you Luddite!!!”
… only if context and matters of degree are irrelevant. “If you’re ok with the printing press you should be ok with the internet” is an analogy so reductive as to be useless.
To address the OP directly:
I’m curious what your list of pros and cons are.
I hear about this happening with music as well. IIRC, there’s ongoing cases trying to decide if it’s considered plagiarism when it’s an AI bot doing the plagiarism. I recall hearing one situation where an AI youtube channel lifted an entire song, no changes, just took the entire song, posted it to it’s own channel then got a copyright strike against the original artist.
I think this was all from a Mic The Snare video, but I’d have to check.
Edit, this is the Mic The Snare video I was referring to. This is going to cause people to stop making music.
For coding specifically, the lack of new training data isn’t such a big deal, because computer programs are self-testable. You (or the AI) can run it, test it, look at the outputs, and measure success on various metrics (does it work, is it usable by a new user, is it fast enough, does it crash, is it secure enough) — all relatively easily. Code can essentially be its own proof.
Most programming languages have a pretty finite syntax and grammar, which an AI can trivially learn in just a few minutes. From there, it’s composing software out of the grammar that’s difficult, like writing a novel out of a finite list of English words. But unlike evaluating human languages, evaluating software written in code is quite a bit simpler, and can be made relatively deterministic and partially autonomous. This self-testability arguably makes code the best-case scenario for AI… the same machine it’s running on can also be the evaluator.
Math is close, I think, as long as you can formalize the proof. It’s much harder for AI to self-verify scientific results (that might require a lab or a lot of time to properly test), politics, marketing, etc. — all the soft and fuzzy real-world stuff that doesn’t give you immediate test results.
For the time being, vibe coding is still a skillset, I think, with measurable differences in output quality depending on how you vibe code. (Without getting into the weeds here, the harness X model X effort X domain X prompt X specification X agentic skills X test suite X verification methodology all matter, sometimes by a lot).
Someone naively typing “Make me a _____” into the free Gemini is going to see worse results than an experienced vibe coder doing the same with a better setup.
But this itself is changing, too, as the models themselves learn to handle long-horizon tasks (complex things that take hours, days, or months to build) better. They’re also getting better and better at subcontracting out their own work to other AI agents, whether copies of themselves or cheaper ones, and prompting each one according to their needs. Claude knows how to prompt Gemini Nano Banana (the image model) better than I do, for example, and I’ll often ask Claude to generate an image for me (something it cannot natively do) by connecting to Gemini and asking it and iterating the results until it likes what it sees. Then it codifies its findings into a self-updating skill that it keeps improving over time. That’s AI learning how to control other AI, and improving that ability on its own every time I use it.
Most of Claude is written by Claude, for example: When AI builds itself \ Anthropic
I fear the opposite: What if “how to use AI” becomes the only worthwhile skill left for students? Today, coding by hand is already seen as quaint, like felling your own lumber with a hand saw to build a log cabin. That change took a little over two years.
I don’t think there is any academic skill a high school teaches today that an AI can’t already do better. A few years from now, that frontier might be at the PhD level — in some fields, it’s already well past that.
What is society even supposed to teach students? I think the lesson they’re mostly learning today is “I’m being forced to learn stupid things that an AI can do better anyway, so my real homework is figuring out how to bypass the AI detectors”.
If we forced them to put away all the devices and learn everything the hard way, by rote memorization and pencil and paper… what are they supposed to do once they graduate? Yesterday’s skilled professionals may be today’s AI shepherds. But nobody has a viable plan for what to do when we become the sheep tomorrow…
Maybe “How to destroy a data center 101” is the new pre-req, and Ted Kaczynski becomes required reading…
It’s a similar situation on the consumer side, too (as the reader). For work I used to have to spend a lot of time (usually days or weeks) learning new material, digesting it, testing my own understanding, etc. But now when something new comes out, it’s just my Claude reading their Claude’s writing, and understanding all of it in a few minutes. It can read and understand an entire book faster than I can finish a page.
It’s happening all across the fields too… in technical documentation, certainly, but also in science, academia, YouTube scripts, etc. AI out, AI in, human optional.
I guess in that sense it’s not that different than how computers have always worked — information exchangers and processors — except now they’ve learned to prefer English as their protocol.
But what it does mean for the humans is that there’s now novel information being created in the world that no human has ever seen, created by robots for other robots to improve yet other robots. We no longer have a monopoly on information processing. The question is whether we should read any of it (for now), and whether we will be able to read it (in the future).
We are the last generation who remembers the “before” times, when humans were still the most capable reasoning machine on this planet. I don’t know what it’ll mean for younger ones to be born into a world where, by default, they’ll never be as educated or as intelligent as the chatbot on their phones. I wonder if this is what the twilight of the Neanderthals felt like.
Re the OP’s comparison between data centers and airports:
There are about 500 airports in the U.S. are open to passenger traffic, and close to 30 major hubs that handle the great majority of it. There are already thousands of data centers in the U.S. (anywhere from ~1400 to over 5000 depending on criteria and who’s counting) and thousands more planned. That’s an awful lot of extra pressure on the electricity grid and water supplies, which are already strained. Being concerned about living in places with data centers existing or planned can’t be handwaved away as “Ludditism”.
The idea that AI is sweeping medicine is not exactly reassuring, unless there are humans every step of the way overseeing it. Whether it’s radiologic exams, biopsies or internal medicine diagnoses, it would be foolish to depend on AI to do it without strict human oversight.*
*there are a lot of people apparently happy to let AI do it; probably the same people who love the idea of being transported in driverless cars, or being passengers in pilotless planes operating in a system with AI-directed air traffic control and no people in the towers. They might be cool with nuclear power plants being AI-operated. No one could possibly hack into any of these systems to create havoc or collect blackmail, right? Mmm, no thanks.
How would you be able to tell? I don’t just mean that in the Turing Test sense, but if AI got good enough at those things and had a error rate similar to or better than people… would you even know it was being run by AI?
Many fields are probably going to move to a hybrid if not AI-first workflow. You’d need a chain of custody proof system to prove something was not made with AI, almost like USDA Organic label but for “100% certified human”.
I’d be willing to pay for that sort of certification for some things (like art and novels), not for other things (computer programs).
I am deadly serious when I say that the OP has not read nearly enough science fiction stories.
Nor enough of the news in the last decade. Nor much history.
Every major innovation has been used by the elite to crush the non-elite for a few generations until they slowly rise up and fight for a larger share of the larger pie. This time will be no different.
Except this time AI may steal the pie from everyone including the hapless elite. Or simply cement everyone and everything into a North Korea style eternal prison planet.
That’s part of the problem on both sides though.
On one hand, science fiction accustoms one to thinking outside the box, foreseeing consequences, and what-not. On the other, it’s entertainment intended to be lurid, highlight worst-case scenarios, and so forth.
It’s like if the OP had asked a legal question, and you’re saying they haven’t watched enough cop shows or legal dramas. Yes, you get a feel for how the system works with the shows, but they’re also 100x more dramatic and exciting than the real fields actually are. Same thing with science fiction.
My suspicion is that AI is highly disruptive, but it’s not so likely to be apocalyptic. We (the SDMB) tends to have a lot of rather absurdist wailing and waving of hands when something is frightening (see Trump), even when there isn’t actual evidence that what we’re anxious about will actually come to pass. I feel like this may be a similar situation.
“Isn’t all of the objection to the Torment Nexus just straight-up Luddite-ism?”
Hell, the actual “straight-up” Luddites shouldn’t even be handwaved away as espousing mere “Ludditism.” They actually had some pretty sensible grievances. They were ridiculed because, then as now, the people who owned the newspapers tended to also own the capital and to be in bed with the government entities that made and enforced the law.
Despite their modern reputation, the original Luddites were neither opposed to technology nor inept at using it. Many were highly skilled machine operators in the textile industry.
The Luddite disturbances started in circumstances at least superficially similar to our own. British working families at the start of the 19th century were enduring economic upheaval and widespread unemployment. A seemingly endless war … Food was scarce and rapidly becoming more costly. Then … British troops broke up a crowd of protesters demanding more work and better wages.
But the Luddites were neither as organized nor as dangerous as authorities believed. They set some factories on fire, but mainly they confined themselves to breaking machines. In truth, they inflicted less violence than they encountered.
As the Industrial Revolution began, workers naturally worried about being displaced by increasingly efficient machines. But the Luddites themselves “were totally fine with machines,” says Kevin Binfield, editor of the 2004 collection Writings of the Luddites. They confined their attacks to manufacturers who used machines in what they called “a fraudulent and deceitful manner” to get around standard labor practices. “They just wanted machines that made high-quality goods,” says Binfield, “and they wanted these machines to be run by workers who had gone through an apprenticeship and got paid decent wages. Those were their only concerns.”
^And that article itself is fifteen years old. Not even in the context of so-called AI, and yet as relevant now as ever.
I’m worried about intellectual property in the future…though I guess IP has been broken for a long time.
Currently there are a couple commercial ideas I am working on, and I’m using AI to assist me getting these up and running. Of course a massive concern is that the AI companies will use my information; either deliberately scraping chats for opportunities or (more plausibly) for training data.
But not using AI would mean these ideas would take massively longer to get up and running. It’s barely an option now, let alone in the near future where resisting AI would be akin to being the one business that refuses to use computers, phones or cars.
It makes me wonder if, in the future, I won’t really own my own ideas. I’ll either need to pay to keep them secret, or even cut the AI in on anything I produce.
I think there might be an interesting branching of specializing and non-specializing in academic fields.
OTOH there will be roles where you’re sufficiently expert at something that you’re adding to human (and AI) knowledge in that field. This will probably need to be more specialized than today, and AI might help direct people into those roles.
OTOH others will benefit from gaining a broad knowledge in different fields and helping to create solutions that bridge across them (without needing to know the details).
Of course all this is describing a world like ours, with crazy smart AI but not AGI. With AGI all bets are off.
In the US, at least, you don’t own your ideas now. And if you rely to a sufficient extent on AI to create the fixed, tangible expression of those ideas, you may not even own the elements of that expression.
I worry that will not always be the case, that new law and international treaties will use the current UK model, where non-human creations can be given copyright protections. I believe, though, that there is some traction in the UK for changing that to only include human-made creations. (And, not being from the UK, my understanding of their law may be entirely off.)
What I meant was something like “security through obscurity”. That is to say: IP law is f-ked, and is mostly about letting big corporations bully upstarts. But right now you can come up with an idea, and at least have a head start on anyone wanting to copy you. And some forms of IP are automatic, like copyright.
In the (near) future, AI will be required to make something even mildly competitive in a market where everyone else uses AI. So: you have to share your idea to get started and you’re also starting on uncertain ground in terms of things like copyright.
Yep but it is complex though – I wouldn’t agree with saying that anything that comes out of AI is public either. I’d be fked, for a start.
Prompts can include useful and novel ideas. And of course there’s a big difference between an image produced from a single prompt and a product made from a year of iterative programming with significant human direction and labor.
Not sure I really agree with that forecast. There’s a huge pushback against gen-AI content right now. Hopefully, that only grows in the future.
Agreed. At that point, the AI is just another tool used by a craftsperson.
But I suppose that brings up another question, whether talent and training should in some way be protected. I mean, is it fair/right/good that I can create photorealistic artwork over a year’s time despite me barely being able to draw stick figures? But that’s probably a question for another thread.