It has become impossible to tell managers mesmerised by artificial intelligence that the tools are not, in fact, helpful. So employees just play along with the fiction to keep their jobs, writes our tech columnist
I 100% agree. I have no idea why my comment gets downvoted so much. I just wanted to point out that AI and even LLMs absolutely do have valuable use cases. I use them a lot and I see people around me making their lives better using them. Are they overhyped? Probably. Are AI companies using public information that should belong to everyone in an unfair way to become rich and powerful? Absofuckinglutely. Is that how it should be? Definitely not. Is AI a technology with massive potential to change the world for the better? Also yes. Does AI have the potential to completely wreck our society and economy if we do the whole thing wrong? Probably also yes. I hope we can find a way to turn this into something positive.
I have no idea why my comment gets downvoted so much.
Because your comment seemed to add an anecdotal āit also helped two peopleā to a long list of evils as if it were equivalent. āSure, people are suffering because of this, but two friends of mine are better off for itā ā is that really worth it?
I use them a lot and I see people around me making their lives better using them.
Aside from the nebulous question of just how itās supposedly making lives better, and whether that actually is an improvement, the question repeats: Is a fee peopleās lives getting better really equivalent to the people losing their homes, water supply or job over it? Itās not that weāre unaware that it might have benefits, itās that those benefits are massively dwarfed by the evils.
Are they overhyped? Probably.
LLMs in particular, and dangerously so. People treat them as virtual humans, use terms such as singularity and trust them with cognitive work a linguistic model plainly isnāt capable of actually doing. They deliver a convincing imitation, but there have been too many instances of that imitation not holding up to any reasonable standard. For a more benign example, Ford had to hire back its fired engineers because it turned out that GenAI canāt actually do their job. Less benign are things like giving harmful āhealth adviceā or reinforcing their usersā delusions and psychoses to the point where they commit violence against themselves or others.
Theyāre still being hailed as the future, despite evidence backing up what theorists have been warning for a while: A text generator cannot generate meaning and should not be used for any purpose where the content actually matters. That your friendsā word salad happened to resemble useful advice is nice, but it wasnāt actually advice, just the product of a high-tech advice imitator that got lucky.
Is AI a technology with massive potential to change the world for the better? Also yes.
This is where we need to specify just what we mean by AI.
Language models are useful for linguistic correlation tasks, but the actual utility of that is hard to gauge, because it heavily depends on the mode of the output and on the recipient. Specialised models may make decent assistants for certain types of specialists.
They certainly arenāt capable of the type of ānobody has to work any moreā utopia that conmen like Musk and Altmann are selling to the gullible. Theyāre fundamentally unreliable and their consumption of resources dwarfs their sloppy utility.
So while some form of AI might hold a key to a radically better world, LLMs and generative AI arenāt it.
Does AI have the potential to completely wreck our society and economy if we do the whole thing wrong? Probably also yes.
Probably? Have you taken a look around or read the whole list you replied to? The hype already is destroying lives, and thereās a slew of experts calling out the fraudulent scheme theyāre pulling on the stock market.
The one point Iāll concede to the scammers is that it has fundamentally altered our world.
Just not for the better.
I hope we can find a way to turn this into something positive.
It starts with being honest and clear about what āthisā actually is, understanding what it isnāt and not making excuses for a deeply destructive technological complex just because the mathematical parrot emits some useful responses now and then.
First of all, thank you for your extensive response. I think discussions like this are important. Hereās my 2 cents, sorry if Iām not that eloquent - Iām not a native English speaker so some of this is a bit hard to put into words for me.
Because your comment seemed to add an anecdotal āit also helped two peopleā to a long list of evils as if it were equivalent.
Well, surely my two friends canāt be the only people AI has helped, that would be highly unlikely. So I do think it is a valid point along the other ones - AI does create value. Otherwise there would be no hype. I just think itās dishonest to claim that AI is completely useless. If we want to turn this into something positive, we have to be honest about all the aspects.
Aside from the nebulous question of just how itās supposedly making lives better, and whether that actually is an improvement, the question repeats: Is a fee peopleās lives getting better really equivalent to the people losing their homes, water supply or job over it? Itās not that weāre unaware that it might have benefits, itās that those benefits are massively dwarfed by the evils.
Of course itās not. Weāre totally on the same page here. I just think itās not a matter of the technology itself, itās how those companies (are allowed to) act. That doesnāt seem like an AI problem to me, itās a capitalism problem. We might have different perspectives on this btw because I live in the EU. People donāt lose their home or water supply here because of AI companies.
People treat them as virtual humans, use terms such as singularity and trust them with cognitive work a linguistic model plainly isnāt capable of actually doing.
Yes. And I do think AI companies are at fault here for marketing their products dishonestly to people who donāt understand how the technology works. They exploit stupid people, and mostly stupid people in positions with too much power.
Theyāre still being hailed as the future, despite evidence backing up what theorists have been warning for a while: A text generator cannot generate meaning and should not be used for any purpose where the content actually matters. That your friendsā word salad happened to resemble useful advice is nice, but it wasnāt actually advice, just the product of a high-tech advice imitator that got lucky.
I think we might fundamentally disagree on this. Advice from a human is nothing but a multivariate combination of things that person has learned in the past. Itās way more complex - brains are orders of magnitude more complex than an artificial neural network - but I donāt see a fundamentally and qualitative difference here unless you believe in something like a soul that somehow magically gives more value to a human advice. If you understand that an LLM is a mathematical tool and not a human, you can use it in very efficient and helpful ways. Some questions actually better answered by something that has gathered a wide range of information from thousands or even millions of sources. Some questions that require fast answers are also better answered by AI (just yesterday I asked ChatGPT for a python script for a certain task - it took about 20 seconds and provided a perfectly working ~500 lines python script that I could use; no human could ever do that). Lots of other questions - questions that require emotional connection, personal experience, specialized knowledge - should be left to humans, of course. The amount of shitty and wrong advice Iāve gotten from people, even doctors and other experts, is immense btw. The crucial thing is that you need to learn how to test and evaluate advice that you get - from people AND from LLMs. Many people would profit from learning that skill - maybe using LLMs might even teach them that, even if it will probably usually be the hard way.
Language models are useful for linguistic correlation tasks, but the actual utility of that is hard to gauge, because it heavily depends on the mode of the output and on the recipient.
Yes, thatās exactly my point. We need to educate the recipients. That is actually one of the most important things imo. AI and LLMs arenāt going to go away, theyāre here to stay. Just look at how it already penetrated all kinds of scientific disciplines. We urgently need to regulate what companies can do with it and we need to educate people on how they work and how to use them, and especially how NOT to use them. And by the way, I wish we had never made the jump from calling it deep learning to calling it AI - I think that might have prevented a lot of problems und misunderstandings.
Look, I think ultimately our views arenāt even that different. Thatās why discourse is important. I just donāt think screaming āAI is badā is going to get us anywhere (and thatās essentially what the original comment that I replied to was doing imo). We need to be specific about what the problem is: the problem is not AI, the problem is that some people are getting super rich on the expense of almost everyone else. THAT is what we need to stop. You canāt fight technological and scientific progress, what you need to fight is malicious use of technology and the exploitation of the masses for the profit of a few. Iād say we might actually in an āindustrial revolutionā kind of situation. Back then, workers rights and communism became a thing. We might need something as fundamental as that to deal with todayās problems. Weāll need all the social sciences now that weāve cut funding for over the last decades because MINT was just more important to us (I say that as a MINT scientist btw).
I find that writing out and defending my position helps me refine it. Sometimes, my perspective on things shifts just by trying to explain it. Discussions are the whetstone by which arguments are sharpened.
Iām not a native English speaker so some of this is a bit hard to put into words for me.
That makes two of us :D
Sometimes, these discussions help expand my Engish skills too.
So I do think it is a valid point along the other ones - AI does create value.
Yesnāt. Itās a text generator that predicts a likely series of words, based on the language patterns it learned from its training material. It doesnāt so much create value as aggregate the value of other peopleās work into a weighted reproduction.
However, those weights are biased by quantity, not quality. It has no way to assess which responses are good, only which ones are likely. The average value it (re-)produces is an average of the value of the training material. Models trained on a wide variety of content (such as ChatGPT) will inevitably include a lot of material of little value to specialised topics.
Hence my argument: It can get lucky and predict a high-value response, but the problem is that it isnāt guaranteed to do so. A layperson doesnāt have the expertise to tell the difference. That not only dilutes the value, it invites false confidence in the results. In cases where accuracy is critical, this may actually produce negative value if people consult an unreliable model rather than a specialist.
And thatās the critical difference to human advice: human experience is shaped by a number of factors beyond just language. We create semantic connections to abstract concepts and attach specific meaning to certain words and patterns.
LLMs donāt have that abstraction. They can predict sequences of text that sound plausible and might coincide with something describing reality, but they canāt tell whether itās accurate.
Advice has to be grounded in reality to be useful, and thatās what LLMs are missing.
Theyāre perfectly suitable for tasks where correlation is enough, but if the task requires actual understanding, LLMs arenāt equipped for it.
Otherwise there would be no hype.
Hype doesnāt always need a solid reason. In this case, a lot of hype stems from the hope that we may one day have the type of Artificial General Intelligence that SciFi has long dreamed of, the illusion that they may be able to do our work for us and the promise to company managers that they may be able to save money by replacing human employees with AI.
The various executives of the big AI vendors are obviously capitalising on that, stoking the hype with grand and utopic visions because they want to sell their product.
Itās not that itās useless. Itās that the utility is far less than the lofty promises made by people milking the hype for all itās worth. And that little utility comes at a terrible social, economic and ecologic price.
We might have different perspectives on this btw because I live in the EU. People donāt lose their home or water supply here because of AI companies.
Iām in the EU too, I just read a lot of US news. I donāt think we should dismiss the consequences our use of US infrastructure has. Worse yet, I donāt think we should dismiss the political dependencies that creates.
We need to be specific about what the problem is: the problem is not AI, the problem is that some people are getting super rich on the expense of almost everyone else.
I think thatās only a part of the problem. The second part is the lack of understanding you mentioned, and the resulting mis- and overuse. Iāve seen people trying to argue with experts because āChatGPT saidā because they genuinely do not understand that ChatGPT is a parrot, not an expert.
And the third part, again, is the disastrous effect on our world.
You canāt fight technological and scientific progress [ā¦]
Iād say we might actually in an āindustrial revolutionā kind of situation.
Iām not fighting it. Iām trying to pull it out of the pit that the current obsession with imitation has dug. When college students, the next generation or scientists, trades their scientific understanding for the convenience of high-tech parrots, that is the opposite of progress. Itās stagnation, fostered by those few people that are happily trading our future for their present profits.
Thatās the mirage of this āindustrial revolutionā analogy: Theyāve built something in the shape of a steam engine, promised the functionality of Spinning Jenny, sold fabric factory owners on the idea who fired their workers to buy these machines. The people they fired were promised that theyād have to work less, but werenāt told that they would be paid less too.
Now these machines turn out to not actually provide the smooth, spinning motion required for spinning thread.
Factories are facing expensive production outages, compounding the expenses of getting these machines installed and canāt afford to hire all the workers back.
The machine shops built to produce these Sloppy Jennies or their parts will eventually find it harder to sell their iventory. Once they go under, their workers will also become economic casualties.
Weāre at the point where we need to reinforce worker protections. We need to push for a system where our livelihood isnāt contingent on the amount of work we do. Only then can that utopia even manifest. And to actually make progress:
Weāll need all the social sciences now that weāve cut funding for over the last decades
Yes.
Weāll need to understand the social dynamics of such technology to better prevent the disastrous side-effects. Weāll need to compare historical developments with present circumstances and plans to account for future developments. Weāll need to study the psychological effects of interacting with human-like machines to be able to correct course where needed.
Whatever control mechanism is supposed to prevent machines from producing harmful output will need heuristics based on those disciplines to assess the dangers of that output.
Iād include philosophy too. We donāt need neural networks that are a lesser version of human ones, nor just scaled-up variants that also replicate all the human inefficiencies, errors and biases. Those are the pure MINT approach to the problem, and itās clearly coming up short.
We need to find a rigorous mechanism for representing semantic knowledge in digital systems that donāt just imitate, but surpass human cognition. We need a workable philosophical grounding for logical and mathematical approaches to modelling knowledge of concepts rather than just language.
We need to get over the error that LLMs are āthinkingā or ājust like humansā. Theyāre not, but as long as weāre stuck on the idea, we canāt fix it.
Also, we really, really should spend less time thinking about whether we could and more about whether we should.
Itās a text generator that predicts a likely series of words, based on the language patterns it learned from its training material.
Yes and no. While I do absolutely agree about the problem of reliability and false confidence, I do not agree on the general understanding of LLMs. You make it sound like itās merely a predictor of likely words, like your phone keyboard may suggest words you usually use in sequence. But it is really not that simple. I disagree with the - especially here on Lemmy - very widespread āparrotā analogy in the sense that imho people have a very oversimplified idea of what large language models actually are. They donāt just count statistics of word sequences and then spit out the most likely combination. The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern. That way, they can absolutely learn concepts and āskillsā like logical reasoning, to a certain extent. The exact patterns they learned arenāt even fully understood, but obviously it does work. Even early GPT models learned to do arithmetic - there are publications that show that GPT was able to calculate arithmetic problems that were not in the training data, although it did make mistakes. Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions. Still, it had apparently learned the concept of arithmetics, just by seeing examples (and maybe explanations - we donāt really know). Modern LLMs absolutely capable of a certain level of logic. You can give ChatGPT a task itās never seen before and chances are it can solve it. Even if it canāt - that doesnāt make its reasoning qualitatively different from how a human brain works. My argument is that humans learn how to think ālogicallyā by seeing examples too - and they fail at logic all. the. damn. time. Go to a city center and ask people logic puzzles - see how many will be able to solve them correctly (and maybe compare the result to an LLM). I wouldnāt even be surprised if by now, even asking medical questions to doctors would, on average, yield worse results than a state-of-the-art LLM. I did a PhD in computer science a few years ago and I can tell you, I would probably rather trust the latest ChatGPT model on most questions regarding my field than myself. It has just become that incredibly good. Iām absolutely not saying itās perfect, it still makes stupid mistakes and you have to be aware of that, but so do people. I have not seen a single convincing argument why the concept of logic or reasoning that humans have is fundamentally different from that of LLMs. Most people suck at logic. And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
In all seriousness - LLMs lack sensorimotor interaction with the world, and even if we give them robot bodies, they will never know what itās like to be human. In that sense, they will not replace us, and they will never be human. I would even agree that they will probably never be conscious (although of course we canāt be sure, especially since we havenāt understood consciousness yet in the first place). Their biases and their āway of thinkingā will also most probably always be different from humans. But I do not agree that they cannot be āintelligentā in a logic/reasoning sense and that they are merely āstatistical parrotsā.
The various executives of the big AI vendors are obviously capitalising on that, stoking the hype with grand and utopic visions because they want to sell their product.
Itās not that itās useless. Itās that the utility is far less than the lofty promises made by people milking the hype for all itās worth. And that little utility comes at a terrible social, economic and ecologic price.
Agreed. Iām fascinated by how far AI has become, but it just isnāt where marketers claim it is, and we canāt be sure it will get there anytime soon.
I donāt think we should dismiss the consequences our use of US infrastructure has. Worse yet, I donāt think we should dismiss the political dependencies that creates.
True. Political dependencies are a serious problem and Iām pretty sure they play a big role in why the US actually lets AI companies get away with all the shit that they do.
Iāve seen people trying to argue with experts because āChatGPT saidā because they genuinely do not understand that ChatGPT is a parrot, not an expert.
Yeah. The point is: ChatGPT will not make you an expert. ChatGPT may act as an expert and even provide accurate information, but citing its answers will not magically make you an expert yourself. That is something that needs to be hammered into peopleās heads.
Iām not fighting it. Iām trying to pull it out of the pit that the current obsession with imitation has dug. When college students, the next generation or scientists, trades their scientific understanding for the convenience of high-tech parrots, that is the opposite of progress. Itās stagnation, fostered by those few people that are happily trading our future for their present profits.
Fair enough. We need to find ways to use AI as a useful tool, not a tool to make people lazy and stupid. Tbh, I think it will happen. It happened with Google as well. People used to use Google wrong all the time when it was still new. They clicked the first search result and believed absolutely everything it said. Iām kinda old, I remember that time. :P
Iām pretty sure it will be similar with LLMs. By using them, we will learn their limitations and shortcomings, and many of us will learn the hard way. The marketing strategies of AI companies are NOT helpful with that.
Weāll need to understand the social dynamics of such technology to better prevent the disastrous side-effects. Weāll need to compare historical developments with present circumstances and plans to account for future developments. Weāll need to study the psychological effects of interacting with human-like machines to be able to correct course where needed.
I absolutely 100% couldnāt agree more. AI has already had a lot and probably will have even more impact on us, on other technologies like medicine, biotechnology, probably chemistry, material sciences, and loads of other key technologies. We will have a LOT of trouble keeping up with that acceleration of technological progress as a society. In a world where even the existence of the internet doesnāt seem to have been digested fully, AI has the potential to completely wreck everything. And certain people will do all they can to exploit that to gain power and money. And we will have to be faster than them.
In that sense, I agree that
Also, we really, really should spend less time thinking about whether we could and more about whether we should.
is probably right. The EUās AI Act is a good first step, but the world is in this together, and with the current geopolitical situation I honestly donāt see us working together here for the good of humankind. So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system. Iām not sure itās possible though.
The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern.
Quantity of training material doesnāt confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesnāt give something the text doesnāt have.
This fallacy is why I say philosophy should be more widely taught: the relationship between symbols and semantics isnāt quite so trivial. In the specific context of computational conscience, the Chinese Room is a well-discussed argument that demonstrates how command of language doesnāt necessarily require or produce understanding of the same. We can argue about the implications for human consciousness (Iād rather not), but the critical part is that processing a foreign language doesnāt translate it into mine.
For a more practical example, consider the issue of legal arguments citing made-up or irrelevant precedence cases. It is trivial to check whether a given case reference actually correlates with an actual case. It is critical that your citation both refers to an actual case and correctly reflects the contents. Someone who understands the nature of legal arguments knows why a certain arrangement needs to be an extant item in a finite set of instances of that arrangement (namely, a topically relevant subset of all legal cases in history, which is also a finite set).
Yet LLMs get it wrong. They get the shape right, but the filling is a game of Russian Roulette. When drawing a semantic connection between the current context and a related case, it should be a no-brainer to correctly write down the reference to that related case, but they donāt do that. They draw on the trained set of symbol correlations to produce something likely.
The same goes for essay prompts in exams where a human should recognise that an instruction to include references to Madagascar, in white font on a white background, isnāt actually part of the question but rather a trap to catch blind copy+paste into AI. The AI doesnāt understand that context or that it should disregard that part. It doesnāt actually know why the instruction is there, it just processes it into part of the context.
Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for. Carryovers (overflow flags) are part of the most fundamental ALU design. The fact that it reproduces human error shows that it doesnāt actually understand the assignment, it just imitates the training material. If it understood that the reason a certain pattern is there is because the human writing it made a mistake, it should be able to correct it instead.
Otherwise, it is a parrot, or perhaps a really studious child thatās great at imitating adults, without any care for the actual semantics. A computer making as many or even more mistakes than humans is useless (for that task; we agree that they can do some tasks just fine). If AI should be useful universally, it needs to understand these semantics.
Language is a tool for communicating thoughts and perceptions, but that doesnāt work the other direction. Words do not imply reasoning.
And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
Yeah, I wonder what material LLMs developed by companies with white, male CEOs are dominantly trained onā¦
So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system.
Iād say itās less about deceleration itself, more about diversifying the efforts and exploring alternate avenues to achieve the things theyāre lacking rather than pouring those resources exclusively into LLMs. The deceleration of LLM development doesnāt have to mean a total deceleration of progress.
Quantity of training material doesnāt confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesnāt give something the text doesnāt have.
You left out half of my sentence though. Size of the model does in fact confer new abilities. See, for example, this paper: https://arxiv.org/abs/2206.07682
All of the problems you describe that LLMs have - itās true, and it holds for current LLMs. But it is not necessarily a fundamental limitation. Time will show how much better LLMs will become. I think the time of super fast progress is probably over, but there will still be improvements.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for.
An LLM is not a computer. An LLM runs on a computer. Youāre mixing up two things here.
I think you vastly overestimate human abilities. Itās a phenomenon I come across here on Lemmy all the time. Itās like people believe their brains are capable of some sort of āanalytical logicā as opposed to ānumerical logicā. As if āunderstandingā something was some sort of godly ability. I donāt believe in a supernatural spirit or anything like that. We learn from external influences, we are statistical parrots as well. There is no absolute truth mechanism in our brains. Weāre conscious, yes, but just because understanding something feels so absolutely logical and true to you doesnāt mean itās not just a result of the statistics your brain has learned from.
I 100% agree. I have no idea why my comment gets downvoted so much. I just wanted to point out that AI and even LLMs absolutely do have valuable use cases. I use them a lot and I see people around me making their lives better using them. Are they overhyped? Probably. Are AI companies using public information that should belong to everyone in an unfair way to become rich and powerful? Absofuckinglutely. Is that how it should be? Definitely not. Is AI a technology with massive potential to change the world for the better? Also yes. Does AI have the potential to completely wreck our society and economy if we do the whole thing wrong? Probably also yes. I hope we can find a way to turn this into something positive.
Because your comment seemed to add an anecdotal āit also helped two peopleā to a long list of evils as if it were equivalent. āSure, people are suffering because of this, but two friends of mine are better off for itā ā is that really worth it?
Aside from the nebulous question of just how itās supposedly making lives better, and whether that actually is an improvement, the question repeats: Is a fee peopleās lives getting better really equivalent to the people losing their homes, water supply or job over it? Itās not that weāre unaware that it might have benefits, itās that those benefits are massively dwarfed by the evils.
LLMs in particular, and dangerously so. People treat them as virtual humans, use terms such as singularity and trust them with cognitive work a linguistic model plainly isnāt capable of actually doing. They deliver a convincing imitation, but there have been too many instances of that imitation not holding up to any reasonable standard. For a more benign example, Ford had to hire back its fired engineers because it turned out that GenAI canāt actually do their job. Less benign are things like giving harmful āhealth adviceā or reinforcing their usersā delusions and psychoses to the point where they commit violence against themselves or others.
Theyāre still being hailed as the future, despite evidence backing up what theorists have been warning for a while: A text generator cannot generate meaning and should not be used for any purpose where the content actually matters. That your friendsā word salad happened to resemble useful advice is nice, but it wasnāt actually advice, just the product of a high-tech advice imitator that got lucky.
This is where we need to specify just what we mean by AI.
Language models are useful for linguistic correlation tasks, but the actual utility of that is hard to gauge, because it heavily depends on the mode of the output and on the recipient. Specialised models may make decent assistants for certain types of specialists.
They certainly arenāt capable of the type of ānobody has to work any moreā utopia that conmen like Musk and Altmann are selling to the gullible. Theyāre fundamentally unreliable and their consumption of resources dwarfs their sloppy utility.
So while some form of AI might hold a key to a radically better world, LLMs and generative AI arenāt it.
Probably? Have you taken a look around or read the whole list you replied to? The hype already is destroying lives, and thereās a slew of experts calling out the fraudulent scheme theyāre pulling on the stock market.
The one point Iāll concede to the scammers is that it has fundamentally altered our world.
Just not for the better.
It starts with being honest and clear about what āthisā actually is, understanding what it isnāt and not making excuses for a deeply destructive technological complex just because the mathematical parrot emits some useful responses now and then.
First of all, thank you for your extensive response. I think discussions like this are important. Hereās my 2 cents, sorry if Iām not that eloquent - Iām not a native English speaker so some of this is a bit hard to put into words for me.
Well, surely my two friends canāt be the only people AI has helped, that would be highly unlikely. So I do think it is a valid point along the other ones - AI does create value. Otherwise there would be no hype. I just think itās dishonest to claim that AI is completely useless. If we want to turn this into something positive, we have to be honest about all the aspects.
Of course itās not. Weāre totally on the same page here. I just think itās not a matter of the technology itself, itās how those companies (are allowed to) act. That doesnāt seem like an AI problem to me, itās a capitalism problem. We might have different perspectives on this btw because I live in the EU. People donāt lose their home or water supply here because of AI companies.
Yes. And I do think AI companies are at fault here for marketing their products dishonestly to people who donāt understand how the technology works. They exploit stupid people, and mostly stupid people in positions with too much power.
I think we might fundamentally disagree on this. Advice from a human is nothing but a multivariate combination of things that person has learned in the past. Itās way more complex - brains are orders of magnitude more complex than an artificial neural network - but I donāt see a fundamentally and qualitative difference here unless you believe in something like a soul that somehow magically gives more value to a human advice. If you understand that an LLM is a mathematical tool and not a human, you can use it in very efficient and helpful ways. Some questions actually better answered by something that has gathered a wide range of information from thousands or even millions of sources. Some questions that require fast answers are also better answered by AI (just yesterday I asked ChatGPT for a python script for a certain task - it took about 20 seconds and provided a perfectly working ~500 lines python script that I could use; no human could ever do that). Lots of other questions - questions that require emotional connection, personal experience, specialized knowledge - should be left to humans, of course. The amount of shitty and wrong advice Iāve gotten from people, even doctors and other experts, is immense btw. The crucial thing is that you need to learn how to test and evaluate advice that you get - from people AND from LLMs. Many people would profit from learning that skill - maybe using LLMs might even teach them that, even if it will probably usually be the hard way.
Yes, thatās exactly my point. We need to educate the recipients. That is actually one of the most important things imo. AI and LLMs arenāt going to go away, theyāre here to stay. Just look at how it already penetrated all kinds of scientific disciplines. We urgently need to regulate what companies can do with it and we need to educate people on how they work and how to use them, and especially how NOT to use them. And by the way, I wish we had never made the jump from calling it deep learning to calling it AI - I think that might have prevented a lot of problems und misunderstandings.
Look, I think ultimately our views arenāt even that different. Thatās why discourse is important. I just donāt think screaming āAI is badā is going to get us anywhere (and thatās essentially what the original comment that I replied to was doing imo). We need to be specific about what the problem is: the problem is not AI, the problem is that some people are getting super rich on the expense of almost everyone else. THAT is what we need to stop. You canāt fight technological and scientific progress, what you need to fight is malicious use of technology and the exploitation of the masses for the profit of a few. Iād say we might actually in an āindustrial revolutionā kind of situation. Back then, workers rights and communism became a thing. We might need something as fundamental as that to deal with todayās problems. Weāll need all the social sciences now that weāve cut funding for over the last decades because MINT was just more important to us (I say that as a MINT scientist btw).
I find that writing out and defending my position helps me refine it. Sometimes, my perspective on things shifts just by trying to explain it. Discussions are the whetstone by which arguments are sharpened.
That makes two of us :D
Sometimes, these discussions help expand my Engish skills too.
Yesnāt. Itās a text generator that predicts a likely series of words, based on the language patterns it learned from its training material. It doesnāt so much create value as aggregate the value of other peopleās work into a weighted reproduction.
However, those weights are biased by quantity, not quality. It has no way to assess which responses are good, only which ones are likely. The average value it (re-)produces is an average of the value of the training material. Models trained on a wide variety of content (such as ChatGPT) will inevitably include a lot of material of little value to specialised topics.
Hence my argument: It can get lucky and predict a high-value response, but the problem is that it isnāt guaranteed to do so. A layperson doesnāt have the expertise to tell the difference. That not only dilutes the value, it invites false confidence in the results. In cases where accuracy is critical, this may actually produce negative value if people consult an unreliable model rather than a specialist.
And thatās the critical difference to human advice: human experience is shaped by a number of factors beyond just language. We create semantic connections to abstract concepts and attach specific meaning to certain words and patterns.
LLMs donāt have that abstraction. They can predict sequences of text that sound plausible and might coincide with something describing reality, but they canāt tell whether itās accurate.
Advice has to be grounded in reality to be useful, and thatās what LLMs are missing.
Theyāre perfectly suitable for tasks where correlation is enough, but if the task requires actual understanding, LLMs arenāt equipped for it.
Hype doesnāt always need a solid reason. In this case, a lot of hype stems from the hope that we may one day have the type of Artificial General Intelligence that SciFi has long dreamed of, the illusion that they may be able to do our work for us and the promise to company managers that they may be able to save money by replacing human employees with AI.
The various executives of the big AI vendors are obviously capitalising on that, stoking the hype with grand and utopic visions because they want to sell their product.
Itās not that itās useless. Itās that the utility is far less than the lofty promises made by people milking the hype for all itās worth. And that little utility comes at a terrible social, economic and ecologic price.
Iām in the EU too, I just read a lot of US news. I donāt think we should dismiss the consequences our use of US infrastructure has. Worse yet, I donāt think we should dismiss the political dependencies that creates.
I think thatās only a part of the problem. The second part is the lack of understanding you mentioned, and the resulting mis- and overuse. Iāve seen people trying to argue with experts because āChatGPT saidā because they genuinely do not understand that ChatGPT is a parrot, not an expert.
And the third part, again, is the disastrous effect on our world.
Iām not fighting it. Iām trying to pull it out of the pit that the current obsession with imitation has dug. When college students, the next generation or scientists, trades their scientific understanding for the convenience of high-tech parrots, that is the opposite of progress. Itās stagnation, fostered by those few people that are happily trading our future for their present profits.
Thatās the mirage of this āindustrial revolutionā analogy: Theyāve built something in the shape of a steam engine, promised the functionality of Spinning Jenny, sold fabric factory owners on the idea who fired their workers to buy these machines. The people they fired were promised that theyād have to work less, but werenāt told that they would be paid less too.
Now these machines turn out to not actually provide the smooth, spinning motion required for spinning thread. Factories are facing expensive production outages, compounding the expenses of getting these machines installed and canāt afford to hire all the workers back.
The machine shops built to produce these Sloppy Jennies or their parts will eventually find it harder to sell their iventory. Once they go under, their workers will also become economic casualties.
Weāre at the point where we need to reinforce worker protections. We need to push for a system where our livelihood isnāt contingent on the amount of work we do. Only then can that utopia even manifest. And to actually make progress:
Yes.
Weāll need to understand the social dynamics of such technology to better prevent the disastrous side-effects. Weāll need to compare historical developments with present circumstances and plans to account for future developments. Weāll need to study the psychological effects of interacting with human-like machines to be able to correct course where needed.
Whatever control mechanism is supposed to prevent machines from producing harmful output will need heuristics based on those disciplines to assess the dangers of that output.
Iād include philosophy too. We donāt need neural networks that are a lesser version of human ones, nor just scaled-up variants that also replicate all the human inefficiencies, errors and biases. Those are the pure MINT approach to the problem, and itās clearly coming up short.
We need to find a rigorous mechanism for representing semantic knowledge in digital systems that donāt just imitate, but surpass human cognition. We need a workable philosophical grounding for logical and mathematical approaches to modelling knowledge of concepts rather than just language.
We need to get over the error that LLMs are āthinkingā or ājust like humansā. Theyāre not, but as long as weāre stuck on the idea, we canāt fix it.
Also, we really, really should spend less time thinking about whether we could and more about whether we should.
Same here. Itās the best way to test my current views and beliefs and reshape or refine them.
Ah yes, hello there, fellow German. :D winkt frƶhlich auf Deutsch ;)
Yes and no. While I do absolutely agree about the problem of reliability and false confidence, I do not agree on the general understanding of LLMs. You make it sound like itās merely a predictor of likely words, like your phone keyboard may suggest words you usually use in sequence. But it is really not that simple. I disagree with the - especially here on Lemmy - very widespread āparrotā analogy in the sense that imho people have a very oversimplified idea of what large language models actually are. They donāt just count statistics of word sequences and then spit out the most likely combination. The amount of text they learn from is so big and their structure is complex enough that they can actually learn underlying patterns about WHY words are usually arranged in a certain pattern. That way, they can absolutely learn concepts and āskillsā like logical reasoning, to a certain extent. The exact patterns they learned arenāt even fully understood, but obviously it does work. Even early GPT models learned to do arithmetic - there are publications that show that GPT was able to calculate arithmetic problems that were not in the training data, although it did make mistakes. Funnily, the mistakes seemed similar to mistakes humans usually make, like forgetting carryovers in additions or subtractions. Still, it had apparently learned the concept of arithmetics, just by seeing examples (and maybe explanations - we donāt really know). Modern LLMs absolutely capable of a certain level of logic. You can give ChatGPT a task itās never seen before and chances are it can solve it. Even if it canāt - that doesnāt make its reasoning qualitatively different from how a human brain works. My argument is that humans learn how to think ālogicallyā by seeing examples too - and they fail at logic all. the. damn. time. Go to a city center and ask people logic puzzles - see how many will be able to solve them correctly (and maybe compare the result to an LLM). I wouldnāt even be surprised if by now, even asking medical questions to doctors would, on average, yield worse results than a state-of-the-art LLM. I did a PhD in computer science a few years ago and I can tell you, I would probably rather trust the latest ChatGPT model on most questions regarding my field than myself. It has just become that incredibly good. Iām absolutely not saying itās perfect, it still makes stupid mistakes and you have to be aware of that, but so do people. I have not seen a single convincing argument why the concept of logic or reasoning that humans have is fundamentally different from that of LLMs. Most people suck at logic. And most men in my life regularly spit out false knowledge with a confidence only a mediocre white man and an LLM can have. ;P
In all seriousness - LLMs lack sensorimotor interaction with the world, and even if we give them robot bodies, they will never know what itās like to be human. In that sense, they will not replace us, and they will never be human. I would even agree that they will probably never be conscious (although of course we canāt be sure, especially since we havenāt understood consciousness yet in the first place). Their biases and their āway of thinkingā will also most probably always be different from humans. But I do not agree that they cannot be āintelligentā in a logic/reasoning sense and that they are merely āstatistical parrotsā.
Agreed. Iām fascinated by how far AI has become, but it just isnāt where marketers claim it is, and we canāt be sure it will get there anytime soon.
True. Political dependencies are a serious problem and Iām pretty sure they play a big role in why the US actually lets AI companies get away with all the shit that they do.
Yeah. The point is: ChatGPT will not make you an expert. ChatGPT may act as an expert and even provide accurate information, but citing its answers will not magically make you an expert yourself. That is something that needs to be hammered into peopleās heads.
Fair enough. We need to find ways to use AI as a useful tool, not a tool to make people lazy and stupid. Tbh, I think it will happen. It happened with Google as well. People used to use Google wrong all the time when it was still new. They clicked the first search result and believed absolutely everything it said. Iām kinda old, I remember that time. :P Iām pretty sure it will be similar with LLMs. By using them, we will learn their limitations and shortcomings, and many of us will learn the hard way. The marketing strategies of AI companies are NOT helpful with that.
I absolutely 100% couldnāt agree more. AI has already had a lot and probably will have even more impact on us, on other technologies like medicine, biotechnology, probably chemistry, material sciences, and loads of other key technologies. We will have a LOT of trouble keeping up with that acceleration of technological progress as a society. In a world where even the existence of the internet doesnāt seem to have been digested fully, AI has the potential to completely wreck everything. And certain people will do all they can to exploit that to gain power and money. And we will have to be faster than them.
In that sense, I agree that
is probably right. The EUās AI Act is a good first step, but the world is in this together, and with the current geopolitical situation I honestly donāt see us working together here for the good of humankind. So maybe trying to somehow decelerate technological progress really can be a reasonable way to take pressure out of that system. Iām not sure itās possible though.
Quantity of training material doesnāt confer new abilities. It makes the resulting weights more representative of the language of the materials, but it doesnāt give something the text doesnāt have.
This fallacy is why I say philosophy should be more widely taught: the relationship between symbols and semantics isnāt quite so trivial. In the specific context of computational conscience, the Chinese Room is a well-discussed argument that demonstrates how command of language doesnāt necessarily require or produce understanding of the same. We can argue about the implications for human consciousness (Iād rather not), but the critical part is that processing a foreign language doesnāt translate it into mine.
For a more practical example, consider the issue of legal arguments citing made-up or irrelevant precedence cases. It is trivial to check whether a given case reference actually correlates with an actual case. It is critical that your citation both refers to an actual case and correctly reflects the contents. Someone who understands the nature of legal arguments knows why a certain arrangement needs to be an extant item in a finite set of instances of that arrangement (namely, a topically relevant subset of all legal cases in history, which is also a finite set).
Yet LLMs get it wrong. They get the shape right, but the filling is a game of Russian Roulette. When drawing a semantic connection between the current context and a related case, it should be a no-brainer to correctly write down the reference to that related case, but they donāt do that. They draw on the trained set of symbol correlations to produce something likely.
The same goes for essay prompts in exams where a human should recognise that an instruction to include references to Madagascar, in white font on a white background, isnāt actually part of the question but rather a trap to catch blind copy+paste into AI. The AI doesnāt understand that context or that it should disregard that part. It doesnāt actually know why the instruction is there, it just processes it into part of the context.
That is a damning verdict for a machine literally invented for computing. If there is one thing a computer should be good at, it should be the thing it was built for. Carryovers (overflow flags) are part of the most fundamental ALU design. The fact that it reproduces human error shows that it doesnāt actually understand the assignment, it just imitates the training material. If it understood that the reason a certain pattern is there is because the human writing it made a mistake, it should be able to correct it instead.
Otherwise, it is a parrot, or perhaps a really studious child thatās great at imitating adults, without any care for the actual semantics. A computer making as many or even more mistakes than humans is useless (for that task; we agree that they can do some tasks just fine). If AI should be useful universally, it needs to understand these semantics.
Language is a tool for communicating thoughts and perceptions, but that doesnāt work the other direction. Words do not imply reasoning.
Yeah, I wonder what material LLMs developed by companies with white, male CEOs are dominantly trained onā¦
Iād say itās less about deceleration itself, more about diversifying the efforts and exploring alternate avenues to achieve the things theyāre lacking rather than pouring those resources exclusively into LLMs. The deceleration of LLM development doesnāt have to mean a total deceleration of progress.
You left out half of my sentence though. Size of the model does in fact confer new abilities. See, for example, this paper: https://arxiv.org/abs/2206.07682
All of the problems you describe that LLMs have - itās true, and it holds for current LLMs. But it is not necessarily a fundamental limitation. Time will show how much better LLMs will become. I think the time of super fast progress is probably over, but there will still be improvements.
An LLM is not a computer. An LLM runs on a computer. Youāre mixing up two things here.
I think you vastly overestimate human abilities. Itās a phenomenon I come across here on Lemmy all the time. Itās like people believe their brains are capable of some sort of āanalytical logicā as opposed to ānumerical logicā. As if āunderstandingā something was some sort of godly ability. I donāt believe in a supernatural spirit or anything like that. We learn from external influences, we are statistical parrots as well. There is no absolute truth mechanism in our brains. Weāre conscious, yes, but just because understanding something feels so absolutely logical and true to you doesnāt mean itās not just a result of the statistics your brain has learned from.