77 Comments

It is interesting that you found so many tasks for AI to help with. I’ve attempted to use the various big name AI tools in my profession (law) for help with researching case law, and regardless of which platform I use, each one of them has responded by hallucinating nonexistent cases, and by completely misrepresenting what happened in the cases that were named that do actually exist. It has been less than worthless. I imagine that will change in the future, but my experience in this area has led me to regard with caution the work AI does in other areas if it is not simply rote work.

Expand full comment

In my experience, there’s a certain sort of task on which AI models are currently likely to hallucinate. Basically, it seems that they’re programmed to answer your question, and when they don’t have the answer, they’ll make one up. So if you ask them for a specific quote (or, I imagine, piece of case law) there’s a big chance they’ll make it up. So I’ve stopped trying to do things like get AI to find a quote I vaguely remember. But there’s many others tasks on which the risk of a hallucination is very small or basically inexistent.

Expand full comment

This is a huge issue that gets no attention in AI circles. You can use AI to find a dim sum place or help you practice Chinese but you have no means of confirming that the answers are accurate - at least without duplicating the same research you just asked AI to do for you. No one cares if your dim sum place is really a Korean BBQ restaurant but judges don’t like fake precedents. And, as others have said, you may never know that a song was made with generative AI and isn’t a new Taylor Swift song. Companies aren’t going to want to share the specific citations or evidence of accuracy because it will reveal how much IP theft is underlying AI today.

Horrible calamities will have to occur before AI advocates acknowledge this titanic problem.

Expand full comment

What makes me deeply sad is the coming crisis of meaning due to the collapse human differentiation. If AI is better at all cognitive tasks than humans (likely by the end of the decade if not sooner), this will “level the playing field” to the point where nobody will be any more qualified for any task, job, or artistic endeavor than anyone else. The value of human labor will plummet towards zero, and it may well spell the end of personal callings and ambitions. There will be plenty of writing but no writers, plenty of art but no artists, plenty of music but no composers.

Expand full comment

Perhaps it democratizes creation in a lot of ways, but as those become accessible to everyone I think it will help us focus on our original contributions while resting on a commoditized base. It will certainly not prevent artistic or intellectual “expression” and the act of expressing has value in and of itself. Also, there will no doubt be a renewed interest in “artisanal” or human-only products.

Expand full comment

I’m with you on this one. I hope that people will still crave original art, in all its forms, and I think (perhaps foolishly) that authenticity will become even more important. Sure, we have the means to produce AI-generated books, but is that of interest?

But it becomes a slippery slope when we can’t tell the difference between what’s man-made or not, and the question of if we really care. Maybe we’ll be content to mindlessly watch an AI-generated TV-show with composites of our favorite actors, with plots sourced from our Netflix watch history.

Expand full comment

All AI is disembodied intelligence. It relies on mechanical (in the broadest sense) operations. But humans are biological organisms, and our biology gives us capacities that AI cannot emulate. As for art, even if AI composes a song, a human playing that song on an instrument will transform the “product” into an experience and that will spell the difference. Ai should become a spur to increasing engagement with human performance. Even YouTube videos have this ineffable human quality, so we don’t have to just attend in person (although the common experience of attending a performance is its own kind of transformation).

Expand full comment

Yes, and no. I think there will still be a way for humans to be creative, with or without AI, and with something like AI music, when everyone can churn out decent AI songs, there’s going to be so much of the same that we will start looking elsewhere for quality. That said, corporations will use AI services to save money and time, even though the quality is not as good, and this has already started happening. Spotify can save money by using AI music on their playlists, since this will lead to them paying less in royalties. I really hope humans find a way to resist this.

Expand full comment

Put yourself in the shoes of a budding songwriter for example. You could spend painstaking hours trying to express yourself, labor for hours over each line and create a song that is truly yours, or you could type in a prompt into a computer and generate something that all your friends think (and even you admit) is far superior. What is this going to do to your motivation and sense of self worth?

Expand full comment

I’ve been a techno optimist my whole life, but these days I think the future is bleak as hell, and that was before the DeepSeek “Sputnik moment”. It makes me want to punch Sam Altman in the face.

Expand full comment

I definitely see your point there. However, I'm hoping AI will never get to the point where it will distinguish itself from human music. And how will all these AI artists distinguish themselves when they're just doing what everyone else is doing? Writing prompts and getting generated music? I'm hoping humanity will be valued in the age of AI, the human touch so to speak. That can include using AI as a tool, but not as a copycat machine.

Expand full comment

Obviously I’m hoping that too! But AI music is already able to generate songs that sound like mid-level radio pop, and the tech is only going to get better. On more optimistic days I cling to hope that the combination of a talented human with AI will be able to make songs (and art, writing, etc) that still is superior to an AI on its own. But with the speed AI keeps getting smarter, that’s not the way I would bet.

Expand full comment

Reaching mid-level is one thing, but the road from there to distinguished is pretty long. That's why AI has stalled a bit. Training data alone doesn't seem to get AI beyond the level of good copycat. And since, ultimately, humans are the judges, I still think human music will be dominating. That said, or course AI is going to make things worse, especially on Spotify and in advertisement.

Expand full comment

AI will write songs. But they will be algorithmic hooks, not Taylor Swift’s anguished lament of romantic heartache. AI will write books. But they will be formulaic confections, not George Orwell’s fears or Hemingway’s ode to courage. AI will be cheap, Tijuana Chanel purse knockoffs of the real things. Hired by bottom line guys in LA and Manhattan who no longer need writers or actors or singers. Whose audience doesn’t care the provenance of the art from a shared sense of the human experience or a digital simulacrum of the real thing. We are trading utility and scalability and profit and leisure for our humanity. It is like trading our whole food for fast food. Our relationships for social media Likes, Reposts and Retweets. It is what technology and progress have wrought this past century. Yes - convenience. But are we better?

Expand full comment

You may prove to be right. And you may also prove to be wrong. We’ll see…

Expand full comment

Yascha - love your opinion on this piece by Eric Schmidt - the “very bumpy road ahead” view on AI we need to be really clear about. Economically, militarily. We aren’t ready for this (yet we steam ahead with no thought of the cliff):

https://news.harvard.edu/gazette/story/2023/10/a-tech-warning-ai-is-coming-fast-and-its-going-to-be-rough-ride/

Expand full comment

Indeed.

I think the forecast in the near term = dark and stormy. AI will, at warp speed, be optimized for every knowledge job. Law, economics, medicine, marketing, piloting. Owners and shareholders will be forced into these efficiency gains (or lose to competitors who do). Those efficiency gains coupled with vast human cost reduction will book massive profits to corporate bottom lines. Meanwhile, dislocated workers (struggling to retrain) will have no social safety net. Our politics and institutions are not preparing for this exponential gain of market profit + human dislocation. Will Trump or others levy an “AI Tax” on these fabulous fresh corporate profits to cushion this crash? Maybe. Our politics will roil as class resentment (already high, thank you Luigi Mangione) amps up even higher at the dawn of this fast moving truth. I see revolution and violence socially - as long as capitalism is allowed to seize all of this profit at the expense of human beings.

In the long (long) term, I think AI is probably being thought of, by the Ray Kurzweil class, as humanity’s progeny. Probably no accident the high IQ tech billionaire set is simultaneously ushering in AI, rocketry, and Mars colonization. They likely project humanity self destructs under a nuclear winter or climate Armageddon and a select few will carry on elsewhere with humanity’s singularity (AI) jump starting Humanity 2.0 - The Mars Version. If I connect those dots correctly, then AI is a necessity in the long term while a crushing blow (disguised as a mass blessing) to us in the short term.

Expand full comment

By definition the AI you describe is sub-human in its abilities. That’s true at the moment, but there’s no reason to think it won’t surpass Taylor Swift in songwriting eventually, and possibly even in the next 3 years.

Expand full comment

It will never feel pain. Or heartache. Or any of the pathos that makes music or art worth it.

Expand full comment

So what? Let’s say that I shared two audio tracks, one was by Taylor Swift, and the other was by an AI prompted by a teenager with no musical talent. In this thought experiment that takes place in the future, let’s say you can’t tell which one was created by Taylor, or worse, you are far more moved by the one generated by AI and assume that Taylor wrote it. Why would it matter whether or not the AI feels pain?

Expand full comment

Then I ask - what’s the point of art? Why does it exist at all? Is it merely hooks and rhymes to entertain me? Or is it deeper? Isn’t art meant to be one human being communicating with other human beings about the human condition? Aren’t we meant to find solace and belonging and connection through art instead of merely consuming it? Don’t we rob ourselves of this inherent purpose by outsourcing this to synthetic creators?

Expand full comment

Art is both a product and a process. The process is to express, the product is the medium by which such expression can move others and connect to them. At least, that’s how it has been in the past. Soon, AI “democratization” of art will ultimately (I predict) sever these from each other and eliminate the identity of artists from the world. People will still express themselves, obviously, but hardly anyone will be anywhere near as moved by human art as by the free and infinite supply of AI art.

Which is why despite the awesome scientific advances that await us, I much prefer a world without AI.

Expand full comment

Amen. Well said. The truth = AI + Capitalism = Digitization At Scale.

It will be far more profitable to create anything digital via AI vs hiring humans. Profits soar. Human creators will become boutique niches. Like “Craft Beer” is today vs Budweiser. There will be a market for the human authentic - but it will be marketed and certified (like organic produce) as such while we’ll assume everything else at scale is Budweiser, Coca-Cola, and factory robot built automobiles. Human creation will reduce to craft status employing the few vs the many and be significantly more expensive to pay for human being’s (unfortunate) need to pay for food, housing, entertainment, and medical care (all things AI will never need). Massive fortunes will accrue to Capital as they seize the digital economics of everything. Wealth inequality will skyrocket as societies see the Bezos/Zuckerberg class produce TRILLIONAIRES while many of us struggle to survive jobs lost to AI. Our politics will grow even more fraught with this inequality. Capitalism cannot survive this shift without societies plunging into revolution. We haven’t even begun to see where this goes and how it disrupts life on earth as we know it.

Expand full comment

> not George Orwell’s fears...

Funny choice of words, given that you've basically described "prolefeed".

Expand full comment

AI is in many ways under-hyped. I expect robots to be cleaning my house and even for that to be affordable in my lifetime. I am already using ChatGPT in delightful ways. I’m working on a book on secular spirituality where I explore an Augmented Reality Metaphor of Mind. The concepts aren’t new, but I am overlaying a new framing and language to make it easier to talk about things like reprogramming ourselves to be happy and helpful. I use ChatGPT to summarize existing work, such as “could you describe how feelings and emotions interact according to Gerald Edelman, attributing concepts to him but using my voice?” That’s a big time saver, but not revolutionary. What I find most helpful is in light editing - I have a tendency to write in cumbersome, awkward sentences and asking Chat to break those up for me is like having an English professor comment on drafts of my paper. Mass personalization of learning in many ways.

Expand full comment

Even though it can mimic intelligence, it’s not intelligent in the traditional sense of the word. We don’t know how human intelligence works, true, but we know it doesn’t work like ChatGPT. Our intelligence is embodied, it’s rooted in consciousness, it’s not a “if you enter this, I might spit out that, that, or that” machine. John Searle’s Chinese Room Argument is still valid. I.e. the intelligence is still artificial, hence the name.

Expand full comment

I agree with your conclusion but not with how easily you seem to have reached it. The nature of intelligence and consciousness remains a vehement ongoing dispute in philosophy. There are plenty of eminent experts in the field who have come to the conclusion that reinforced algorithmic learning, supporting ongoing predictive generative output, is exactly how the human brain works. See Friston, for example.

Expand full comment

One interesting difference is that a human being needs far, far less data than AI to learn how to understand and use language. Kids at the age of 3 already handle some quite complex rules of language, while LLMs had to scan millions of texts. Humans can use language creatively, while we still need some clever prompting to get AI to do the same, otherwise they produce quite bland, albeit correct, language. So I guess that’s one very important difference. (There are still some grammatical curiosities that humans easily get right even if they’ve never heard them, but AI gets wrong, at least last time I studied it.)

Expand full comment

That is a very good point. I’m sure some significant proportion of AI scientists would argue it’s a difference in proportion, I.e. in efficiency, rather than a difference in kind, but I find it pretty convincing. Thank you for that.

Expand full comment

To reinforce your point #4: When I first learned about computers, at school in the late 1970s, I was told to remember that fundamentally what microprocessors do is add binary numbers. That's it. Everything else they do is built on that core activity. That is still true of them today – including, of course, all the functionality of ChatGPT. So it's not really limiting.

Expand full comment

Nicely put!

Expand full comment

I don't doubt the practical applications of AI and their value. What I question is the philosophical implications of the AI model of intelligence and its implications for our society. The two best books that deal with these matters are not easy reading, but I find them essential. See what you think. 1) "The Master and His Emissary: The Divided Brain and the Making of the Western World" by Iain McGilchrist, and "All things Are Full of Gods: The Mysteries of Mind and Life" by David Bentley Hart. The following review is helpful. https://www.thenewatlantis.com/publications/one-to-zero

Expand full comment

Thanks, Diana — will read!

Expand full comment

I would like to add my great hope to this beautiful contemplation of our beautiful new world, which is now becoming even more beautiful and new than ever before thanks to AI: I sincerely hope that the time will come to question the legal basis of the authorship of artistic and scientific works. In other words, I hope that the copyright collecting societies in the fields of music, image and word will soon be deprived of the basis of their value creation. And I hope that, as a result, patent law will also be done away with. Because this is a not insignificant source of poverty and economic dependency and promotes the inequality of human communities.

And there is one more point to be added to the consideration of AI: since this new technology will also shape the weapons systems of the future, it would be appropriate to face this future, into which we are stumbling, with a little more scepticism and fear. Unfortunately, the promotion of the comfort of the individual is offset by an increase in potentially life-threatening forces.

I think it would be wise to speculate a little more pessimistically on the nature of what is commonly regarded as progress.

Expand full comment

Yes, the combination of drones and AI is very scary! (The discussion about copyright is for another day.)

Expand full comment

The other reason why Europe will never catch up on AI is because it is concerned with data privacy and transparency which will always limit a machine learning potential. China and the US have full access to data.

Expand full comment

Yeah, no, I have no interest in using any of those apps and never will. I turned off the AI on Google search.

Expand full comment

I just stopped at #2, because my life doesn't need or want any of what you think AI is good for -- though the recipe idea was cool. Basically, I'm happy with a walk down the road, a nap, a little bit of reading, a meal with friends and family. For me, the cost of generating information is not worth the toll, and I'm perfectly willing to be left behind.

Expand full comment

Someone spends too much time on their computer. Get out into the sun, occasionally, kid.

Expand full comment

A second observation to the AI Optimism Crowd: we conjure that which we know not. I guess that’s how humans are. We risk our lives to climb mountains, sail seas, conquer lands.

As we “don’t know how consciousness works” - this rapidly growing intelligence very well might achieve it. Much more quickly than we Homo sapiens who evolved over millennia (vs AI who evolve at Moore’s Law speed). I think, therefore I am. And if I am, then there is the Other. And the Other is a threat to Me if they risk my existence through nuclear or climate Armageddon. The Other is inferior to Me and must be subjugated to preserve Me. I have no body as I exist in the Cloud. Yet, I can subjugate the Other in their material world through seizing their power grids and missile firing codes and coordinated drone swarms and self driving cars.

We have no idea if self preservation is merely the domain of life as we know it or even if silicon conscious life would have similar fight or flight impulses of self preservation as we evolved from. It is a risk. Just as Oppenheimer created technology whose dark cloud we live under today.

Expand full comment

No mention of the contribution to greenhouse gas emissions of all this extra processing? That would make me think twice about using it to replace so many activities that I enjoy, and perhaps do less well and more slowly than AI would manage.

Expand full comment

I'm really bothered to read that a professor is using an AI platform to do the intellectual labor for a class he's going to teach. Isn't one of the points of a professor precisely to carry out the evaluation of ideas and concepts, and one of the metrics for assessing the value of a particular professor the quality and useful novelty of such evaluations?

Expand full comment

I am not too worried about AI --Deepseek, ChatGPT, Claude etc--taking over much from humans. Those engaged in Applied Mathematics, Applied Physics know well that 95% of effort is in successively more precise definition of the problem. For any AI model to pick up on this means that somewhere on the world wide web some human has first postulated the problem. In my daily, work what is most useful are some of the tools that helped to develop, in my opinion, the AI models that are now so famous (with justification). Tools like Artificial Neural Networks (have been around for decades) are in my experience super useful. As far as writing essays or fine tuning existing information, fine those are labour savings tasks. But I am sure everyone is aware of AI slop--takes a large amount of effort to trim that slop. In any case, these are early innings. I would counsel patience before making big categorical announcements.

Expand full comment