The Ankler has a very interesting article from a producer about how he uses LLM AI, or imitative AI, in his workflow. It is a fascinating read by someone who is taking the tool and the harm it can do seriously and attempting to make the life of his assistant easier and more productive, and you should really go read the whole thing. It is perhaps the best laid out plan for using imitative AI for a business I have ever seen, but it still highlights how much hype is following these technologies.
The writer, Erik Barmack, presents his interactions with AI in what I consider the best possible light. He knows that the tools are inadequate in many areas, and he knows that the best use of them given those inadequacies is to augment his assistants, not replace them. He leans very heavily on script coverage — reading and analyzing scripts for commercial potential — as a place where AI can make his assistants (the class of worker who traditionally does script coverage) better at their jobs. He is upfront about the dangers of AI, noting that AI script coverage inevitably reverts to the mean:
Because the analysis tools rely on identifying patterns and trends, there’s definitely a bias towards scripts that follow standard formulas. Sometimes that’s exactly what you want — after all, these formulas work for a reason!
If a buyer asks for something like The Man From Uncle but in Mexico with a female protagonist, these AI tools will be great at identifying whether your script for La Mujer Del Tio is hitting the beats it needs to.
But if you’re relying too heavily on AI analysis, you’re going to miss those truly new and iconic projects. Psycho (spoiler!) kills off its lead around 45 minutes into the film.
Here’s one more spoiler alert: AI isn’t going to like that.
And that is my first potential problem with this usage. AI won’t find really good movies, merely ones that follow a pattern. It would have rejected, as he notes, Psycho, a classic film. Perhaps that is fine in the purely commercial world of film production, but I doubt Barmack actually thinks that way. Most people who produce want to produce great movies that are profitable. Like all business, they will settle for average movies that are profitable, but I doubt Bramack or any serious producer would be happy to have missed out on a classic film.
Bramack also mentions that AI can hallucinate material in a script, meaning that you cannot be certain the analysis is entirely fair or accurate. In both cases, it seems that there is a bit of a jump between using AI and getting to what Bramack considers its real value:
With Claude’s help, we spend way more time looking for new projects now that we can quickly discard scripts that don’t fit our mandate. I’ve also found the quality of analysis from my team has gotten stronger as they’ve focused less on summarizing.
That is a fascinating comment. He clearly sees value in the usage, but it’s not clear how he goes from warning about missing the next Psycho to better focused assistants. Perhaps he really is just focusing on standard beat-sheet movies and content to let the oddballs slide. Maybe he considers the AI summarization to be a clear enough guide to weeding out only the complete disasters (such as “… movie about a Black, teenage clone of Heinrich Himmler going to a school for dictators is a pass (a real project I was pitched).” Remind me to never badmouth my own ideas ever again.) and putting all of the not clearly disasters in front of his more discerning human assistant. All of that is plausible, but it doesn’t seem revolutionary in terms of how they work. At best, they have gotten a first cut “crap” filter (which I do not mean to downplay, I am sure it does save them time). At worst, well, they miss the next Psycho because a human never saw it.
It does seem likely that even the crap filter use case is likely to throw away some otherwise intriguing scripts — given that AI calculates the next most likely token based on its training data, a preference to the median entertainment seems built into the products. Imitative AI will inevitably, well, imitate. If you don’t put human eyes on it, then you run the risk of AI throwing away the next classic because it does not adhere to the median script structure. Barmack even encourages writers will benefit from this kind of coverage because:
Do you know how many times a producer says they loved your script and actually read it? Zero. That’s an exaggeration — it’s probably more like one — but with AI directing which scripts they should spend their time on, and with efficient beat sheets to keep them reminded of key plot points, they’ll probably be able to actually read them and provide better, more thoughtful, feedback.
Again, the focus on standard modes of storytelling being enabled by AI raises its head with little in the way of conversation about how this method of script coverage gets you to the next Psycho.
Which raises a question for the AI industry as a whole: given that Barmack’s team is not replacing human assistants with AI script coverage, is there enough savings to Barmack to justify a price that will sustain the huge storge and compute costs inherent in this generation of imitative AI? Given the financial issues at StablityAI and the closure of Inflection AI, I don’t think the answer is yes. Barmack’s workflow, while perfectly logical and obviously helpful to him, doesn’t seem the kind of money driver needed to keep these companies afloat.
This meandered somewhat, I know, because I don’t think we quite know how all of this will play out. But I do think that it seems likely that using AI even as intelligently as Barmack does will likely, almost inevitably, lead to a more median entertainment experience rather than a better one. And I do think that the limitations of the current and foreseeable generations of imitative AI manifest themselves to people who use it to try and do serious work, we will find that it moves the needle on cost and efficiancy only incrementally. For individual business, that can be helpful. But it doesn’t seem nearly enough to justify the huge amounts of money being set on fire to support these imitative AI systems.
It looks more and more like imitative AI doesn’t have any unicorns. Just maybe a zebra or two with a fake horn tapped to their heads.
