I tried this (though with a different tool called aichat) for extremely simple stuff like just "convert this mov to mp4" and it generated overly complex commands that failed due to missing libraries. When I removed the "crap" from the commands, they worked.
So much like code assistance, they still need a fair amount of baby sitting. A good boost for experienced operators but might suck for beginners.
Plus you need to know the format of your source file to design the command correctly. How many audio tracks, is the first video track a thumbnail or the video, are the subtitles tracks forced, etc.
And in some situations ffmpeg has some warts you have to go around. Like they introduced recently a moronic change of behaviour where the first sub tracks becomes forced/default irrespective of the original forced/default flag of the source. You need to add "-default_mode infer_no_subs" to counter that.
To me it's more like a guy getting his first car and complaining that the car is driving him in a direction that may or may not be correct, despite his best efforts to steer it where he wants to go. And the only way to know whether he ends up in the right place is to get out of the car, look around, and maybe ask more experienced drivers. Failing that, his only option is to get back in and hope to be luckier in the next trip.
Or he can just ditch the car and walk. Sure, it's slower and requires more effort, but he knows exactly how to do that and where it will take him.
The beer brewers in my home town used to have a self-driving horse and cart which knew the daily delivery route going by all pubs and didn't really need a human to steer it or indeed be conscious during the trip. Expectedly, the delivery guy would get drunk first thing in the morning and just get carted about collecting the money.
Pony & trap could be largely self-driving, after an initial training period. That would have been a distinct negative to "upgrading" for some, I'd imagine.
I no longer check with these AI tools after a number of attempts. Unrelated, a friend thought there was a NFL football game last Saturday at noon. Checking with Google's Gemini, it said "no", but there was one between two teams whose season had ended two weeks before at 1:00 Eastern Time and 2:00 Central. (The times are backwards.)
libx264 is the best encoder for h264 ffmpeg has to offer so it's pretty important you bundle it in your ffmpeg install. Those commands are perfectly standard, I've been using something like that for 10+ years
I don't disagree that we need to be cautious with LLMs, but I've personally stopped asking GPT-4/GPT-4 mini for technical answers. Sonnet 3.5 and DeepSeek V3 (which is much cheaper but still not as good as Sonnet) are your best bet for technical questions.
Where I find GPT to perform better than Sonnet is with text processing. GPT seems to better understand what I want when it comes to processing documents.
I'm convinced that no LLM provider has created or will create a moat, and that we will always need to shop around for an answer.
Unfortunately you need to be tier 2 to use o1-mini. The only time I really use GPT is to summarize documents and for that, GPT-4o mini works well enough and it is significantly cheaper than other high quality models, so I never really rack up an OpenAI bill.
o1 is such a joke, worse than 4o in some ways like multiturn,
The months old sonnet feels a generation ahead of any OAI product I've used, I'll believe the hype on o3 when I see it, remember the sora and voice roll out?
I had this bizarre bug in rust networking code where packets were getting dropped.
i dumped all 20k lines into o1pro. it thought for about ten minutes and came back telling me that my packets had a chance of being merged if set in quick succession and i needed to send the length before each message and scan packets in a loop for subdivisions on the client. this bug hadnt happened before, only when running locally on a newer faster machine, and was frequent but hard to replicate.
it was correct, and provided detailed pseudo code to solve it.
the second case involved some front end code where during an auth flow ios would force refresh on returning to the browser causing authentication state to be lost. o1pro thought for about 5 minutes before telling me ios has a heuristic with which it decides to close an app on context switch based on available ram, etc, and that i needed to conditionally check for ios and store partial state in local store on leave assuming the app could be deloaded without my control.
it was correct. with some more back and forth we fixed the bug.
these are not the kinds of problems that claude and gpt<4 have been able to help with at all.
I also used voice, and video voice extensively for translation tasks in korea, japan, and taiwan, and for controlling japanese interfaces and forms for tax documents and software.
o1 is not a general-purpose model, and it's not very good at multi-turn; it should instead be given all the context upfront: https://www.latent.space/p/o1-skill-issue
what exactly do you want the llm to do here? if the ask was so unambiguous and simple that it could be reliably generated, then the interface wouldn't be so complicated to use in the first place! LLMs are not in any way best suited for one-shot prompt => perfect output, and expectations to that effect are extremely unreasonable. the reason why LLMs are still hard for beginners to use is because the software is hard to use correctly. as with LLM output goes life itself: the results you get from using a tool can only ever be as good as the (mental) model used to choose that tool & the inputs to begin with. if all the information required to generate the output were contained by the initial prompt, then there would be absolutely no need to use the LLM at all in the first place.
Hate to be that guy, but which LLM was doing the generation? GPT-4 Turbo / Claude 3.x have not really let me down in generating ffmpeg commands - especially for basic requests - with most of their failures resulting from ___domain-specific vagaries that an expert would need to weigh in on m
You have a fair point. Some LLMs are better at some tasks, and prompts can make a difference no doubt.
Perhaps at some point there will be a triage LLM to slurp up the problem and then decide which secondary LLM is most optimal for that query, and some tertiary LLMs that execute and evaluate it in a virtual machine, etc.
Oh I talked to some guys who started a company that does that. This was at an AI meetup in SF last year. They were mainly focused on making $/token cheaper by directing easy/dumb queries to smaller dumber models, but it also increases output quality because some models are just better at certain things. I'm sure all the big companies already have implementations of this by now even if they don't use it everywhere
"Writing working commands first try for every single ffmpeg feature that exists" is the highest bar I've ever heard of, I love it. I'm gonna start listing it as a requirement on job postings. Like an ffmpeg speedrun.
I don't think there's a single human on or outside of this planet that can meet that requirement, but Claude has been pretty good to me. It's certainly a much better starting point than pouring over docs and SO posts.
I know I struggled on getting a good command to “simply” make the videos from my Z8 smaller (in file size).
Usually the color was wrong and I don’t care enough to learn about colorspaces to figure out how to fix it and it’s utterly insane how difficult it is even with LLMs.
Just reencode it as is but a little more lossy. Is that so hard?
I think in the non LLM world though you at least have the trail of documentation you can unwind once you're in a bind. I don't care for prompt-a-mole fighting.
So much like code assistance, they still need a fair amount of baby sitting. A good boost for experienced operators but might suck for beginners.