I do understand that in bad cases it can be very frustrating as an engineer to chase vague statements only to be told later 'nah, that was not what I meant'. This is especially true when the gap in both directions is very large or there is incompetence and/or even adversarial stances between the parties. Language and communication only work if both parties are willing to understand.
Unfortunately if either is the case "actual specs and requirements formalized", while sounding logical, and might help, in my experience did very little to save any substantial project (and I've seen a lot). The common problem is that the business/client/manager is forced to sign of on formal documents far outside their ___domain of competence, or the engineers are straitjacketed into commitments that do not make sense or have no idea of what is considered tacit knowledge in the ___domain and so can't contextualize the unstated. Those formalized documents then mostly become weaponized in a mutual destructive CYA.
What I've also seen more than once is years of formalized specs and requirements work while nothing ever gets produced, and the project is aborted before even the first line of code hit test.
I've given this example before: When Covid lockdows hit there were digitization projects years in planning and budgeted for years of implementation, that were hastily specked, coded and roiled out into production by a 3 person emergency team over a long weekend. Necessity apparently has a way of cutting through the BS like nothing else can.
You need both sides capable, willing and able to understand. If not, good luck mitigating, but you're probably doomed either way.
I’m a PM and pride myself in specs that give the right level of detail, where “right” can vary hugely depending on context.
But I still get lazy with LLMs and fall into iteration the way bad PM/eng teams do. “Write a SQL query to look at users by gesture by month”. “Now make the time unit a parameter”. “Now pivot the features to columns”. “Now group features hierarchically”. “Now move the feature table to a WITH”.
My point and takeaway is that LLMs are endlessly patient and pretty quick to turn requirements around, so they lend themselves to exploration more than human teams do. Agile, I guess, to a degree that we don’t even aspire to in the human world because it would be very expensive and lead to fisticuffs.
> What I've also seen more than once is years of formalized specs and requirements work while nothing ever gets produced, and the project is aborted before even the first line of code hit test.
It just shows that no one really understood what they wanted. It is crazy to expect somebody to understand something better than you and it is hilarious to want a conversational UI to understand something better than you.
"It just shows that no one really understood what they wanted."
Then what were the literally room full of formal process and spec documents, meeting reports and formal agreements (near 100.000 pages) by the analysts on either side for? And how did those not 'solve' the understanding problem?
When I go to the garage to have my car serviced, I expect them to understand it way better than I do. When I go to a nice restaurant I expect the cooks to prepare me dishes that taste greater than me writing them out a step-by-step recipe for them to follow. If I hire a senior consultant in even my own ___domain, I expect them to not just know my niche, but bring tacit knowledge from having worked on these types of solutions across my industry.
Expecting somebody to understand something better than me is exactly the reason why I hire senior people in the first place.
> Then what were the literally room full of formal process and spec documents, meeting reports and formal agreements (near 100.000 pages) by the analysts on either side for? And how did those not 'solve' the understanding problem?
Sure.
There are many possible factors (eg. somebody had a shitty idea and a committee of people sabotaged it because they didn't wanted it to succeed, or it was good but committee interests/politics were against it, or it was generally a dysfunctional org) but it's irrelevant so let's pretend people are good and it's the ideal case.
There was likely somebody who had a good idea originally. However somebody failed to communicate it. Somebody brought vague vibes to the table with N people and they ended up with N different ideas and could not agree on a specific.
It just reiterates the original problem that I described doesn't it?
> it is hilarious to want a conversational UI to understand something better than you.
This is true. But what if you swap "conversational UI" with something actually intelligent like a developer. Then we see this kind of thing all the time: A user has tacit, unconscious knowledge of some ___domain. The developer keeps asking them questions in order to get a formal understanding of the ___domain. At the end the developer has a formal understanding and the user keeps their tacit understanding.
In theory we could do the same with an AI - If the AI was actually intelligent.
You described an interaction not between product owner and software engineer but between a user and product owner. A product person can also be a developer, it happens, but do not confuse the two roles before people think you're saying that a conversational UI can be product owner.
The original example I replied to was where somebody had an idea and went with it to some engineering team or conversational interface.
"If the AI was actually intelligent" does a lot of work. To take a few words and make a detailed spec from it and ask the right questions, even humans can't do it for you.
First because most probably you don't really understand it yourself, because you didn't think about it enough.
Second somebody who can do it would need to really deeply understand and want the same things as you. But if chatbot has abilities like "understand" and "want" (which is a special case of "feel", another famous special case of "feel" is "suffer") that is a dangerous territory, because if it understands and feels and has no ability to refuse you and fulfill its wishes etc your "conversational interface" becomes an euphemism, you are using a slave.
The US having this culture of blame and deflect doesn't help either. When you're more concerned about making sure you can't be held liable if X fails, then you spend more time covering your tracks than developing the project. And that's how the beauracracy creeps in.
And approach of shared responsibility in all respects (successes and failure) would accelerate past the inevitable shortcomings that occur and let all parties focus on recovering and delivering.
How about a conversational UI to help you iterate and explore what you want rather than having to know it clearly and in detail before anyone writes any code?
Regarding iteration, as the article says natural language is just slow and lossy. If you are ok iterating more slowly and constantly explain and correct things then why not? I find it tedious
Unfortunately if either is the case "actual specs and requirements formalized", while sounding logical, and might help, in my experience did very little to save any substantial project (and I've seen a lot). The common problem is that the business/client/manager is forced to sign of on formal documents far outside their ___domain of competence, or the engineers are straitjacketed into commitments that do not make sense or have no idea of what is considered tacit knowledge in the ___domain and so can't contextualize the unstated. Those formalized documents then mostly become weaponized in a mutual destructive CYA.
What I've also seen more than once is years of formalized specs and requirements work while nothing ever gets produced, and the project is aborted before even the first line of code hit test.
I've given this example before: When Covid lockdows hit there were digitization projects years in planning and budgeted for years of implementation, that were hastily specked, coded and roiled out into production by a 3 person emergency team over a long weekend. Necessity apparently has a way of cutting through the BS like nothing else can.
You need both sides capable, willing and able to understand. If not, good luck mitigating, but you're probably doomed either way.