I think much of it has to do with a natural ability of certain people to functionally break down problems into discrete solvable atomic sub-components.
I've observed that most people do not demonstrate this ability. An example, while doing a consulting gig for a large agro-business a few years ago they wanted to solve a single question
"should we brand our new product with verbiage about it being all natural?"
Helping them resolve this question involved coaching a dozen very well educated people through a simple process of functionally breaking this question down into solvable sub-components.
"how do consumers feel about the term 'all natural'?" ->
("how can you find out?",
"are there equivalent valid terms we can test?") ->
(("do you have data that represents a consumer voice?",
"is there a way to process this data to find sentiment?"),
("who in your company might know equivalent industry terms?",
"can we ensure the regulations around these equivalent terms allows us to use them in this case?")) ->
((("lets get data from site A, B and provider C",
"let's use these tools and techniques to extract terms, timeline the usage and score for sentiment"),
("let's meet with these folks and collect there terms and use them in the above process",
"send the top 3 results to legal for regulation confirmation")
Basically just keep breaking down the questions become so trivial they basically solve themselves.
This was a revolution to them. The only method they were aware of to answer the question above was to blanket shopping malls with market survey teams at a very high cost. People in general tend to think of establish procedure not in functional decomposition and problem solving.
I've observed that this holds true for nearly everybody I've encountered who couldn't learn to program. This kind of problem solving technique was something that they simply hadn't learned to do and it's exactly what you have to do to learn to program.
"I need to write a program to do X, how do I do X?" they need to learn to break that problem down into smaller and smaller pieces until those pieces become orthogonal with the language they need to use. X might be made up of 1000 sub-components, and arriving at those is simply a matter of dividing X down in a series of phases, asking "can I answer this with the language I know yet?"
Even people who do fine in other classwork, say they breeze through calculus, aren't taught this and as far as I know it's simply not taught in academic situations outside of some of the more formal maths (e.g. any math where you spend more time with proofs than mechanical problem solving) because schools teach via composition instead of decomposition (or synthesis instead of analysis).
What I mean is that schools start from first principles, in the maths it'll be axioms and mechanics, in history it's ancient history, in language it'll be simple reading and writing skills, etc. and build up to some standard. But at no point are people taught to take a complex problem and break it down into more simply solvable sub-components. This is exactly the kind of skill that people need to learn to program and IMHO should be the entire first year of any degree program that requires programming of any sort.
The beauty of course is that if you can't break something down further, and you don't know how to solve it, it's probably solvable with a technique you simple don't know at that point...and that's where education can slot in.
AFAIK there aren't many...the only time I ever encountered anything that was like this was in the higher maths.
Even various systems/enterprise Architecture classes don't quite view the world this way. Sure you start with a high level problem, but very quickly you find yourself building up a solution rather than functionally decomposing the problem down to easily solvable atoms.
For example:
Problem: "Design an enterprise system for a company that meets these high level abstract goals"
Several areas high level engineering, logistics, and business deal with breaking problems down into their parts. Unfortunately the people that run big projects often have not studied them.
But I'm wondering if we're just doing this all wrong. If this kinda of mental technique shouldn't be taught very early in many disciplines...maybe even as early as elementary school.
It's not to say we should abandon the bottom up approach in use today, it's very useful for understanding many subjects. But there's very little done from a top-down style approach I'm thinking of...and it gives us a deficit in people who are unprepared for learning programming or similar disciplines later on.
I've observed that most people do not demonstrate this ability. An example, while doing a consulting gig for a large agro-business a few years ago they wanted to solve a single question
"should we brand our new product with verbiage about it being all natural?"
Helping them resolve this question involved coaching a dozen very well educated people through a simple process of functionally breaking this question down into solvable sub-components.
"how do consumers feel about the term 'all natural'?" ->
("how can you find out?", "are there equivalent valid terms we can test?") ->
(("do you have data that represents a consumer voice?", "is there a way to process this data to find sentiment?"), ("who in your company might know equivalent industry terms?", "can we ensure the regulations around these equivalent terms allows us to use them in this case?")) ->
((("lets get data from site A, B and provider C", "let's use these tools and techniques to extract terms, timeline the usage and score for sentiment"), ("let's meet with these folks and collect there terms and use them in the above process", "send the top 3 results to legal for regulation confirmation")
Basically just keep breaking down the questions become so trivial they basically solve themselves.
This was a revolution to them. The only method they were aware of to answer the question above was to blanket shopping malls with market survey teams at a very high cost. People in general tend to think of establish procedure not in functional decomposition and problem solving.
I've observed that this holds true for nearly everybody I've encountered who couldn't learn to program. This kind of problem solving technique was something that they simply hadn't learned to do and it's exactly what you have to do to learn to program.
"I need to write a program to do X, how do I do X?" they need to learn to break that problem down into smaller and smaller pieces until those pieces become orthogonal with the language they need to use. X might be made up of 1000 sub-components, and arriving at those is simply a matter of dividing X down in a series of phases, asking "can I answer this with the language I know yet?"
Even people who do fine in other classwork, say they breeze through calculus, aren't taught this and as far as I know it's simply not taught in academic situations outside of some of the more formal maths (e.g. any math where you spend more time with proofs than mechanical problem solving) because schools teach via composition instead of decomposition (or synthesis instead of analysis).
What I mean is that schools start from first principles, in the maths it'll be axioms and mechanics, in history it's ancient history, in language it'll be simple reading and writing skills, etc. and build up to some standard. But at no point are people taught to take a complex problem and break it down into more simply solvable sub-components. This is exactly the kind of skill that people need to learn to program and IMHO should be the entire first year of any degree program that requires programming of any sort.
The beauty of course is that if you can't break something down further, and you don't know how to solve it, it's probably solvable with a technique you simple don't know at that point...and that's where education can slot in.