What happens if you play with a club(s) that has the wrong flex? The clubhead can enter the impact area at a different angle of attack, the club face can be trailing (leaving it open), and you can be losing distance because the shaft isn’t fully loaded. These are just a few of the reasons to have a proper fitting shaft.
One thing to keep in mind is that the shaft should deliver a square club face at impact. Your shaft flex should not just match your swing speed, but also your swing type, and swing tendencies.
The wrong shaft can create inconsistencies at impact causing an inconsistent ball flight which can lead to a player losing the confidence in their swing, especially of the tee during a pressure match.