@lisihocke great article and summary!

My 2 cents:
Input validation is indeed a topic that is easy in theory, but can quickly become a can of worms in reality. In the case of BFF (like systems) you’ll easily run into ‘confused deputy’ like issues as you want the gateway as lean as possible, meaning the ‘the least amount of business logic’ and input validation increases the amount of logic, while on the other hand it’s the ideal arbiter to block or flag suspicious messages. Another problem that you’ll run into is that the sources might change without notification causing the input validation to break functionality.
In a sense it’s the same battle that IDS/IPS/WAF systems have had for decades.

Splitting the validation into a technical (ie headers) and business (ie body) section sounds reasonable (and is the approach most WAFs take), but also the least valuable one as that is the section with the lowest attack surface (propagating header manipulation is really hard and basically limited to cookies).

IMHO there is a lot to gain from validation on the businessy parts, but you’ll have to find a way to prevent problems by identifying changes early on. Contract based integration tests could be a solution to this and would also bring dev and sec together.