An anecdote on what it was like to work with Boeing's defense side a little over a decade ago. At the time I convinced myself that the problems I saw were the result of this project being kind of a prototype/proof of concept thingy, but after the last few years I think I was seeing the overall rot.
We were subcontracting for Boeing on a situational awareness/common operational picture system. The concept was that you had services running on various platforms reporting status and combat readiness, objects in the battlespace obtained from human and sensor observations, and a few other things - these services were written by the various subcontractors. I was responsible for the service that received battlespace object reports (HUMINT and sensor). These services would send updates to a separate GUI service running on its own platform, which would integrate and present this data for a warfighter to monitor and validate before being forwarded on to the COP at HQ - this GUI service was written by the Boeing team.
Except, here's the problem - the GUI service didn't actually integrate or manage any data. Instead, they exposed hooks to all the graphics primitives (tables, maps, etc.) and relied on the back-end services to perform the actual screen management - the back-end services were responsible for managing the screen state.
I didn't just forward BSO reports to the GUI service - I actually sent the commands to add or remove rows from the table that displayed those reports on the GUI - over a network with a quarter-second latency. And the API Boeing provided only allowed me to add or remove one row at a time, which proved to be a major issue during usability testing. Same thing was true for all the other services. The positioning service that was responsible for updating the map had to send commands to draw the actual graphics primitives on the map itself.
The GUI team (Boeing) successfully deferred all the hard work onto the backend services. And the GUI service still didn't quite work right.
Now, this design had two big problems. The biggest problem was that it could not meet functional requirements. Anytime the warfighter selected an object on the map, all the relevant tables were supposed to automatically update to show status, combat readiness, etc., for that specific object. This design specifically prevented that from happening - the backend services didn't communicate with each other, and since the GUI didn't actually keep track of anything it was displaying, it couldn't send the corresponding command to the back-end services to update their information either.
The second problem was performance - all the backend services were doing screen management over a network, which was already slow. Add to this that the APIs were exceedingly primitive. Like I said, I could only add or remove one row at a time from the table I was responsible for, and with network latency each command took about a quarter second to execute. Not a problem when you're monitoring 5 or 10 objects, deadly when you're managing a couple hundred (it took on the order of 5 minutes to update the table if you were managing hundreds of reports). Under realistic scenarios the whole thing just fell down hard. We suggested to the Boeing team that they add calls to perform bulk updates (or at least a call to delete all the rows in the table), but those suggestions were dismissed as being "too hard".
The API code that we called was generated from a spreadsheet that was chock full of spelling errors - I had at least one build break a day because I insisted on spelling it "activity" instead of "avtivity".
How this got past review and approved (especially since it couldn't satisfy some key functional requirements) is a mystery to me, meaning it was never really reviewed. Someone at Boeing should have kicked that design back with "are you f___ing kidding me" stamped on every page in red ink, but didn't.
In the interests of full disclosure, I have to point out that my code was hot garbage. It was my first adventure in multithreading and I botched it badly enough that I put the schedule at risk all by myself. But ... I wouldn't have had to do that if I didn't have to do two completely separate jobs in the first place. My service shouldn't have cared about what rows were actually visible on the screen, it shouldn't have cared about the state of a dropdown menu, it shouldn't have cared about anything but maintaining a connection and sending updates.
At the time I told myself it was because this wasn't something in a critical path so they weren't putting their best team on it, but it was part of a larger modernization program and proved to be an accurate representation of the overall effort. The Army finally got a clue and cancelled the whole thing. And while it ultimately cost me my job, as a taxpayer I was thrilled.
