I know what deadbands are, but let me make sure I understand why they're used.
Well, take a couple steps back and enjoy the notion that everything you mentioned is, in one way or another, a valid argument for the use of deadbands in any control system -- especially an attitude-hold autopilot that uses rocket thrusters for control moments.
From the steps-back perspective, consider that you nailed the notion of discrete control variables. Often a control system must provide inputs that are discretes (i.e., on-off values; e.g., opening the water inlet to the washing machine), or stepwise-variable (e.g., firing up the second-stage burner in an HVAC system), or continuously variable only within a narrow band (e.g., car engine speed cannot drop below about 700 RPM, and cannot exceed red-line speed). RCS thrusters are not continuously variable. They provide either a fixed thrust, or stepwise variable thrust in pulse mode.
And in the case of the LM ascent, the thrust is vastly oversized for the need. Consider a home heater whose heating element (whether electrical resistance heater or combustion burner) has prodigious heat injection capacity. If your thermometer drops only a degree below the set-point, firing up that huge beast of a heater -- even for a very short amount of time -- may rocket your temperature several degrees
above the set-point.
Before we get into the mechanics of thruster ignition transience, which is correct (but a second-order consideration), let's examine the other effect you nailed head-on, because it follows directly from what we observe above and from what you noted about the mass properties of the spacecraft. The spacecraft's center of mass changes, as you note, from fuel depletion and from other payload shifting such as fuel slosh and crew movement. The IMU on a manned spacecraft is incredibly sensitive. For example, the IMU on board Apollo 1 registered the motion of the entire launch vehicle due to the movement of the crew attempting to extinguish the fire and escape. If even the slightest measurable error produced a corrective moment, there would be constant correction, overcorrection (due to non-discrete controls), and a non-stop fight between opposing controls. Deadbands provide vital slack to prevent this. Other techniques include more sophisticated control laws that incorporate integrated and differential process variables. The Apollo digital autopilot implemented differential control (i.e., error rates) plus a pilot-selectable deadband.
And yes, the goal is simply "good enough" guidance, not error-free guidance.
The other concern is the classic hysteresis effect. Between the time an error is first measured to the time the system returns to acceptable is an interval during which the system is reacting. It may take only a few milliseconds for the error to generate a control input, but it may take considerable time -- perhaps several cycles of the control system -- for the control input to be reflected in the process variable. When your thermostat notes that the measured temperature in the room matches the set point, it stops adding heat to the room by turning off the heating element. But because of latency in the distribution system and latency in the thermometer, heat may continue to enter the system and cause the measured temperature to rise. Thus, aiming instead at a deadband rather than at a precise value allows for overshoot and latency in the control loop.
Spacecraft sometimes have this constraint. There are missions that require attitude errors to be corrected within a certain time proportional to the error magnitude. That is, you can accept a certain magnitude of error, but you cannot accept an out-of-tolerance condition for very long. So sometimes time-optimal control is required, not fuel-optimal control. Different control laws, and different deadband requirements.
I presume the optimum deadband can be computed from the loss in effective thrust that comes from a particular attitude error. This varies as the cosine of the error, which for small angles is approximately 1 -- i.e., small errors are inconsequential. So the optimum deadband would be the one that equates the RCS propellant needed to maintain it with the wasted propellant by the main engine due to those cosine errors.
Am I right?
Well that's one way to formulate it, yes. The science of spacecraft guidance is a tool kit of all such kinds of models that match various mission constraints. Many modern missions are orbital only, and rely chiefly upon attitude control -- pointing constraints. Planetary missions have constraints more in line with Apollo ascent guidance, and are reckoned in terms of allowable dispersion. Dispersion on orbital approach is expressed as an allowable window in the state vector -- a literal geometric window in planet-fixed space through which the spacecraft must fly, and a conical distribution of acceptable velocity vectors. Dispersion for landing is the landing-site ellipse. Dispersion for rendezvous can be expressed as tolerances on the sacred 6-tuple of orbital elements.
Abstractly considered, dispersion simply accepts that no matter how adept a guidance system may be, identical starting conditions will not result in arbitrarily repeatable end conditions, due to the accumulation of low order effects that do not repeat. Guidance dispersion is therefore simply the error analysis for guidance. And each mission (e.g., LM ascent and rendezvous) contains an acceptable dispersion. For most LM ascents, the acceptable dispersion was vast. In contrast, for LM landing on Apollo 12, acceptable dispersion was considerably narrow.
So the more accurate expression of your sentiment above is how much known guidance error can I accept, integrated (and hopefully averaged) over ascent time, and still "land" in orbit with only nominal dispersion. It's not so much the loss of thrust because the motor is slightly off-axis, but rather the naked fact that you're going the wrong direction. So you can write the optimization problem several ways now. You can optimize the deadband, for example, to balance between fuel for control during the ascent against fuel to correct the orbit.
In practice the Apollo deadband was simply switched between two fixed values, depending on the pointing constraint for the specific mission phase. The deadbands were 1 degree and 0.1 degree, respectively, if I recall correctly.