Presumably NASA had a planned schedule of tasks and experiments to be performed on each mission and each Apollo only carried the equipment required for those particular tasks. Obviously the earlier missions didn't need to utilise the space. Each module was built to a standard design though, so the space was there if the plans were amended (or for ballast, like it says in the pdf).
I'm only guessing though! I'm sure there are other people here who can give a better answer.
Well, what would be the alternative?
If they knew that later missions were going to have to carry additional equipment, the only option open to them would be to have a different design of CSM. The cost and logistics of that would be immense.
Now I'm just guessing here, but IMO there is also the issue of weight & balance.
Apollo 15, 16 and 17 were the only ones to carry the lunar rover. It was installed in the descent stage of the LM and that additional weight would have needed balancing on the "other side" of the Centre of Mass, which would put it right about the forward end Sector 1 of the CSM
______________________FWD___________________ROVER