As a Brit, I have to say I stare in slack-jawed amazement at the system in the US that allows politicians and courts to mess around with how votes are collected and counted during an election. Challenging the outcome is one thing, but actively seeking to count or discount votes that have been cast, changing how many polling stations are set up in any given county, and arguing over when the final result is actually in and valid, all while votes are being cast, is just incredible to me.
You're not alone. It's truly amazing that there aren't set deadlines for these things, subject, of course, to reasonable situational issues, such as we are currently facing. As you said, once the process of voting has started, there's little valid argument for challenging the process. It's somewhat like an ex post facto law approach. I voted on date X, following all laws and regulations at that time, but on date X + 30 days, someone said that my vote is no longer legal, and won't be counted. Um, no. And that doesn't even get into the issues Jay and others raised about polling and drop-off locations, changes to ID requirements, etc.
One thing I haven't seen people address in the whole, "mail-in voting is going to be riddled with fraud" argument is that it's the responsibility of the current administration to ensure that the voting process is valid, to protect it from interference and fraud. Therefore, if there is fraud, it will literally be due to a failure of the administration that is worried it will happen; they won't have done their jobs.