ApolloHoax.net
Apollo Discussions => The Reality of Apollo => Topic started by: molesworth on July 11, 2016, 10:38:52 AM
-
I'm sure I'm not alone in being fascinated by the computer technology used for Apollo* (and other missions, manned and unmanned), so I was pleased to find out that the source code for the AGC has been uploaded onto GitHub.
Here's a link to the original article - http://qz.com/726338/the-code-that-took-america-to-the-moon-was-just-published-to-github-and-its-like-a-1960s-time-capsule/
And here's a link to the source on GitHub - https://github.com/chrislgarry/Apollo-11
Enjoy... ;D
* - quite probably a childhood fascination with space technology and computers is what spurred me into a career in software :)
-
OK, where's the 1202?
-
OK, where's the 1202?
line 205 in https://github.com/chrislgarry/Apollo-11/blob/master/Comanche055/EXECUTIVE.s
-
See also line 146 in the same file. That's the 1201 alarm. Note from the comments that they are variations on the same error. The 1202 alarm comes from having filled up the core-set table, roughly equivalent to the process table in a modern OS. It represented the EXECUTIVE's (the AGC equivalent to the kernel) knowledge of a job being executed. The 1201 alarm on line 146 arises when a program tries to obtain a range of erasable memory to use for its variables. The problem in each case, as you probably know, was the failure of real-time tasks to complete on time and exit, which in turn was caused by an abundance of counter interrupts from the rendezvous radar. In the mode selected at the time, the radar's 800 Hz power supply signal was not in phase with the AGC's reference 800 Hz signal, thereby causing double the number of interrupts.
-
That pesky landing radar :) o what did they do to the program to eliminate those two alarms?
-
They fixed the peripheral hardware bug that triggered the problem. There really was no way to fix it in the software itself, except maybe to disable the interrupt entirely when it's not expected.
-
That pesky landing radar :) o what did they do to the program to eliminate those two alarms?
Rendezvous radar. The gist was to have it operating during the landing, so that in case of an abort the LM would know where the CSM was immediately and wouldn't have to wait for the radar to warm up and start sending good data. Having the radar in that mode was a late, and relatively untested, addition to the flight plan. It wasn't known at the time that it would cause the unsynchronized radar electronics to send interrupts at twice the rate.
It's unclear what software changes if any were implemented between Luminary 1A and Luminary 1B; the mission report just informs us that spurious interrupts would not be processed. It goes on to say that with the radar selector switch in anything but the computer-control mode, the radar angle registers in the control electronics would be zeroed and not cause interrupts, so it's likely this is how the problem was fixed. It's a better solution anyway than trying to special-case the software. It's my understand that the Apollo 11 tweak -- keeping the rendezvous radar "hot" -- would still work, but simply not send data to the computer until the radar mode selector switch is set to LGC. The computer would then immediately begin receiving good data.
-
The problem was a remarkably subtle one. Like so many bugs, it's obvious in hindsight only.
The LM rendezvous radar measured antenna angles as well as range and range rate. This being before the modern era of optical shaft encoders, it used angular resolvers, a rugged and venerable old technology closely related to the synchros widely used in WW2 to aim naval guns. (Resolvers are generally 2-phase, synchros are usually 3-phase.)
Resolvers and synchros are a special kind of motor/generator. An AC excitation is fed via slip rings or rotary transformer to the rotor, inducing AC in two (resolver) or 3 (synchro) field windings that depends on the position of the rotor. In a synchro, the field windings are connected to those of another synchro whose rotor is fed from the same AC supply. Nothing happens when you turn on the power until you twist one of the shafts; then the other turns in precise synchronism with the first.
A resolver is similar except instead of driving a second device the induced AC stator signals are digitized and fed into a computer to read the angular position of the resolver shaft. In a 2-phase resolver, one stator signal has an amplitude that varies with the sine of the rotor position and the other with the cosine. Take atan2(cosine,sine) and you have rotor angle.
But this requires that the computer sample the same AC signal powering the resolver and this wasn't the case on Apollo 11. In the specific configuration of the system, for some reason the resolver and the computer were given 800 Hz AC signals from separate oscillators. They were locked in frequency but not in phase, so when they powered up they settled down into one of several (three?) relationships at random.
Turns out that some of these phase relationships posed no problem, but others were such that even a tiny jitter in the relative phase of the two supplies could turn into large changes in the resolver signals, making the computer think that the resolver rotor was turning slightly when it was actually not turning at all. Hence the very high spurious interrupt rate on the computer.
-
Nothing happens when you turn on the power until you twist one of the shafts; then the other turns in precise synchronism with the first.
And the control twist was often provided by a mechanical analog computer. The fire control computers on these warships were works of art.
[F]or some reason the resolver and the computer were given 800 Hz AC signals from separate oscillators. They were locked in frequency but not in phase, so when they powered up they settled down into one of several (three?) relationships at random.
The radar had three modes: SLEW, AUTO, and LGC. SLEW means manual control. The radar antenna is stationary unless the crew manually operates the shaft and trunnion driver switches. You would use this to roughly point the antenna in the general direction of the CSM. AUTO mode is useful when the antenna is pointed close enough to the CSM that its own electronics can maintain a track. If the CSM drifts off center, the controller electronics can send shaft and trunnion drive commands.
But mostly you wanted LGC mode, in which case the radar was sending range and angle data to the computer via counter interrupts. Also, the LGC could command the radar antenna by sending shaft and trunnion commands. Due to mission ops overkill, flight controllers generally knew what approximate shaft and trunnion angles would apply to all the steerable antennas at all times, given the spacecraft orientation, and it was generally easiest for the crew to aim the antenna by switching out of SLEW or AUTO mode into LGC mode and punching in the shaft and trunnion angles read up to them.
But no one entirely trusted the computer, so there were methods of flying both spacecraft that didn't rely on the computer. And in the case of having to fly a rendezvous under AGS control or (heavens forbid) manual control, there had to be a way of interpreting radar data that assumed the LGC was entirely dead -- including its 800 Hz reference. Thus in SLEW and AUTO modes, AGS got the range and angle data crammed into it by the separate power supply. And the cockpit displays were also driven by the same reference signal. In theory an ascent and rendezvous could be flown by reading off the raw shaft and trunnion angle displays and the range strip. But I'd bet only Aldrin would have been able to do it.
Why in heaven's name they didn't phase-lock the LGC and radar power supplies to start with, I'll never know. But I think you are probably on the right track when you note that slightly out-of-phase signals were not considered especially problematic, especially among analog control system designers who were just cutting their teeth on digital controls.
Incidentally I ran into a similar problem in 2002. A contractor putting together an assembly for us "fixed" a design error by tying two tachometer wires to the same contact pin, from different rotating components. The "fix" was discovered late enough in the process that I and my best software engineer had to figure out a way to decode the convolved tachometer pulses on the wire. And yes, as you insinuate, the signals can combine such that there are twice as many pulses as expected, the correct number of pulses (only much longer), or the agonizingly ambiguous case where the falling edge of the outgoing signal combines with the rising edge of the incoming signal. Luckily these were square wave pulses and we got by until we could justify the expense of reworking the assembly. I have one of these assemblies on my Shelf of Shame in the office.
-
Had they just done all this on a Raspberry Pi, they probably would have had plenty of CPU cycles left even with the spurious RR interrupts...
-
So, are the 1201 and 1202 alarms something akin to a "stack overflow" in a CPU?
-
Sort of. Or malloc() returning NULL.
-
Had they just done all this on a Raspberry Pi, they probably would have had plenty of CPU cycles left even with the spurious RR interrupts...
Were They available in the late 60's?
-
.... I have one of these assemblies on my Shelf of Shame in the office.
Just how large is this shelf? :)
-
So, are the 1201 and 1202 alarms something akin to a "stack overflow" in a CPU?
If you want to compare it to a POSIX compliant system, 1201 would be equivalent, as ka9q has said, to malloc() returning a null pointer. Systems based on call stacks weren't well known when the AGC was designed, and the AGC had none. There wasn't any such thing as a stack overflow in that architecture. The few thousand words of erasable storage were treated as a POSIX heap divided into contiguous fixed-size regions. All program local variables had to fit within the assigned region. As the programs were in the non-erasable core rope, code patches had to appropriate one or a portion of one of these regions. This feature was used on Apollo 14 to hold the software patch to fix the flaky Abort button.
1202 would be more akin to fork() returning -1. There wasn't as evolved a concept of a process as there is in POSIX, but think of the AGC as having a process ulimit of 6 (AGC) or 7 (LGC). Those numbers were worked out in testing and represented a very careful allocation of computer resources at each point in the mission. With the LGC in P63 or P64, no more than 7 task slots ("core sets") were needed and the LGC was operating at 90% capacity.
Keep in mind, however, that this operating system was a brilliant combination of revolutionary, simple, and reliable. Those 7 core sets were filled in priority order. So the process denied a process table entry (1202) or an erasable memory allocation (1201) was the lowest priority process that would be running at that time. And at 90% capacity, the programmers envisioned that either of these alarms might occur momentarily and then go away if the computer load spiked. Steve Bales' handwritten notes on program alarms bear this out -- the 1201 and 1202 alarms were a problem only if they were persistent. That would mean a program in the LGC was persistently being starved of attention and might cause a problem later.
The story behind Bales' notes is famous. During the last MOCR simulation a 120x alarm had been thrown and Bales had mistakenly called an abort. His backroom guy Jack Garman set him straight, and Bales dutifully recorded it. And if you listen to the flight controllers' loop during Apollo 11's P63, you hear Bales call "go" on the 1202 alarm, and then say "That's the same thing we had!" referring back to the problematic sim. He later reported that Garman had been on the spot with the recommendation on the guidance loop while Bales was still skimming his notes to see if 1202 was in the group of acceptable intermittent alarms. But had they not simulated program alarms, Bales might well have called an abort on Apollo 11 without there needing to be one. You never know what bit of your training is going to be the one that saves a mission.
-
...
The story behind Bales' notes is famous. During the last MOCR simulation a 120x alarm had been thrown and Bales had mistakenly called an abort. His backroom guy Jack Garman set him straight, and Bales dutifully recorded it. And if you listen to the flight controllers' loop during Apollo 11's P63, you hear Bales call "go" on the 1202 alarm, and then say "That's the same thing we had!" referring back to the problematic sim. He later reported that Garman had been on the spot with the recommendation on the guidance loop while Bales was still skimming his notes to see if 1202 was in the group of acceptable intermittent alarms. But had they not simulated program alarms, Bales might well have called an abort on Apollo 11 without there needing to be one. You never know what bit of your training is going to be the one that saves a mission.
Yes thanks to simulations they got through what would have been a definite downer! I wonder if Neil would have ignored the alarm and just gone it an landed? I remember a NOVA(?) documentary concerning Deak Slayton and the original Mercury astronauts chatting with each other. During the show Alan Sheppard was musing if he might have ignored an abort when the radar was giving problems. I believe that was fixed with a reset of the AGS.
-
I wonder if Neil would have ignored the alarm and just gone it an landed?
I rather doubt it because he was known as a fairly by-the-book pilot. If Kranz had called an abort, I'm pretty sure Armstrong would have obeyed the order.
During the show Alan Sheppard was musing if he might have ignored an abort when the radar was giving problems.
Ed Mitchell was of the very firm opinion that Shepard would have ignored an abort order, which worried him slightly because it implies a "land or die trying" attitude. Apollo 14 was Shepard's last flight anyway, so that would have encouraged him to push the envelope a little.
-
It was best for all concerned that the systems were working during the final moments of each flight. :)
-
Kranz's opinion (IIRC) was that Shepard may well have attempted a landing even without the radar, but that the propellant margins were so thin that he would eventually have been forced to abort. I don't think even Shepard would commit deliberate suicide rather than fail to land on the moon.
I love the call that saved that landing. Just like the call that saved Apollo 12, it was nothing more than "try turning it back off and on again and see if that works". In the Apollo 14 case, the radar had incorrectly powered up in its low range mode so it was unable to sense the surface when first expected. Cycling power got it into the correct high range mode, and it switched normally to low range mode at the proper time.
-
When all else fails, re-boot. :)
-
...Bales might well have called an abort on Apollo 11 without there needing to be one. You never know what bit of your training is going to be the one that saves a mission.
As happened on the very next mission with John Aaron's call of, "Flight, try S-C-E to Aux."
[Edited to add: Oops, already mentioned by Ka9q in post 18.]
-
I think almost every Apollo lunar mission had some glitch that could very easily have aborted the mission.
Apollo 11: computer restarts during landing; low propellant margins
Apollo 12: lighting strike during launch
Apollo 13: (we know about that one)
Apollo 14: T&D docking problem; LM abort button intermittent short; landing radar startup glitch (more than their share)
Apollo 15:
Apollo 16: LM steerable antenna failure (not sure this ever really risked an abort); CSM gimbal motor problem
Apollo 17:
I can't think of a mission-threatening glitch in 15 or 17, but that doesn't mean there wasn't one.
-
I think almost every Apollo lunar mission had some glitch that could very easily have aborted the mission.
Apollo 11: computer restarts during landing; low propellant margins
Apollo 12: lighting strike during launch
Apollo 13: (we know about that one)
Apollo 14: T&D docking problem; LM abort button intermittent short; landing radar startup glitch (more than their share)
Apollo 15:
Apollo 16: LM steerable antenna failure (not sure this ever really risked an abort); CSM gimbal motor problem
Apollo 17:
I can't think of a mission-threatening glitch in 15 or 17, but that doesn't mean there wasn't one.
I remember that the explanation for the A11 low fuel was the rock strewn area, caused by the trajectory being long since they had delta v error nearly at mission abort rules. I don't remember what reason was discovered for the higher delta v, though.
-
Well, a contributing factor to the low propellant on Apollo 11 was crew distraction caused by the computer alarms. Armstrong thought he could have picked a better landing spot if he hadn't been distracted.
Also, the propellant wasn't quite as low as it seemed. The quantity light latched on when a low level sensor was uncovered, and this happened early because the propellant was sloshing around in the tanks.
-
I thought the high delta v error occurred when they were in P-63 or P-64, before taking over manual control.
I'm thinking it was one of the controllers and Kranz, talking about the high delta v in a NOVA series. But you may be correct as to when it occurred.
-
I thought the high delta v error occurred when they were in P-63 or P-64, before taking over manual control.
I'm thinking it was one of the controllers and Kranz, talking about the high delta v in a NOVA series. But you may be correct as to when it occurred.
A contributory fact was that that they separated from the CSM before the docking tunnel had reached zero pressure. This imparted a small delta-V to the LM. That allied to an imperfect knowledge of the Lunar gravitational field caused them to land long.
-
What's strange about that one is that even if there had been a slight blow-off from tunnel pressure, that should have been seen by the LM's IMU, which would have adjusted its state vector to reflect it. That is, unless they later initialized the LM's state vector to one produced before separation, which would seem like a strange thing to do. In fact, they were tracking the LM throughout its independent flight, although they might not have been confident enough in the results to actually upload it into the LM's computer. (Tracking only gives you range and range-rate along the moon-earth line, and just a fraction of an orbit of this would not constrain some of the numbers very well).
My bet is still on a bad lunar gravity model. You can have absolutely rock-stable, accurate and drift-free IMUs and accelerometers, but without a precise gravity model you're going to get bad results.
Not until the recent GRAIL mission did we finally get a really good model of the moon's gravity field, on both sides. (The far side was especially poorly known until then.)
-
Here is the mission report and the only aspect I can decipher is that the de-orbit burn occurred late, thereby allowing for the 20K feet up range error from the original landing point. Page 54-55
https://www.hq.nasa.gov/alsj/a11/A11_MissionReport.pdf