Buzz Armstrong said they "couldn't see stars on the moon." Therefore, you can't get a bearing from a star for navigation. Therefore, you are all stuck on the moon. The optics were just a telescope. If you can't see a star with your eyes, you can't see it with a telescope. Nobody went anywhere. End of story. Sorry.
http://www.hq.nasa.gov/alsj/aot.htm
Who's "'Buzz' Armstrong"?
You can if looking the telescope blocks out the ambient light sources that would make it so that you could not see stars and allowing your eye to dark adapt, which is exactly what it was for. In fact,
Neil Armstrong, in a much quote mined by conspiracy theorists
interview during the Apollo 11 post-mission press conference says directly "We were never able to see stars from the lunar surface or on the daylight side of the Moon by eye
without looking through the optics." [bolded for emphases]
As often pointed out ,the moon has no atmosphere. The only thing stopping you from seen stars on the surface are the amount of light from other sources making stars hard to see. Things like, the sunlit lunar surface and the lighting in the cabin,or the sun shining through the windows.
The optics allowed them to block out most of that light and take star sightings for making alignment checks.