The fact that all of the witnesses didn't point to one location as the source of the gunshots, and the fact that one of the locations that they pointed to couldn't possibly have been the source of the shots, means that at least some of the witnesses identified the wrong location as the source of the shots. In other words, they were confused by the echoes.
Not only were they confused by the echoes in Dealy Plaza, most of the earwitnesses were unfamiliar with the sounds of rifle fire. Many first thought a motorcycle had backfired or even that some kids had tossed firecrackers into the street. Only experienced hunters like Governor Connally and Officer Baker immediately recognized the shots for what they were.
Even without echoes, each rifle shot creates at least three distinct sounds from three different directions: the shock wave from the supersonic bullet; the muzzle blast from the rifle; and the bullet hitting its target. Only the muzzle blast seems to come from the position of the gun.
The shock wave from the bullet appears to come from an entirely different direction, namely the position of the bullet at the moment it created the shock wave that you hear. People in different locations will perceive it as coming from different points along the bullet's path, and some will even be off by 180 degrees. Adding to the confusion is the fact that the bullet shock is usually louder than the muzzle blast and arrives before it.
I speculate, but have not verified, that the 'crack' of the bullet will seem to come from the position of the bullet when its velocity relative to the observer (not its actual speed) drops to the speed of sound.
Those who are highly familiar with rifles learn to ignore the bullet shock wave and associate the muzzle blast with the location of the gun. Connally and Baker both correctly identified the direction from which the shots were fired.
I'm sure the witnesses closest to the assassin would have had the best chance of identifying the source of the gunshots.
Indeed. In that category we have Oswald's coworkers on the floor below him, who even heard the ejected shells hitting the floor. And we also have Howard Brennan across the street who actually saw Oswald firing the shots. Isn't it curious how the conspiracists invariably ignore or dismiss these witnesses, relying instead on a few hand-picked distant earwitnesses who were confused by the physics and the acoustics into giving the direction the conspiracist prefers?