One of the drawbacks of the current state of the art in hearing aids is that they remove most of the localization cues (formed by fine changes in relative frequency content and phase by tiny reflections among the pinna of the external ear). This means that the Cocktail Party Effect is much diminished.
A similar experience comes when you attempt to record sound on location; since most setups lack those fine localization cues, again, environmental noise and the long tail of natural reverberation makes the recording seem much more "muddy" and "indistinct" than the sounds felt when listened to in person.
Incidentally, I have a friend who describes himself as tone deaf. He can actually carry a tune (if the bucket is large enough) but to him a flute sounds identical to a clarinet. He would, however, have no trouble telling a trumpet from a xylophone, because the attack and evolution of those sounds are so distinct.
(I suspect, also, that in a trumpet most of the energy is in harmonics of the fundamental after the starting transient, but in a xylophone multiple modes exist -- longitudinal versus transverse to the bars, for instance -- which are harmonically unrelated. And die out during the sustain, simplifying the timbre over time).