It's actually momentum, which is just mass times velocity, as opposed to kinetic energy, which is one half mass times velocity squared. When you fire a gun, nearly all of the kinetic energy goes into the bullet but exactly equal (and opposite) amounts of momentum go into the bullet and into the gun (and whoever is holding it).
In that case, imparting kinetic energy to the bullet is the desired outcome since the bullet will, in turn, deposit that kinetic energy into its target.
A rocket is both like and unlike a gun. While the basic physics are much the same (both are heat engines) the ultimate purposes are very different. A chemical rocket first converts the energy stored in the propellants into heat energy by burning them in the combustion chamber. Then it uses a nozzle to convert that heat energy into kinetic energy of its exhaust. Unlike the gun, this is not the ultimate goal but it's necessary if you want to impart momentum to the exhaust, which in turn imparts an equal and opposite momentum applied to the rocket. That's the ultimate goal. On the other hand, the goal of a gun is to impart kinetic energy to the bullet, and the recoil is just an annoying but necessary side effect. In a rocket, the recoil is the desired effect and the kinetic energy imparted to the exhaust (and the effect on anything it happens to hit) is just the annoying cost of doing that.
Unfortunately, because momentum (which you want) goes up only linearly with velocity, while kinetic energy goes up as velocity squared, increasing exhaust velocity to achieve more impulse (thrust times time) with a given mass of propellant requires a linearly greater total amount of energy for each unit of impulse. That's the famous "tyranny of the rocket equation."
