Does a power amplifier actually amplify power?

No but it controls within it`s limitations the ratio of output power to input power. Almost all of the output power is furnished by the dc Voltage source that feeds the power amplifier. If that dc Voltage source that feeds the amplifier is considered as a part of the amplifier then I guess you could say that a power amplifier actually amplifies power.