I've been having intermittent fan speed and high CPU temperature when my CPU usage is nearly 0%. I don't run any weird apps. If I put my computer into sleep mode and wake it back up, this fixes the issue. I think this is an System Management Controller(SMC) issue, but I'm not sure. Anyone know how to fix this permanently?
Try resetting the SMC as noted in the question for your MacBook Air and see if the problem persists.
If the problem does persist it may be a failing temperature sensor, resulting in the fans running faster than normal to worst case fail-safe full-speed state, your only option at that point would be to take it to Apple Support for repair / replacement.