The issue is caused by Gatekeepr. It uses "path randomization" to run applications, until the user moves the app in it's final destination (/Applications for example). This causes the app to be run from /private/var/.. and Dock treats this as a new app.
To remove the quarantine attribute execute (removes recursively com.apple.quarantine):
sudo xattr -dr com.apple.quarantine /Application/SomeApp.app
Recursion may be needed for apps, that contain helpers.
To list the current attributes for an app execute:
sudo xattr /Application/SomeApp.app
Sources: