I organize my 100+ apps in a very particular order, only to find out that a few days later they are completely out of order. Imagine your iPad or iPhone doing that - would be very annoying.
Is this a bug or can this "feature" be turned off?
While from my experience this bug seems to be resolved since OS X Mountain Lion, it might be that your Launch Pad database already was corrupted when upgrading, and upgrading does not necessarily repair it.
You could try rebuilding your database and see if that resolves your problem, but please note that this will remove your current ordering and grouping of apps in Launchpad completely.
To remove/rebuild the database, you can execute this command from the Terminal:
rm ~/Library/Application\ Support/Dock/*.db; killall Dock