DMG is short for disk image, a format which is commonly used in OS X to install a software.
Another possibility is to use packges (
.pkg) which are installed using the OS X Installer.
DMG Images are like customizable folders that cannot only contain the application itself but also other files. Most often, a custom background is used and you're presented with an option to drag/copy - and thus install - the application in OSX. That's it. You copy by dragging to...
/Applications to install for all users
~/Applications to install for yourself only (~ is the home directory)
As you may guess, the uninstallation is done simply by dragging the application from it's current location in
(~)/Applications to the Trash. There will still remain some related files (settings, cache,...) of that application on your computer, but they generally do not take up much space. You can use third-party software to help you automatically find and remove such data.