For instance I downloaded Java and it's listed as a device. After i ran the installation, i clicked on the little eject-like looking button beside its name under Devices and the downloaded installation file disappeared from the Desktop. Where did it go?
Many applications you download for your Mac come in a file which is similar to a virtual CD/DVD. Those are
.dmg files or Disk Images. Whenever you double-click those you can imagine they are kept in the memory of your computer. Even if you delete the
.dmg the content is still there as long as the image is not ejected (it is currently "mounted").
When you eject ("unmount") them, the content in memory is no longer there. However, the
.dmg is at the same location where you opened/double-clicked it. Most likely for newly downloaded things, this is in
~/Downloads which is the Downloads folder in your user's home directory.