Can I delete a
zip file after I install its content? I mean if this is a little app can I delete the installation file without breaking the installed application?
To answer your question(s), generally speaking, yes, you can delete the container file whether it be a .pkg, .dmg or .zip file. This of course assumes that all content of the container file has been installed. In other words, some .dmg files might contain extras in a separate folder within that are not installed when you drag and drop the app into Applications.
So use common sense when making the decision to delete the container. Obviously if the container contains a single file and you install it, then there is no need to retain it if you don't mind downloading again if for some reason it's needed again.