Jailbreaking an iOS device is the process of taking away the restrictions Apple places on their devices giving the user root access, access to the raw file-system, and the ability to run software not signed by Apple. Most jailbreaks include Cydia, which is the jailbreak equivalent of the App Store.
"Jailbreaking" refers to the process of hacking one's device to control and modify it to an extent beyond what is intended by Apple. For an app to be run on an iPhone or iPad, it must first be digitally "signed" by Apple, so that the device recognizes it as authorized. Jailbreaking allows users to run "unsigned" applications, by removing the restrictions that exist on these devices. Most importantly, it allows for possibilities beyond what apps are normally capable of, such as OS-level tweaks.
Jailbreaking is not as popular as it once was, as many things which have historically required jailbreaking are now possible by way of official features, and there are now ways to digitally sign apps for one's own use by way of a free developer account. However, as there are still many things that can only be done on a jailbroken device, there is still a large and active jailbreak community for the iPhone consisting of many users as well as developers.
Cydia is the jailbreak equivalent of the App Store and serves as a central method of accessing software for jailbroken phones. It is usually installed by default by most of the popular jailbreak methods. Jailbreak software includes a range of games, themes, and all sorts of system tweaks. Jailbreak software has historically given the iPhone features that Apple did not add until later, such as copy and paste, Skype over cellular data, and Wi-Fi sync. Also, many applications that have been rejected by Apple will end up on Cydia, often because Apple does not approve of some aspect or functionality that the application provides to users, e.g. Google Voice applications before Apple changed their mind.
Jailbreak was previously deemed illegal under the DMCA until the Library of Congress issued a specific exemption for the situation. Apple will not support jailbroken devices, however, since it does violate their EULA. This is generally not a problem, since a jailbroken device can almost always be restored to its plain, "vanilla" state.