I know how to add explicit tags to individual tracks, but I'd like to know how to add the tag to albums/the album title as well. I'm using windows if that affects anything.
It seems that this is a tag within the file. If you are using Windows, follow this guide here. Per Mod request, I've put the instructions below.
Begin by downloading MP3Tag, whether the most recent version or a portable version.
Open iTunes and find a song that you’d like to mark with an appropriate tag. Click the song once, so that it is highlighted, and then press “Ctrl + I.” The next window will have a button marked “File.” Under it you can see if the track is an AAC or MP3 (iTunes calls MP3 ‘MPEG audio’). Unfortunately, adding these tags to an MP3 file is not possible, so any track you wish to mark must be AAC.
Right-click the track and choose to “Create AAC Version.” In our case, we’ll use a copy of Hard-Fi’s “Suburban Knights.” iTunes will play a tone when the AAC version has been created, and it will appear alongside the original version in your library. Use “Ctrl + I” again to double-check which file is the AAC version if you are unsure.
Open MP3Tag at this point, and right click the song in iTunes to see a list of actions. Choose “Show in Windows Explorer,” then drag the song into MP3Tag.
Right-click the entry within MP3Tag and select the “Extended Tags” option. The tag is not part of MP3Tag by default, so you’ll have to add it via the first of the three buttons.
After clicking the first of the three buttons, another window will appear asking for two properties: the tag name and the tag value.
Call the tag “ITUNESADVISORY” (without the quote marks), and choose the correct value. Entering “0” will mean that no icon appears, just as before. Entering “1” will give the song the “Explicit” tag, and entering “2” will give it the “Clean” tag. Leaving the value blank will give no tag.
After modifying the details for the song, press “Ctrl + S” in MP3Tag to get a window informing you that the changes have been saved.
Remove the songs you’re changing from iTunes. As iTunes cannot register changes to the song, you’ll have to remove and re-add them. Make sure you choose to “Keep Files.”
Drag the modified tracks back into iTunes, and you’ll see the corresponding tag. Sync them with an iOS device and the same remains true.
Do you use Apple Music? If so, there won’t be a way to do so, IIRC.