The title pretty much says it all.
I cannot find any explanation of the difference between the two buttons.
"Remove" just removes the entry (or entries) you've selected in the list. Notice it's disabled if you haven't selected anything.
"Remove All", as the name implies, removes all entries, regardless of the entries you have selected. It's essentially the same as selecting all entries (via Cmd+A) and pressing "Remove".