There are a few resources (1, 2, 3) that describe how to get more information on Recent Servers (and other Recent Items). I'll summarize them here:
You can locate the RecentServers.plist or RecentServers.sfl in ~/Library/Application Support/com.apple.sharedfilelist/
To convert the plist file into a more human-readable format, use plutil:
plutil –p <plist_file> > <output_file_here>
I used json as the file extension, which you can open up in a text editor.
The information this file gives you is limited. There are no dates associated with the data. What I found useful was the URL for Key 16.
I believe this only works to macOS 10.13 and below. It seems that in 10.14 that the Recent Items files are now sfl2 files.