Without further access to the host (e.g. ssh or ARD) and no service running on it (e.g. iTunes Music Sharing/File Sharing) you can't get the computer name based on the IP-address.
The computer name in macOS is not necessarily indentical to the hostname or the Bonjour name! You can simply check this with scutil --get { ComputerName | LocalHostName | HostName }
. ComputerName is the user friendly name, LocalHostname is the Bonjour name - both also visible in System Preferences > Sharing - and HostName is the hostname of the device. The hostname is usually related to or used with DNS, NIS, hosts file or similar methods of name resolution. On a newly installed consumer system no hostname is set. ComputerName and LocalHostName are usually derived from the first user's name and the Mac model (e.g. Admin's iMac | Admins-iMac).
Without a name record for the remote Mac in your hosts file or on a DNS-server in your local or a public network neither of the tools mentioned in the other answers (arp, ping, host) can be used to resolve the hostname.
If a known Bonjour service is running on the remote Mac you can get the Computer and the Bonjour name of an IP in a two-step process by retrieving all hosts offering it with dns-sd -Z _nfs._tcp local
(the example service here is NFS - other service names can be found here) and pinging all listed Bonjour names until you find the proper IP. As an GUI alternative you can use the Bonjour Browser.
In some cases you can get a name similar to the Bonjour name by entering smbutil status <IP>
. The name is the NetBIOS name which is the "all-caps and without allmost all special characters" version of the Computer/Bonjour name.