It OS X impervious to attack? No. Does malware exist that can affect OS X? Yes. Does OS X benefit from malware protection (AntiVirus software). No. Is your Mac safe? Depends.
Understanding the inherent structure of OS X and the protection built into the system would undoubtedly give you a much better opinion of "safe."
OS X operates on a certified UNIX core. The architecture of the operating system is not unlike Linux. Linux is undeniably harder to target for attack since it is far more restrictive in granting privileges.
Files are either given read, write, or execute privileges. Read and write is self-explanatory, while execute is probably a little less clear. The execute command allows a bit of code to do just that: run. Programs need to access to RAM and other components of the OS to function. Images, documents, movies and other similar files should never be given execute privileges. Unfortunately, in Windows, every file is provided such heightened access which means a nefarious individual can (and has) hide malicious code in a jpeg (or as a zip file). Once launched, the program will then be allowed to execute its commands and begin to infect the system.
Naturally, as OS X does not grant such privileges, hackers have to be crafty in their attack vectors.
OS X has now for the past several generations begun to sandbox parts of the OS. This means the code in question is isolated and only given access to specific components it needs to function (rather than having total control). The sandboxing is currently in its early stages, but as it matures, attack vectors will be culled.
Additionally OS X employs a service Apple has dubbed GateKeeper. It not only provides a method of protection in which a user can restrict the installation of programs to identified developers (those that have registered their credentials with Apple), but also validates software against a list of known malware (this list is continually being updated by Apple). It's not as robust as an AntiVirus, but it isn't that far off.
Apple has been shifting its own content to the Mac App Store and many other developers have followed suite. The protection offered by this store is unparalleled. Not only does Apple ensure that all its developers are trustworthy, they also rigorously check the code of each app for any malicious activity and go as far as restricting its code to proper, documented APIs. This may mean that software must adopt a more rigid platform, but it also means that every application found within the Mac App Store is entirely free from any nefarious code (the same applies to the App Store for iOS).
While the Internet is still the wild west when it comes to safety, Apple has begun to build a robust platform with security in mind.
Historically, OS X has been hit with a very low number of attacks. Mac Defender surfaced some years ago under Snow Leopard under the guise of an AntiVirus software (ironically). The Flashback malware was probably the most prolific and relied on the horrendously insecure java framework. Apple released a removal tool, but also took initiative in shutting down the servers that were tied into the malicious code. The most common attack vector used was a fake update for Adobe Flash Player. The Yontoo Trojan infected a user by way of a browser extension. Incidentally, Safari is safe from extension tampering as every developer must register with Apple as part of the Safari Developer Program and is issued their own certificate to which they must use to sign the extension. Apple can easily revoke this certificate, breaking the extension immediately.
While OS X is far from unhackable, the number of threats it has seen over the years is paltry compared to Microsoft Windows. Most malware found on OS X centres around social engineering, where a user is tricked into installing the malicious code. While a variant of Flashback bypassed the requirement for an administrative password, it is exceptionally rare to see malware of such caliber. Additionally, Apple takes a very proactive stance on security, taking away much of the need for a 3rd party AntiVirus.
Based on the above, I recommend anyone against running AntiVirus software. OS X is built on a secure platform and most AntiVirus software offers an extremely degraded user experience, mostly because they are built for Windows and then ported to OS X. It is not uncommon for them to either fail at identifying risks or even cause problems themselves.