I've pulled my sms.db file of my iPhone 4S running iOS 5.0.1 (of off live system using scp) and was able to browse the DB as normal using the sqlite client included with OSX Lion. Possibly the current format is somehow incompatible with older versions of the client. The version of the client included with Lion is 3.7.7:
$ sqlite3 sms.db
SQLite version 3.7.7 2011-06-25 16:35:41
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .databases
seq name file
--- --------------- ----------------------------------------------------------
0 main /Users/tyr/sms.db
sqlite> .tables
_SqliteDatabaseProperties message
drafts msg_group
group_member msg_pieces
madrid_attachment settings
madrid_chat templates
sqlite> select * from settings ;
1|version|2.0.0
2|notificationCode|*N#
3|useNotifications|0
4|isDefault|1
5|compatibilityMode|0
6|useKeybCorr|1
7|useKeybCapi|1
8|useConfirmSend|1
9|nameorder|0
10|useShowAlerts|1
11|useStatusIcon|1
12|landscape|0
13|appSerial|0
14|lastMenu|0
sqlite>