Windows 7 user here, though access to a Mac (in case that'd be the only way to do this) wouldn't be impossible.
I know I could download iTunes and sync all the photos, but I have nearly 4000 of them and I'd only consider that a last-resort option. What would be simply perfect: Can I just browse them in Windows Explorer on my iPhone? If not Windows Explorer, some App. If I could even get a list of Albums part of my dreams would come true.