The feature has been around for a number of years. It's enabled for every app by default and an opt-out for developers per app version. You can remove an older app version from download through iTunes Connect → My Apps → app → App Store → Pricing & Availability → Last-Compatible Version Settings → Available Versions.
The key information as to why you are seeing the feature not working is mentioned in this page:
Some of your customers may not have a device that's compatible with your latest app version. To allow them to continue using your app, we make the last-compatible version of your app available to them.
If one or more of your app versions have significant usability or legal issues, such as infringement, deselect it below. This isn't recommended for any other reason. Any version you deselect won't be available for redownload from iTunes in the Cloud. You can't deselect your current App Store version.
iTunes Connect Last-Compatible Version Settings, emphasis mine
Downloading a previous version of an app requires the app be downloaded from iTunes in the Cloud, i.e. the app must be in the user's purchase history. This means it does not apply to new downloads, but rather re-downloads.
For a user to download an app which requires a later version of iOS, they must first either:
- purchase/get the app on a compatible device
- purchase/get the app in iTunes
This adds the app to the user's purchase history and is now eligible for last-compatible version download through the feature popup as you mention.