I tried to put two 4GB sticks (OWC) into a MacBook Pro early 2011 17 inch. Either of the 4GB sticks works with either of the stock 2GB sticks but both the 4GB sticks do not work. It causes kernel panics. Firmware is already at the latest version (MBP81.0047.B27). I am well aware of fitment process. Officially it can support 8 gb max.
Macs are quite a bit more tolerant of what RAM they can take and handle well. If RAM can be upgraded at all. A 2011 MBP has a wide range of options. Using Apple approved RAM modules (at a small premium) with the same specs as a non-approved module of the same brand usually only gets you small advantages: they pass the hardware test at the Genius bar without issues and are practically guaranteed to work in a Mac. But that doesn't mean the non-approved modules do not work. It is true that the module has to meet certain specs. But as long as it is DDR3 SO-DIMM with the correct voltage and not too slow the MBP 8,2 from 2011 should be able to handle it.
That means two DDR3-1600 sticks each at 8GB is way out of Apple's official spec. Yet the machine works fine with them. The 1333 option is playing it safe and slower. Apple's advice: is doing just that. It doesn't even matter whther they are DDR3 or DDR-3L since the L variant is rated at 1,35V but required by spec to also run at 1,5V like the MBP 2011 wants it to. But choosing the L variant doesn't give you any energy savings in this case. To summarise: the machine in question can run at least up to 16GB RAM with 1,5V at at least up to 16000MHz just fine.
And the OP already confirmed that the sticks are working in some configurations but not in others.
Concerning the OP question there seems to be a hardware fault: It might be the logic board (more specific the ram slots being even pickier). Or it might be that the two 4GB sticks have issues. "Mismatched" modules it says in the title. Absent the knowledge of how this old problem went forth the advice had to be:
Most likely at least one of the 4GB modules is somehow flawed. To hunt it down and exclude other possibilities:
- reseat the modules: test one module at a time; then both, swapping modules from slot to slot (this not only should pin down any problems with the individual modules but also might clean the slots a bit. Cleaning the slots per se might be an option as well.)
- stress test the modules separately at first, then together in the combinations noted above
- get the logic board tested, RAM slot failure is not that uncommon
- buy modules of the the capacity, same specs both at the same time together (this is belt+suspenders advice; mixing modules should work – and did as the OP demonstrated)
Macs are notoriously picky about memory. The specs of your memory must match exactly what the Mac is expecting.
Your Mac is expecting:
DDR3 PC3-10600 • CL=9 • Unbuffered • NON-ECC • DDR3-1333 • 1.35V
If yours is not exactly this, you will have problems.
I believe your Macbook can take up to 16GB, so 8GB should be fine.