No, it doesn't affect your performance negatively. Rather it is more likely that having a higher wattage charger could hurt performance (ever so slightly).
The reason it doesn't affect your performance negatively is that if your computer for shorter periods of time uses more than energy than your charger can supply - it simply drains from the battery. Ofcourse if you have sustained loads for a very long period of time (i.e. hours) of extremely high load, then you might simply run out of battery, which you could ofcourse say is a total loss of performance.
However, the difference between 85W and what your MacBook Pro can maximally charge is so small, that in practice this is not something you'll run into - unless you have a really edge case of performance hungry work.
If you connect the 140W adapter to the computer and the battery is rather low in charger, then it can fast charge the battery very quickly. This will generate some heat, and that could negatively affect performance for sudden, very demanding tasks. Ofcourse that only happens in that short period of time where fast charging happens.