A non-linear Compressor, or non-linear compression response rather is where the amount of gain reduction decreases at higher input levels, and/or it simply means the application of a soft knee or a curve in the compression.
Linear means there’s either no compression with no gain reduction, or a hard knee an abrupt gain reduction.