What did CDSP plugin's input meters show when the Kenwood was at -5 clicks of max volume? Adjusting any gains in the plugin or Dirac DLCT does not effect CDPS's input ADC level. Only source head-unit or 2/4V DIPs can change that level. So maybe you were overloading after the AD conversion? There are lots of gain adjustments after that.Yes the ADC going in minidsp.
You can hear it overload when the signal does not clip (on a scope) but audibly started to saturate when the kenwood was about 5clicks to max volume.
In the Dirac plugin there’s a master output and individual output sliders for each speaker. There all a input gain, I used the master slider and turned it down to -5
And problem was gone.