Backport of "audioaggregator: Don't overwrite already written samples" into 1.18 - blocker issue for investigation

Apparently !1185 (merged) might potentially introduce some unexpected side effects (increased latency), needs investigation to see if we should back it out or keep it for 1.18.5