The issue how to quantify quality is an old one. In antediluvian times it used to be a common place that you cannot e.g. map quality on straight lines, because quality is a multi-dimensional category whereas quantity is one-dimensional.
In business such philosophical qualms are generally perceived as academic and irrelevant: In HR and marketing for instance, where the “quality” of employees, teams, processes, campaigns, ads, tags, key words and channels evidently represent the foundation of quantifiable outputs controllers push the respective departments to translate quality into quantity to justify the incurred expenditures. With perfect legitimacy.
Interestingly, the reverse translation from quantity to quality is rarely regarded as being problematic although the exercise is by and large the same. Big figures are widely regarded as being either good (sales) or bad (cost), and small figures conversely. Yet, when it comes to relate quantifiable outputs to quantifiable inputs things are never easy, obvious or transparent. KPIs by themselves usually do not tell the whole story but need to be complemented by other KPIs because quality is, after all multidimensional. All interesting questions in business go back to the field of quality: Which KPIs need to be gathered and which need to be put into context to gain an adequate impression of a particular process?
This topic is not at all academic but very relevant, not least because due to the incommensurable complexity of both fields, quantity on the one hand and quality on the other it is very easy for business units to reject responsibility for poor outcomes, point the finger at other units or blame dire market environments or temporary circumstances for poor results.
The only way how to arrive at sufficiently valid and company-wide accepted judgements on quality is by i) accepting that KPIs can only give an approximate picture of the quality of a company’s input and ii) accepting that the validity of a set of KPIs as “evidence” for a higher or lower degree of quality of processes, employees, teams, measures or whatever is based on the acceptance of the respective KPIs by all stakeholders. If e.g. we “talk” scrum, project owners, team members, masters and ideally also users, managers and customers commit themselves to the validity of a respective set of KPIs by COLLECTIVELY DEFINING THEM until they COLLECTIVELY agree to change that set. Such collective and repetitive decision-making is time-consuming, difficult to organize and painful at times. But it is certainly worth the pain. Because the alternative is conflict, lack of direction, and uncertainty.