CONCLUSION

4

CONCLUSION

OUR EXAMINATION OF THE CURRENT

ASSESSMENT METHODOLOGIES TO EVALUATE

CORPORATE PERFORMANCE ON SOCIALLY

RESPONSIBLE OPERATIONS SHOWS THAT

DIFFERENT BIASES CAN BE INTRODUCED IN

THE EVALUATION.

Addressing the biases in measuring corporate performance on socially responsible operations

Our examination of the current assessment methodologies to evaluate corporate performance on socially responsible operations shows that different biases can be introduced in the evaluation. These biases can determine whose interest is represented in developing the methodology, what the indicators measure, how they are being measured, and the assessments’ ultimate usability of the assessments.

Our review of stakeholder representation in the organizations that develop assessment methodologies found that the decision-making groups in these organizations mostly comprise people with backgrounds in international NGOs, corporations and investors. We noted that the voices of players at the bottom of the supply chains—suppliers, workers and affected communities—are significantly underrepresented. This imbalance of perspectives can lead to a lack in both the quantity and quality of indicators in certain areas of assessment. For example, a common outcome indicator in diversity and employment discrimination criteria is the percentage of employees disaggregated by gender. While this indicator may be meaningful to a corporate manager who would need this information for regulatory compliance, it does not reveal much about the quality of the positions available or employment mobility opportunities for a female worker who seeks to build a career at the company.

Moreover, our research has identified that a significantly large proportion of indicators concern the companies’ internal management practices, while they are lacking in measuring the impact on the rights of affected communities and supply chain operations. This result correlates with the limited proportion of members with community organization and supplier representation background in the makeup of the decision-making bodies of the organizations developing the assessment methodologies.

Previous studies on ESG ratings and social indicators repeatedly reported that the vast majority of indicators focus on measuring companies’ inputs, activities and immediate outputs, rather than the realized outcomes on workers and affected communities. Our research reaffirms the previous findings and points out that most of these indicators are measured with a checkbox approach. These findings show the assessment methodologies’ tendency to measure what is convenient over what is meaningful. Our in-depth review of the indicator criteria related to reporting and disclosure also shows that the companies are measured for simply providing information regarding their policies and procedures, rather than showing their effectiveness on stakeholder impact.

Biases in the assessment methodologies may also influence the usability of the assessment results for various users. The assessment methodologies included in our research showed a varying mix of process-based indicators and topic-based indicators. A company that has a strong external statement on its commitment to human rights may perform well on a process-based methodology, such as the UN Guiding Principles Reporting Framework, that highly values management practices companies implement in addressing human rights issues. This indicator would be useful to an impact investor, because it tells where the company is planning to allocate its resources in the coming years. However, this indicator would be meaningless for a local civil society organization that is attempting to negotiate living wages for the workers employed by one of the company’s suppliers.

Toward inclusive development of social performance indicators

The goal for the next generation of assessment methodologies of socially responsible operations should be to build a strong set of indicators that reflect the realized outcome of corporate activities—and one that can drive value creation and improve well-being for workers and affected communities in global supply chains. The first step forward is to understand the desired outcomes of different corporate activities that are currently being measured. A lesson from the monitoring and evaluation field and “Theory of Change” framework is that we can sequentially map all of a project’s activities to one or more outcomes (USDOL, 2018).19 Aligning the activities to outcomes will help us identify the common sets of outcomes that can be measured and where gaps exist in the current measurement of outcome indicators.

We would then need to add, refine and prioritize these sets of outcome indicators based on what outcomes are meaningful for different stakeholders. Analyzing the dependencies among different outcomes, and how an outcome takes precedence over others in addressing one or more issues, would also be valuable. In this process, incorporating the perspectives of people at the bottom of the supply chains—workers, suppliers and affected communities—is especially important.

For example, an investor concerned with the company’s ESG risks may value risk-based indicators, such as compliance with minimum wage requirements and reported incidents of forced and child labor. However, factory workers may desire forwardlooking outcomes, such as skills development and employment mobility into management roles.

The 2016 Amnesty International report on the artisanal cobalt mining in the Democratic Republic of Congo shows the complex nature of the human rights issues that workers face (Amnesty International, 2016).20 While the primary concern from the international society was child labor, the report suggests loss of wages and income of the children’s parents as a contributing cause, and access to education as the desired outcome for UNICEF and other nongovernmental organizations. An inclusive understanding of what outcomes matter the most will help reduce bias in the types of indicators used.

We recognize that the current checkbox approach is not conducive to creating an outcome indicator that is informative and meaningful for stakeholders. For example, a company may indicate that it has a collective bargaining agreement with the workers in its manufacturing facilities, but the indicator cannot speak to the quality of the agreement. The trade union may be pressured or co-opted to include exploitative clauses in the agreement. We need to include the perspectives of those who are the most affected by these outcomes in discussing the appropriate proxies to measure the desired outcomes, both quantitatively and qualitatively.

19 Resources for developing an OCFT comprehensive monitoring & evaluation plan (CMEP). (2018, February 13). In US Department of Labor. Retrieved from www.dol. gov/sites/dolgov/files/ILAB/CMEP%20Resource%20Document_FINAL%2002132018.pdf. 20 “This is what we die for”: Human rights abuses in the Democratic Republic of Congo power the global trade in cobalt. (2016). In Amnesty International. Retrieved from www.amnesty.org/en/documents/afr62/3183/2016/en.