Challenges with ESG data

The lack of consistent standards in relation in ESG data has resulted in a wide range of information being produced using differing approaches. Environmental data can be difficult to gather because it can be scattered across multiple departments and systems. Social data can be sensitive and confidential as well as qualitative and subjective, making it difficult to quantify and measure.

There is a lack of a single regulatory framework, although various guidelines exist as has been discussed earlier on this course (for example in Step 1.10 in Week 1). There is a lack of auditing, which limits the reliability of disclosure reports. There is no universal system to verify reported data, and public reports tend to highlight positive contributions to ESG and underplay or omit less flattering reporting. There is no regulation governing the assessment of sustainability requirements and performance, and every ESG rating agency has its own evolving methodology.

Consider greenhouse gas emissions – there are different methodologies that can be used including the actual value method and the default value method. Emission factors have been created that convert fuel usage and an activity into an estimated CO2 output. The factors will vary depending on type of fuel used and the efficiency of the process. CO2 is only one of relevant gases – methane, nitrous oxide, and trace gases such as the group of ‘F-gases’ – could/should/may be reported on.

Unlike credit rating data sources ESG data does not follow regulated methodologies. Companies will use internal data as well as external sources to gather the underlying raw data for the same ESG issues. Some providers and users make their methodologies transparent, but many do not. As a result, and given the differences in ESG analyses and ratings, stakeholders would be better-off considering them as ESG data “opinions”.

Key corporate challenges

Differences in data can include the definitions of:

• materiality

• normalisation techniques

• aggregation and weighting

• survivorship bias and missing data

• use of standards and metrics

• creation of benchmarking and peer groups

• sources and timing of data collection

• conduct versus product-based scoring methodologies.

To ensure credibility and integrity, ESG data needs to follow the same data management processes as other material corporate data (especially those in place for financial performance). This includes validation and data quality checks.

The quality of an ESG assessment will depend on the comparability of its source data and the methods used to analyse this data. As ESG reporting standards evolve, organisations will increasingly need a single data management platform that integrates data from multiple points and uses Artificial Intelligence (AI) and Machine Learning (ML) principles to enhance its robustness. A key priority is to encourage a self-serve model, where data collection from suppliers is automated and data quality rules are applied from the outset.

In addition, data from countries with diverging regulatory standards, can be tainted by gaps or bias. Companies with overseas operations may have data that varies in quality significantly from one location to another – perhaps due to the level of technology, skills or resources available.

Some of the data required (such as greenhouse gas emissions) may never have been collected before whilst others may not have robust processes to support data quality and integrity. In both cases the collection of accurate data may take several years before it can be considered sufficiently reliable.

Comparisons across different ESG data providers

In addition to the challenges faced within a company in aggregating its own data, the same issues are faced by ESG data providers when comparing different companies. For instance, if a company has a best practice diversity & inclusion, employee health and safety, or other type of ESG policy, but does not make the policy publicly available, certain ESG ratings providers will give that company a low rating (or the industry average) for that ESG issue. At the other extreme, companies that publish well written, comprehensive policies will see themselves receive higher ratings even though these policies may not be followed at the individual companies.

Some ESG data providers can also be sector-neutral, meaning that companies even in sectors with significant ESG risks (such as the oil and gas sector) can still score high on ESG metrics. A high ESG rating therefore does not necessarily mean a company is more sustainable or takes “better care of the environment or society”. Hence the peer group or benchmark that is used to determine the ESG score or metric becomes of paramount importance.

There are also size and geography biases. The materiality of an ESG issue can vary depending on the country or the region of the firm. For instance, while issues such as “worker health and safety” are very important in mature markets, developing markets may favour job creation over health and safety, so as to reduce poverty levels. This may create biases in the ratings of firms in developing countries (unless normalised for regional differences), since most ESG rating agencies tend to be from Western countries.

Stakeholders understand that given the inconsistencies between ESG data providers, they need to apply critical judgement and perform an in-depth analysis when integrating ESG ratings into their investment decisions. A crucial first step is the due diligence of any ESG data provider used. Assessors will then often overlay their internal knowledge (from internal ESG specialists and/or from engagement with the company), and insights from ESG technical experts (such as engineers, scientists, and researchers) onto the raw data to create their own proprietary ESG scores.