Limitations of Data Science in an Organisation

In 2017, Gartner published a data and analytics maturity model to assess the progress of the organisation in its data and analytics journey.

Organisations should be doing better with data

As Nick Heudecker, a Research VP at Gartner observed: ‘Most organizations should be doing better with data and analytics, given the potential benefits … Organizations at transformational levels of maturity enjoy increased agility, better integration with partners and suppliers and easier use of advanced predictive and prescriptive forms of analytics. This all translates to competitive advantage and differentiation.’

In a worldwide survey conducted by Gartner, it was found that 91% of the organisation had not reached the transformational stage despite data and analytics being the highest investment category for the CIOs. So, why are organisations struggling?

The development of data science and analytics falls into one of four categories: 1. Mindset; 2. Skill Sets; 3. Toolset; and 4. Dataset.

1. Mindset

As discussed above, data science is a combination of applied mathematics/statistics, technology and business acumen along with design thinking and problem-solving.

Learning about these multidisciplinary fields requires a new mindset. The tools and methodologies in this field continue to develop very rapidly. Thus, a learning and experimentation mindset needs to be adopted to be an effective data scientist.

2. Skillset

Universities have started data science courses; however, such courses are still in their nascent stages. Their structure does not include all the skill sets required for a holistic understanding of data science; rather, such courses continue to focus on developing applied mathematics and technology separately.

There are separate courses on data science, data engineering and business intelligence. No emphasis is placed on developing business acumen, design thinking and problem-solving.

This ideology is also driven by organisations’ demand for specialised skills in each area.

3. Toolset

As discussed above, there are a plethora of tools and technologies available for organisations to enable data science. A major challenge for organisations is to align the right tool with the right kind of problem.

The information technology (IT) department aims for efficiency by limiting the number of tools and technologies that can be installed and accessed within the company firewall.

Companies should have a portfolio of analytics tools

IT departments remain wary of open source technology platforms. However, as the problem space in data science is dynamic, companies need to have a portfolio of analytics and data science tools.

This portfolio should be reviewed consistently to ensure the most effective tools are being used. Without a portfolio, technology will be ineffectively used, which will hinder a company’s growth in adopting data science.

An efficient approach that covers business requirements and data security requirements is needed.

4. Dataset

The issues with data can be classified as either quantity or quality. Many organisations have started to believe that they have enough data to embark on their data science journey.

If some data are not available within the organisation, they invest in third-party data sources to fill those gaps. Companies are also working on various data-sharing agreements and platforms. For example, Data Republic has developed a platform that enables data sharing across organisations.

Data quality challenges

The quality of data is still perceived as a challenge for many organisations. However, the quality of data is critical if data scientists are to have confidence in their business recommendations.

Such recommendations are not used by business executives if they lack trust in the quality of the data. Much focus has been placed on data quality management by organisations, but further work is needed.

A technology-driven approach

Most organisations, especially in Australia, have adopted a technology-driven approach to data science that focuses heavily on toolsets, datasets and processes but ignores the mindset and skillsets required to successfully set up data science in their organisations.

Organisations treat data engineering, data visualisation and data science as different skill sets. Further, in business, business acumen and problem-solving are still handled outside the data team. The data and business teams struggle to communicate due to the lack of a common language.

This often causes friction between the two teams and leads to ineffective data science implementation in organisations.