AI readiness assessment tools: how a mid-sized company can choose one

Overview

A mid-sized company planning to introduce AI into their workflow needs to examine its systems, data and employees before committing funds. Which AI readiness assessment tools can inform that decision? This article compares six assessment approaches and draws on three research papers to explain what AI readiness means, how it can be measured and what to consider when choosing an assessment. It also outlines questions to ask providers before your company introduces AI.

Research papers help explain what readiness means and how it is studied. Provider pages describe the tools available, including their scope and scoring methods. Our recommendations for choosing an assessment draw on these sources and our professional judgement. They have not been tested as a selection method.

What does AI readiness mean?

AI readiness concerns whether an organisation is prepared to introduce and use artificial intelligence into their day-to-day operations. Researchers approach this question from different perspectives, examining both the conditions needed for this technological implementation and people's readiness to make the change.

Joehnk, Weissert & Wyrtki examine whether an organisation has the assets, capabilities and commitment needed for its planned use of AI. Their study, published in Business & Information Systems Engineering, draws on interviews with 25 AI experts. ^[1]

The researchers identified 18 proposed readiness factors and 58 illustrative indicators across five categories: strategic alignment, resources, knowledge, culture and data.

To understand how these elements work together, consider data accessibility. This is a readiness factor that concerns how easily people can access the data they need. The authors identify access management as one way to support it. ^[1]

An indicator makes the factor more specific by identifying what should be examined. An assessment item then turns that indicator into a question or statement that someone can answer. For example, an item might ask: Does access management give employees authorised access to the data sources they need?

This is more specific than simply asking whether data is accessible. It directs attention to the arrangements being assessed and makes the response easier to interpret.

A different perspective comes from organisational psychology. In a 2009 paper published in Implementation Science, Bryan Weiner proposed a theory of organisational readiness for change based on two concepts:

• Change commitment: the shared determination of an organisation's members to implement a particular change.
• Change efficacy: their shared confidence in their collective ability to implement it.

Weiner also proposes that organisational resources, structures and other conditions shape people's confidence in their ability to make the change. ^[2]

These perspectives address different aspects of readiness. Examining data accessibility helps establish whether a condition for the planned use is in place. Asking employees about their commitment and confidence helps establish how prepared they believe they are to introduce AI.

Both can inform an assessment, but they measure different things. Keeping these two kinds of evidence separate helps organisations understand whether a concern relates to their operating conditions, people's perceptions or both.

What the available assessments cover

AI readiness assessments come in different forms. Some are online self-assessments focused on a particular technology platform. Others provide templates or scoring guides that organisations can complete internally. Consulting engagements involve external specialists reviewing areas such as systems, data and organisational processes.

Not every published readiness framework is an assessment service. For example, Cisco's AI Readiness Index is a survey-based study drawing on responses from senior business and technology leaders. ^[6] Teamvoy, meanwhile, describes its own consulting assessment in a comparison article that ranks it first. ^[9]

The following table summarises the published scope and scoring information.

Published provider approaches to examining AI readiness, from the six sources reviewed.

Providera	Type	What it covers	Scoring, time and costb
	Online self-assessment	Azure AI production readiness, including organisational and application questions.	Approximately 45 minutes, with personalised guidance. Scoring formula and price not stated.
	Survey-based study	Strategy, infrastructure, data, governance, talent and culture.	Indicators scored according to reported deployment levels, with weighted categories and four readiness groups.
	Scoring guide	Data, strategy, governance, people, technology and use case value.	Ratings from 1 to 5. Weights: data foundations 25%, strategy and leadership 20%, governance and compliance 20%, people and culture 15%, technology 10%, use case value 10%. Five maturity levels. No stated external fee for using the guide; internal staff time is required.
	Consulting engagement	Strategy, data, organisation, infrastructure, governance and model management.	Four-week engagement with a roadmap and implementation plan. Scoring formula and fee not stated.
	Consulting engagement	Strategy, data, infrastructure, governance, culture, codebase health and decision-readiness.	Fixed fee determined after a technical scoping call. Scoring formula not stated.
	Template	Leadership commitment, strategy, policy, people readiness, infrastructure readiness, monitoring and policy implementation.	Team-agreed ratings using a Likert scale, from strongly disagree to strongly agree. Responses can be converted into numbers. Includes six-month goals. Document retrieved without charge.

These differences matter because an assessment designed for one purpose may leave important questions unanswered for another. An online platform assessment, for example, may help examine technical readiness without fully addressing employees' ability or willingness to use AI in their daily work.

The first consideration is therefore whether the assessment covers the areas relevant to your planned use of AI.

How readiness scores are built

Assessment scores depend on how answers are collected, combined and interpreted. Understanding these rules is important because two providers can produce similar-looking scores while measuring different aspects of readiness.

Scoring approaches can involve several elements:

• Scoring rules determine how individual responses contribute to the result.
• Category weights determine how much influence different areas have on the overall score.
• Readiness or maturity levels classify scores into predefined bands.
• Benchmark groups provide a reference for comparing results with those of other organisations.

OvalEdge provides an example of weighted scoring. Its model gives data foundations a weight of 25%, compared with 10% for technology. This means data foundations contribute two and a half times as much to the total when the category ratings are otherwise equal.

The guide instructs users to multiply each category rating by its weight, add the weighted results and multiply the total by 20. With category ratings ranging from 1 to 5, the resulting score ranges from 20 to 100. ^[7]

These weights reflect the relative importance assigned to each category within OvalEdge's model. They do not, by themselves, establish that the same priorities are appropriate for every organisation or AI application.

Cisco also uses weighted categories but does not publish the individual weight values on its methodology page. Its readiness groups range from Laggards, with scores from 0 to 30, to Pacesetters, with scores of 86 or above. ^[6]

In contrast, Joehnk and colleagues propose readiness factors and indicators without specifying how they should be prioritised or weighted. ^[1]

A score of 80 from one assessment is therefore not necessarily equivalent to 80 from another. Even when both use a 100-point scale, they may differ in what they measure, how categories are weighted and what their score boundaries represent.

What research says about measurement

An assessment's scoring method explains how a result is calculated. It does not necessarily establish whether that result is reliable or suitable for the decision an organisation intends to make.

Two important measurement concepts help clarify this distinction:

• Reliability concerns the consistency and precision of assessment scores.
• Validity concerns the evidence supporting how scores are interpreted and used.

The Standards for Educational and Psychological Testing, developed by three American professional associations, emphasise the need for evidence supporting intended score interpretations and uses. They also call for appropriate evidence of reliability and precision. ^[4]

This distinction is particularly important for AI readiness because organisations may use assessment results to inform significant investments or changes to employees' work.

In his 2009 paper, Weiner referred to two earlier reviews that found limited reliability or validity evidence for most publicly available measures of organisational readiness for change. His observation concerned organisational change more broadly, rather than the specific AI assessments compared here. ^[2]

In our view, consistency alone is not enough to demonstrate that a score supports a particular decision.

Consider a company planning to introduce an AI assistant that helps customer-service employees find internal guidance. Managers might consistently report that the necessary documents are accessible. However, those ratings do not establish whether employees can recognise incorrect AI-generated answers or check them against reliable sources.

The company would need additional evidence about how employees perform those tasks before making an adoption decision. Our article on AI evaluations explores a related challenge: interpreting assessment results beyond the conditions in which they were studied.

Further insight comes from a 2020 systematic review and content analysis by Isomi Miake-Lye and colleagues, published in BMC Health Services Research. The researchers examined organisational readiness assessments relevant to health-care delivery.

Implementation climate refers to how an organisation supports and expects its members to use a particular intervention.

The review also found that assessments were typically tailored to specific interventions or contexts. ^[3] This aligns with Weiner's emphasis on readiness for a particular change rather than change in general. ^[2]

For AI adoption, the implication is that an assessment should reflect the technology being introduced, the work it will affect and the people involved. Broad measures of organisational readiness may be informative, but they may need to be supplemented with evidence specific to the proposed application.

Choosing for a mid-sized company

The starting point is the decision your organisation needs to make. Before comparing providers, define the planned use of AI, which parts of the business it will affect and when a decision is required.

In our test design work, we begin by defining what an assessment score is intended to mean before deciding how it should be calculated. The same principle applies here: the purpose of the assessment should guide what is measured and how the results will be used.

Once the scope is clear, identify who has the knowledge to provide information about each area. This may include managers, employees who will use the technology, and specialists responsible for data, infrastructure and technical operations.

For example, imagine a mid-sized company considering an AI assistant for internal customer-service guidance.

The company would need to examine whether relevant documents are accessible, whether its technical systems can support the assistant and how employees will check the responses it produces.

Some of this information may already be available internally. Document inventories, system records and employees' knowledge of customer-service processes could support an initial assessment using a template or scoring guide. Other areas may require external specialists, particularly where the company lacks the relevant technical expertise.

Employee readiness deserves separate attention.

What people report about their commitment and confidence is different from what they can demonstrate in their work. Both forms of evidence may be useful, but they should not be treated as interchangeable.

In our surveys and people analytics work, we define whose views are being measured and which specific change the questions concern.

For AI readiness, this means asking employees about the particular technology and changes they are expected to adopt, rather than AI in general.

What people report about their commitment and confidence is different from what they can demonstrate in their work.

Weiner's theory treats organisational readiness as a shared property. Individual responses therefore need to show sufficient agreement before they can be interpreted as a collective view. ^[2] Differences between employees' accounts are also important and should be recorded rather than hidden within an average.

Employees should understand why information is being collected, who will have access to their responses and whether findings will be reported at individual or group level.

Where the results identify a need for training, define the specific tasks employees need to learn and how their performance will be examined afterwards.

Questions to put to a provider

A provider should be able to explain what its assessment measures, how results are calculated and what evidence supports the conclusions presented in its reports.

Start by requesting a sample report and a description of the proposed scope. Select one recommendation and trace it back to the questions asked, the people who answered and the scoring method used.

This can help establish whether the recommendation follows reasonably from the information collected.

Where the report compares your organisation with others, ask how the benchmark group was selected and whether those organisations are sufficiently similar to yours.

You should also request evidence supporting the intended interpretation and use of the scores. Relevant studies should explain who participated, what was examined and how much uncertainty surrounds the findings.

Finally, ask how score boundaries were set and whether different results would lead to meaningfully different recommendations.

These questions apply to our own assessments as well.

At The Inner View, our AI Readiness and Workforce Assessment is a 25-minute inventory that provides individual and team-level reporting.

It examines three areas: reported capability (what people say they can do today), disposition (how they say they respond to change) and context (what they report their working environment allows).

Our Human-AI Capability Model extends this work to employees, teams and leaders, examining skill gaps, workflow alignment and areas for change as organisations prepare for AI-enabled work.

As with any assessment, the findings need to be interpreted in relation to their purpose, scope and supporting evidence.

What the evidence does not establish

The sources reviewed provide useful information about AI readiness concepts, assessment scope and scoring approaches. However, none independently compares the six tools or establishes whether their scores predict later success with AI.

The research papers also have limitations:

• Joehnk and colleagues propose readiness factors and illustrative indicators that require further validation. ^[1]
• Weiner presents a theory of organisational readiness for change and identifies the need for further measurement development to test it. ^[2]
• Miake-Lye and colleagues examine the content of readiness assessments without evaluating the validity or robustness of the findings in the studies reviewed. ^[3]

These limitations do not make readiness assessments unhelpful. They clarify what conclusions can reasonably be drawn from the available evidence.

Provider descriptions help organisations understand what an assessment covers. Research offers a basis for examining the reliability and validity evidence for its scores. Neither replaces evidence about how a proposed AI system performs in practice.

For that reason, organisations should consider supplementing readiness assessments with a pilot of the planned application. Observing employees using the technology can reveal practical difficulties, training needs and risks that may not appear in questionnaire responses.

These observations can then inform the decision alongside the readiness findings.

From assessment to an adoption decision

Choosing an AI readiness assessment begins with understanding what your organisation needs to know.

A useful assessment should help you identify which organisational conditions have been examined, whose views have been collected and which questions remain unresolved. Its findings should also be clear enough for decision-makers to understand how conclusions were reached.

Before introducing AI, decide which conditions must be satisfied and which unresolved concerns would justify postponing implementation.

The goal is to make an informed decision based on the evidence available, while recognising where further investigation is needed.

AI and human behaviour

Explore The Inner View's work on AI and human behaviour, including workforce readiness, assessment and employee development.