.png)
How to Assess CIO & CTO Candidates When You're Not Technical
A board or senior leadership team does not need to know as much about technology as the CIO or CTO it is hiring. However, it needs to distinguish a genuinely strong technology leader from someone who interviews extremely well.
Senior technology executives are often accomplished communicators. They may have worked for impressive organisations, led programmes with significant budgets and become fluent in the language of transformation, cloud, AI, architecture and operating models. None of that tells you about the impact they had, how good their judgement was or how well they will perform in a different organisation.
Assessing CIO and CTO candidates properly means getting behind the CV and how they present themselves. You want to understand what the candidate inherited, the decisions they made, what happened as a result and how they behaved when circumstances were difficult.
Start with the job they are actually being hired to do
A CIO being hired to stabilise an expensive, unreliable technology estate should not be assessed the same way as one being hired to reshape investment around a growth strategy.
The same applies to CTOs. Someone inheriting a mature engineering organisation faces a very different challenge from an executive expected to rebuild technical capability, modernise architecture or change the relationship between product and technology.
Before interviews begin, the board should be clear about what should be different 12–18 months after the appointment and which important decisions will sit with the new executive. Without that clarity, the discussion easily becomes a comparison between different versions of a “good CIO” or “good CTO”.
One candidate has stronger sector experience. Another appears more technical. Another is particularly convincing in front of the board.
You want to know whether their experience prepares them for the particular problems they will inherit.
Find out what they personally changed
A candidate may have worked in an organisation that completed a £500 million transformation, migrated extensively to cloud, improved resilience or rebuilt a major technology platform. That tells you something about the environment they have experienced. It does not tell you what they did. Large programmes have many contributors.
When assessing senior executives, you need to establish what somebody inherited, where their responsibilities started and stopped, which decisions they made personally and what changed because of their leadership.
That context can change how a track record looks. A candidate who inherited a well-funded programme, a strong leadership team and an agreed strategy may have executed it extremely well. Another may have arrived with weak capability, little executive confidence and no coherent technology direction, and had to create the conditions for change before the programme could begin. Both can be strong appointments, but the experience they bring is different.
Test technical judgement through decisions
A non-technical board can test technology judgement without resorting to technical trivia and jargon. Ask about significant decisions the candidate has already made.
For a CIO, that might involve a major platform investment, legacy modernisation, a difficult supplier decision or the decision to stop a programme that was no longer delivering enough value. For a CTO, it might involve architecture, technical debt, build versus buy or a significant change to engineering capability.
Then stay with the decision.
What was the problem? Which options were available? Who disagreed? What trade-offs did the candidate accept? What worried them? What evidence changed their mind? What happened afterwards?
Those questions tell you much more about judgement than asking someone to explain a current technology trend.
A strong technology executive should also be able to explain difficult decisions to people who do not share their technical background. That does not mean making everything sound simple. Some decisions genuinely are complex. The skill lies in explaining enough of the reasoning, risk and consequences for the board to make an informed judgement.
Be wary of candidates who use complexity to establish credibility, but fluency can mislead in the other direction too: a polished explanation does not prove that the underlying decision was good.
Be sceptical of the word “transformation”
Transformation is one of the least informative words on an executive CV unless you establish what actually changed. It might mean replacing core systems and materially reducing operational risk, restructuring the technology organisation and improving engineering performance, or bringing an existing programme back under control. The work behind the label is what needs examining.
If a candidate says they improved technology performance, ask how they measured it. If they reduced cost, understand where the savings came from and what happened to service. If they improved resilience, ask what changed operationally and how the organisation knew the risk had reduced.
The same scrutiny should apply to claims about cloud, AI, digital platforms or modern architecture.
The technology is rarely the outcome the board is buying. Better evidence is whether the organisation became more resilient, less costly to run, faster to change, better able to serve customers, or better able to make decisions.
Find out how they operate when other executives disagree
Much of a CIO or CTO’s work depends on people they do not control. Technology investment may depend on the CFO. Transformation can require operational teams to change established processes. Responsibility for data may sit elsewhere. Cyber, risk and AI increasingly cut across several executives and board committees.
A strong track record inside the technology function is therefore only part of the assessment. Look for examples of serious disagreement: something the executive believed needed to change when another senior leader had a different view. Who resisted? Why? Was their concern reasonable? How did the candidate handle it?
This is usually more revealing than asking whether someone is a good stakeholder manager.
Some executives depend heavily on positional authority. Others build consensus so carefully that they avoid necessary conflict. The strongest CIOs and CTOs tend to know when to persuade, when to compromise and when an issue is important enough to escalate.
The board will eventually be on the receiving end of that judgement. It needs an executive who can explain when an investment assumption is unrealistic, when a programme is failing or when the organisation is carrying more technology risk than senior colleagues realise.
Ask about the decisions they got wrong
Executive interviews naturally favour success stories, which can distort the picture in technology, where many important decisions are made with incomplete information. Architecture evolves, suppliers underperform, acquisitions reveal unexpected problems and programmes encounter constraints. A strategy that looked sensible when it was agreed may no longer fit as the business changes.
Someone who has held significant responsibility for long enough should have examples where the original decision did not work as expected.
You want to understand when they realised that, how quickly they responded and whether the experience changed the way they now approach similar decisions.
The strongest answer is rarely a perfectly packaged ‘failure story’ that conveniently ends as another success. You are looking for evidence that the candidate's judgement has developed through experience.
Keep enough structure to compare candidates properly
Executive interviews should have room to develop, so you should not turn a CIO or CTO assessment into a rigid questionnaire. Completely unstructured interviews create their own problems, however. CIPD guidance recommends pre-agreed questions and scoring criteria to make candidates easier to compare and reduce bias, as well as independent scoring before the panel discusses candidates. UK Government guidance similarly supports fair and structured interviews.
At executive level, the principle matters more than the mechanics. Agree beforehand which aspects of the role will determine success, test every candidate properly in those areas and capture the evidence before the panel begins comparing personalities. This is particularly useful when the finalists bring very different strengths.
One may be the stronger technologist. Another may have led much more difficult organisational change. One may know the sector intimately while another brings experience from an adjacent environment.
The assessment should help the board understand which of those differences actually matter for this appointment.
Separate what you know from what you are assuming
Boards often reach the final stages knowing a great deal about a candidate while still carrying a few important assumptions.
A person has worked in an organisation known for excellent engineering, so the panel assumes they are an exceptional engineering leader. Someone has held a senior role in a large listed company, so board-level capability is taken for granted. Another candidate happened to lead a function during a period of rapid growth and is credited with outcomes that have never been examined closely.
It is useful to separate three things: what has been evidenced, what appears likely and what remains unresolved.
The unresolved areas should then shape the rest of the process. Another interview may be enough for some; others may need technical scrutiny or a targeted reference conversation.
It also improves the final board discussion. Instead of debating whether one candidate simply “felt stronger”, the panel can see where the evidence is convincing and where the appointment still requires a degree of judgement.
Every executive appointment involves judgement. The board should know where it is taking that judgement rather than allowing assumptions to disappear into the overall impression of the candidate.
Bring in technical expertise where it genuinely helps
A non-technical board should not try to validate every specialist claim itself, and some appointments require deeper technical examination. A CTO taking responsibility for a large engineering organisation may need their architectural and engineering judgement challenged by someone with real depth in those areas. A CIO inheriting a major ERP programme, complex infrastructure or significant cyber risk may need specialist scrutiny around particular parts of the role.
That expertise can come from an existing technology leader, an adviser or another credible technical expert involved in the process. It should strengthen the board's evidence rather than transfer the appointment decision to the most technical person in the room.
An exceptional technologist can still be the wrong executive. A highly persuasive board communicator can still lack the technical credibility required to lead the function.
The appointment depends on both.
Use references to investigate the remaining questions
References are most useful when they investigate something rather than confirm a decision already made.
If a candidate says they turned around a difficult programme, speak to someone who saw them do it. If their ability to challenge the board matters, a former CEO or chair can tell you much more than someone who reported to them. If you're unsure how they behaved when a transformation became difficult, find someone who experienced that period alongside them.
The reference can then return to the evidence gathered during assessment.
What did the candidate inherit? Which decisions were genuinely theirs? How did they behave when things deteriorated? How much of the result would the referee attribute to their leadership?
That is considerably more valuable than asking a generic set of reference questions once everybody has already decided who they want to appoint.
What should the board know before it makes the appointment?
By the end of the process, the board should understand much more than where the candidates have worked and how impressively they interviewed. It should know what they inherited, what they personally changed, the decisions that reveal their technology judgement, how they deal with disagreement, how they respond when things go wrong and which parts of their track record remain less certain.
A non-technical board does not need to replicate the candidate's technical expertise. It needs enough evidence to judge whether that person's experience, decisions and leadership are right for the organisation they are about to join.
Related analysis and executive search
For organisations appointing leaders responsible for enterprise technology, transformation, investment and technology performance.
For organisations appointing leaders responsible for engineering, architecture, platforms and technical capability.
How do we know whether we need a CIO or a CTO?
A guide to defining the leadership requirement before deciding which role to recruit.
Our previous technology executive search failed. What should we change before going back to market?
How to diagnose what went wrong before beginning another search.
Assessing a critical technology appointment?
Lucent Search assesses candidates by looking closely at what they have personally delivered, the decisions they have made, the judgement behind those decisions and the conditions in which they have succeeded.
Rebecca can also support client interview panels where additional technology-search and assessment experience would strengthen the process.
Comments