Hiring data analysts should be straightforward.
You need someone who can:
Take messy data, understand a business question, figure out what matters and finally – communicate it clearly.
That’s it.
And yet somehow, the process looks like this:
Whiteboard SQL (because apparently analysts code with dry erase markers now)
Brain teasers (because nothing says “business impact” like puzzles)
“Take-home assignments” that reward polish over thinking
At some point, the whole thing starts to feel like a bad episode of Mystery Science Theater 3000 – except instead of watching a bad process from the outside, we’re all somehow acting in it.
The Core Problem
We don’t know how to measure analytical thinking.
So we measure proxies:
Syntax, speed (finish this exam in 3 hours), presentation (is that what your analyst is doing daily?), and confidence.
The problem with proxies is simple – They drift.
Eventually, they stop representing the thing they were supposed to measure.
Or, to borrow from Hamlet – The method is there… but the madness is doing the hiring.
Enter AI (Making It Worse)
Now layer AI on top.
Suddenly – anyone can generate clean SQL (and pretty good with the help of the ol’ ctrl+c ctrl+v), anyone can build a polished dashboard (don’t know what streamlit is? it doesn’t matter anymore), anyone can write a convincing explanation (even without reading it)
So the already weak signals?
They’re now completely unreliable.
What’s Missing
We don’t evaluate:
1. How someone approaches ambiguity
2. what they choose to explore
3. What they ignore
4.how they validate themselves
In other words – we don’t evaluate how they think.
Where This Is Going
There’s a growing shift toward evaluating analysts through actual work.
Not exercises.
Not puzzles.
Work.
One example (disclosure: I built it) is XP Lab – a platform focused on evaluating analytical thinking through messy, real-world scenarios.
Because right now?
We’re basically running hiring like a chaotic side quest in Dungeons & Dragons, hoping the dice rolls land on “good hire.”
And hoping, well, that’s a lousy way to do business.
