I have umm… a personal problem: I aspire to be good in what I do.
No, no, it’s not my interviewing technique to suggest that my flaws are my advantages, not me.
It’s a problem because at times it can consume my present in a way which is… less than healthy.
On the other hand, this totality is very good in driving me forward — the problem being that there’s no end to it.
So when I just started off as an analyst, I often asked myself — what makes a good analyst, how does a good analyst behave, talk, teach, learn and analyze.
It would be an exaggeratedly blatant lie to say I have the answer, so maybe I’ll just say I have my idea which is right for me in my present position and I can share it, knowing that it’s just, like, my opinion, which would speak to some and not so much to others?
Well, at any rate that’s how it is.
So, ‘who is a good analyst’ is a topic for another post, but what we’re going to talk about now is analysis quality.
So what is a good analysis?
I tried breaking it down to things I like the most about analyses, things that make me think “Damn, you did good with this one”.
Repeatability
The concept of repeatability is something I borrow directly from the scientific world — peer review and the ability to reproduce an analysis and reach identical or very similar results is critical, for the credibility of both the research and the analyst (which is sadly a real problem — you can’t separate the analyst from the analysis, or vice-versa), as well as for dealing with objections the analysis can generate.
How do we produce repeatable analyses?
I wrote about it a couple of times in the past — disclaimers, disclaimers, disclaimers. Note your presumptions, note everything you took into account, and its always better to run it through the team senior.
Ultimately someone might not agree with your assumptions, but it takes the discussion somewhere much more matter-of-factly and much less “it feels like”…
You omitted data? Filled in empty cells? Normalized on a logarithmic, rather than standard, scale?
It’s all good, just write it down, if possible in an orderly fashion, so it could really be reproduced.
Documentation — every code segment, every query, every call — should be written clearly and legibly and with transparent documentation. The same reason applies — so that when push comes to shove you could always send your code.
Relies on a business rationale
On it’s face, yes, it’s completely obvious.
In practice, I ran into a fair number of analyses whose research made no sense, negligible questions whose ultimate value, even if they drove action, was so negligible that they were practically a waste of time.
So when you plan an analysis, try and think beforehand what impact it could have. How exactly? The thought process is “If the results show A, what could we do with that?”
Accounts for past analyses
If there’s one thing about which I’m absolutely certain that we (analysts, humans) are bad at, it’s knowledge management — organizational memory — we tend to think that analysts who have been in the company or team for a long time can be this organizational memory, but the truth is that we need a knowledge management system (and by “system” I don’t necessarily mean a software, simply an agreed method) which would allow us to not only avoid redundancies, but also reproduce analyses, build on past analyses and, for all that is good and beautiful, not start every. Iteration. From. Zero.
Encompasses the question and the information.
Analyses which examine a month-over-month, or quarter-over-quarter, or year-over-year, change — are bad analyses.
Not only because they are misleading (because how much can two data points contribute to our business understanding?) but also because we can’t do anything with them, and in practice, I saw how when they show a rise everyone happily demonstrate how it was their actions that lead to it, and when there’s a decline, suddenly we’re looking for explanations.
This type of analytic approach is one aimed at pleasing the “client” and it’s a bad approach that hides the truth.
A good analysis would consider a number of angles and layers to make sure there’s no bias, and if there is one, neutralize it by using other perspectives.
Offers possible and attainable courses of action.
A good analysis is either actionable, as the youngsters say, or material.
An actionable analysis is one ending with recommended actions, a material one gives a satisfactory explanation for a particular reason.
Both types are hard to get by, but hey, the harder it is, the better it is (Yep, that’s the title of my sex tape #teamJakePeralta).
Hope that raises some points and it helps you feel better about your analysis!
