Artificial Intelligence Resume Matching vs. Human Cognition
Over the years, I have had the opportunity to evaluate several of the “big name” resume and job matching applications that claim to use artificial intelligence and I can say that the claim that they can find the same resumes that an “experienced recruiter” would choose is both accurate and inaccurate.
Photo: Scott Ingram Photography
From my experience, most AI matching applications can return some well matched resumes based on an example resume or job description. However, some of the results that are returned are definitely NOT good matches, although I can see why they were returned in the results. This is expecially prevalent when searching for job descriptions/resumes/hiring profiles in which many different types of candidates can mention the same words in their resumes.
What I’ve found from my own extensive hands-on experience testing these applications is that in addition to some well matched resumes, every AI matching application I have tested also ranks some results relatively highly who match the keywords, but not the “essence” of the job or resume. By “essence” I mean what the person will be responsible for/what the candidate has been primarily responsible for. If the search results don’t match the “essence” of the example resume or the job descritpion – they are of little-to-no value. Although in all fairness, junior sourcers or recruiters who are not very proficient at running Boolean searches may similarly struggle in creating Boolean search strings that return a high percentage of results of candidates who match the “essence” of an example resume or job description. Hopefully your staffing organization does not consist of only junior and/or sourcers or recruiters who are not proficient at creating effective and precise Boolean queries.
I believe the root of the limitation of AI matching apps is that essentially, all an AI matching application is capable of doing is finding resumes that it “thinks” are matched, based on it’s algorithms and pre-programmed search logic. These applications are simply taking words from an example resume or a job description and matching them to words that someone (the person or team who programmed the application) decided was related and relevant. In other words – regardless of marketing hype – the applications are simply buzzword matching, based on someone else’s programming. I’d argue that this is also essentially true for applications that make the claim that they “learn” as they are used. Let’s not forget the operative word in “Artificial Intelligence” is “Artificial.”
What should be especially disturbing to most people is that there is no objective way to assess whether the matching technology finds the BEST, or ALL of the potentially qualified candidates in the database or system. In my opinion, finding SOME matches is not a significant accomplishment, and certainly does not qualify an AI matching application to claim to be able to replace or be as effective as a human in the role of a sourcer or recruiter.
I think a good way to think about what all of these solutions do is to take a look at Amazon’s “you might also like” feature. While searching Amazon for a particular product, Amazon’s AI/matching engine tries to suggest other products you might be interested in based on the data collected from other online shoppers who have looked for or purchased similar products. Sometimes the suggestions are accurate – meaning you might actually be interested in the suggested product, but sometimes the suggestions are WAY off. What can we expect? After all – it’s just a computer program trying to “think” like a human (let alone YOU specifically).
Pretty much everyone in the staffing industry is aware of how imperfect resumes are in terms of their ability to represent a person’s experience and capabilities. The same is true of job descriptions. So you can pick your poison – allow a human sourcer or recruiter to interpret the the job description or the resume of the “ideal” candidate and create a search in an attempt to find qualified candidates, or allow an AI matching application to perform some exact and “fuzzy” buzzword matching.
Let’s get real here – a software application does not have the analytical and interpretive power of human cognition. With no example resume and a bare-bones job description, a good sourcer or recruiter can still find well qualified candidates. A software application cannot do the same thing – it can only “work” when given text to work with. Poor job descriptions will throw AI matching apps for a major loop and effectively render them useless. And while some job descriptions come with plenty of words – pages worth – in many cases all of the words actually don’t in fact accurately describe the position, it’s requirements, or define the ideal candidate.
Matching resumes is equally fraught with issues. Two candidates of similar skills, experience, and capability often have very different resumes. After all – the people we are looking to hire are not professional resume writers, nor should we expect them to be. Experienced sourcers and recruiters can “read between the lines” using cognitive processes and interpet resumes for skills, experience, and capabilities that are not explicitly mentioned in the resumes. This is something that an AI matching application may claim to do, but is hard pressed to prove. Let’s remember – AI matching apps can only work with text they are given – both the example resume or job description and the resumes it searches for. Although the people who program these applications can draw relation between multiple words/terms that can represent the same thing, they cannot possibly account for the unpredictable variety of ways in which a person can express – explicitly or implicitly – skills, experience, and capability.
AI matching applications do have value – in my opinion they provide “suggested reading.” They can certainly be used to assist junior sourcers/recruiters, or sourcers and recruiters who are not particularly adept at creating effective Boolean search strings. However, I would never implicitly trust that the matching engine is finding all of and the best matches in your database. The funny thing is – you’ll never know if this is the case or not…unless you’re really good at running searches, because then you can run your own searches and see if you can find more and better candidates that the matching engine did not find. Which is what I do.
This should go without saying, but one thing you should always keep in mind is that that all of these vendors with AI matching engines are SELLING you a product. They know that creating effective Boolean search strings can be hard to learn so they want to sell their solution as something that relieves sourcers and recruiters of having to run searches to make matches.
In my opinion, an ideal solution for most staffing organizations would include a resume/candidate database/ATS with BOTH a fully configurable search interface that supports full Boolean logic and ideally extended Boolean (with at the very least configurable proximity and variable term weighting), AND an AI matching engine – not one at the expense of the other. AI resume and job matching is great, but it is not a magical solution that replaces the human cognition of talented sourcers and recruiters. While AI matching apps may help junior recruiters, or those people who simply aren’t “wired” to run Boolean searches, it is not something anyone or any staffing organization should trust with making all of their matches. That would be scary!
If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.
.jpg)
Thanks! Nice post.
Спасибо за пост! Добавил блог в RSS-ридер, теперь читать буду регулярно..
ErvinTW – thanks!
wroreHopidork – Ви дуже вітаємо – Дякую за читання. Якщо вам подобається те, що Ви читаєте – будь-ласка, поширенню цього слова. Спасибо!
Hi,
Good post, and good views on the issues. However, it is important to look at ALL existing matching solutions instead of only those that match against predefined profiles or templates.
Matching should indeed be more than comparing against pre-defined templates. It’s about comparing core data. Text and numerical data. And as this data is always combined in resumes (or jobs), you should be able to search for it in one go; one query containing a combination of structured data, unstructured data, and numerical data. See it as a huge boolean string (however, matching is more than boolean searching, we’ll come to that later) where you can type in things you would like to find, whithout needing to re-query over and over again. When something is not found, matching should find results which are close to the initial criteria looked for. This we call “fuzzy matching”. The results of it should be ranked according the matching level, and according the relevancy. And preferably also say WHY the ranking is like that (to avoid wasting time analysing the results manually).
To make it a little more AI (as this seems to be liked in the initial post
, we have also built a huge knowledge base which performs Guided Search. This system looks at the query, looks at the database, and suggests extra words which it found in the database, and which are related to the queried words. It will then indicate how many more, or how much better results it can find if you decided to allow it to search also for those words. Good matching systems allow also to search based on sound. If you don’t know how to spell a name, how to write a word (is Excel with an “x” or “xc” in the middle? with one or two “L”s at the end?). Just type in the word, followed by a special character, and we will find words sounding like “exel”.
So I agree matching is more than comparing against pre-defined templates. It is ok to be able to do so, but that should be an application on top of a matching engine, not the core matching technology itself.
We’ve been around for some time, supplying matching platforms to the HR industry for years, and world wide. It is right to say that it will be difficult to replace humans by technology. But that is also not the objective. The objective is to make the recruitment process as good and as efficient possible, and it is not possible to do that whithout technology. Our systems offer many advantages, even in times of economic downturn. Feel free to look at our web site, or contact me for additional info.
Kind regards,
Bart
I noticed I referred to our site whithout mentioning it: http://www.actonomy.com.
Hello Bart,
A very good article.
We are a Recruitment Software provider. Currently we have boolean searches in place but not AI. I am recruiter of 17 years experience and “grew up” with Boolean search. We are in the progress of developing AI in to our product. Why? Our customers are asking for it.
What advice could you give us so that we could create a practical tool for our client base?
Thank you in advance
Daryl Keeley
Hey Bart! Nice post,
I am currently working in a company which deals with automated matching of resumes with job postings. We have really intelligent AI matching engines. We not only deal with simple whole text to whole text matching between resumes and jobs, but also with meticulous information of candidates experience, education, summary, skills set with corresponding job posting. We tag each section then within sugtags then we get more implicit information of location, latitutdes and longitudes of the job location, salary information accurately (everything automated ofcourse!). Finally I would like to mention the name of our company
Burning Glass Technologies. Please do let me know if anyone wants to try our products. We train our computers to act like HRs and Recruiters. We hire for you! 
Please catch me at http://www.linkedin.com/myprofile?trk=hb_tab_pro
Thanks a lot,
Priyadarshan