Allsorter CV Optimization process has benchmarked improvements for reducing turnaround time for recruiters. This is achieved by parsing and reformatting resumes on runtime thus eliminating the need for using manual transformation. This objective pertains to highlight any differences between original CV and reformatted CV using an ensemble of advanced text similarity methods.
Text Extraction - Gaps Assessment and Detection
A minimum threshold was set for similarity score. Any text in the source not matching any non-aligned text in the target with a score beyond the threshold is marked as a gap. In our experiments, we found that the optimal similarity threshold wavers between 72% and 80%.
Evaluation: In our test-case scenario, we find that the gap analysis framework achieves an annotation accuracy in excess of 96%. Evaluation was manually carried out on a random sample of 10 CVs. Two employees from BPIS’s QA testing and Data Analysis team were assigned to manually inspect the annotation (gap highlights) created by the gap analysis framework for those 10 CVs. This form of assessment is acceptable since both analysts have HR domain expertise and are very experienced with tasks of this nature. Each analyst is expected to record the errors (wrongly highlighted text) and omissions (texts which should be highlighted but not) and for the CV, and to come up with a score of the accuracy. The system achieves an F1 score of 94 % and an accuracy of 96%. This finding corroborates the production use-case where feedback from commercial users of Allsorter has been overwhelmingly positive.
Potential Bias Information Highlight – AIRE has determined potential bias elements in a CV after interviewing recruiters from multiple organizations. In addition to a person’s ethnic origin, age, and gender, their work experiences in certain organizations or education from certain institutions may also impact on their chances of hiring. This objective will allow identifying and highlighting all such items on a CV that a recruiter may view on the original CV and then have the option of removing these items. To achieve this objective, several subtasks were defined as below.
In order for AIRE to achieve this objective, the developed system must show a capability for language understanding in order to approximate the meaning of skills and job description. For instance, a good system must understand synonymous terms as well as related terms. A very simple example is the skill ‘Java’ which though not similar to the terms ‘Developer’ and ‘Programmer’, are clearly related. Existing systems would fail to match CVs where an exact keyword being searched is missing. This has the potential of missing out on tons of candidates whose skills and experience may be written on their CV differently from the query terms being used by the recruiter.
AIRE has determined that in order to achieve a skills-based CV ranking, it is important to improve the skills extraction pipeline of Allsorter. Allsorter currently extracts some hard and soft skills, classified into 4 classes, namely – Professional, Computer, Language, and Soft skills, however, the current skills extraction system is not optimal. The IA has worked with the company’s domain expert to annotate some CVs and job descriptions. The annotation includes skills as well as their left and right surrounding context. The IA has performed a scaled evaluation of the skill extractor model and has observed an F1 score of 92% with a rough increase of 4% when hard skills alone are considered.
Performance Testing:
The semantic annotation pipeline described earlier was testing using feature ablation and cross fold validation (K=10) resulting in an F-measure of 0.93 (93% precision and recall on test corpus)
