The words “humans” and “machines” seem inherently at odds with each other. However, with a mindful implementation, machine learning adds lift to traditionally human processes. In this case study, I illustrate one of those human processes – employee staffing – and how machine learning allowed us to reimagine the process in a way that was intelligent, efficient, and, most importantly – human-centered.
What is employee utilization?
Centric is considered a Professional Services firm as we typically derive revenue from “billable hours” worked by our consultants and charged to our clients. Law and accounting firms are examples of other types of Professional Services firms.
For Professional Services firms, the key lever to maximizing profitability is utilization. Utilization is the ratio of hours billed to all hours worked by a consultant. Non-billable time is sunk cost, eating into profit margins. Thus, increasing utilization drives profitability.
If utilization is the lever to profitability, staffing is the lever to improving utilization. Staffing is the process of matching the right person to the right project at the right time to optimize utilization.
Beyond profitability, utilization also is a significant factor in employee satisfaction. Consultants like to be busy! We call the downtime between projects being “on the bench.” Being on the bench can be a welcome relief after finishing a tough project. It is also an excellent time for catching up on training and other administrative tasks.
That said, I have observed if a consultant sits on the bench too long, they get antsy. When they become antsy, they become prone to quit. A while ago, I pulled the numbers to see to validate my observations. At that time, our average utilization of departed employees was 67.3%. Average utilization of current employees? Almost ten points higher: 76.3%.
Thus, strong utilization is important to both profitability and employee satisfaction in Professional Services firms.
How we used to staff
Staffing used to be pretty painful, with a lot of cumbersome parts.
First, we created a massive spreadsheet of every available consultant and every upcoming project. It had to be maintained manually, which means it never was up to date.
Next, we held a weekly all-comers call to discuss the massive spreadsheet. And by all-comers, I mean at least two representatives from each of our business units plus representatives from leadership, HR, and recruiting. There were a ton of people on the call!
On the call, all those people went line by line through the massive spreadsheet. Talk about painful! And, to make the pain even more acute, it was not very productive. Despite having all the data, and all the people, present and accounted for, we still seemed to churn when it came to staffing. There was too much information, presented haphazardly.
They say necessity is the mother of invention. For us, the agony of the massive spreadsheet call birthed the Staffing Prioritization Algorithm.
We were drowning in data. There needed to be a way to prioritize our staffing needs.
To help cut through the chaos, we created an algorithm to prioritize people for staffing assignments. Instead of featuring all the available resources, we would hyperfocus on only the top ten people who needed staff. But how to prioritize?
We chose four factors:
How we determined predicted utilization
We gathered historical data on employees, including past utilization, tenure, skills, travel preference, job title, performance review rating, and operating group. We then categorized everyone in this group as either high utilization or low utilization.
Now it gets fun. We created a machine learning model to ingest this data, learn from it, and make predictions on our current employee population.
We used Azure Machine Learning Studio as our tool of choice. It’s drag and drop interface makes experimenting and deploying machine learning models a breeze.
Here is a step by step account of how we built the model.
Step 1: Import & Join Data
Machine Learning dies without data! Our data was in CSV files making it a snap to import into Azure Machine Learning Studio. However, the tool plays nicely with other types of databases as well.
Step 2: Prepare Data
Just like any data science project, you must first scrub and ready your data. In our case, we took the following cleansing steps:
Step 3: Create
Now it’s time to do some machine learning! We split our historical data set into groups – the training set and the testing set. We chose to put 75% of the data in the training set and the remainder in the test set. We had a fairly small data set to work with, so we chose a smaller training set to avoid overfitting. Had we had more data, we may have used a larger percentage of the data to be in the training set, and/or adding a validation group as well.
We also implemented the SMOTE function. SMOTE stands for Synthetic Minority Oversampling Technique, which is a fancy way of saying that it makes extra copies of training data to ensure a balanced number of positive and negative examples.
Step 4: Train Model & Evaluate
One of the nicest features of the Azure Machine Learning tool is that it allows you to test multiple algorithms in parallel to see which ones may perform best for you. We tested the following binary classification models: Logistic Regression, Decision Jungle, SVM, Neural Network, and Boosted Decision Tree.
For each algorithm tested, we scored its predictions against the test data set from Step 3. There are several metrics that you can evaluate algorithms for, such as utilizing a confusion matric or by minimizing log loss. In our case, we chose to optimize for AUC, or “Area Under the Curve.”
When we optimized for AUC, the Neural Network algorithm produced the best results with an AUC of .746.
Step 5: Evaluate Current Employees
Now that we had a trained machine learning model, we needed to evaluate our current employee base for predicted future utilization. This was a relatively straight forward step, as we only need to connect a file of current employees, and their attributes, to our trained model. The model then produced an output file of each employee and their likelihood to leave, on a scale of 0 to 1—the closer to 1.0, the more likely to have high utilization. The closer to 0.0, the employee is more likely to have low future utilization.
Machine Learning: The Human Component
Up to this point, we have only discussed the technical implications of deploying our utilization machine learning model. However, understanding the human impact was even a more critical factor in its deployment.
Machine Learning is powerful, giving us the ability to synthesize and evaluate information broader, quicker, and deeper. Insights can be identified that far exceed the capabilities of the human mind.
But Machine Learning is not without risk. Just like anything, it can be subject to bias and misinterpretation. However, because of machine learning’s perceived infallibility, these shortcomings tend to be minimized or downright ignored.
It was important for us to implement this model in a way that was thoughtful and kind. Yep, I just said kind. We were modeling humans, and using the predictions, to make decisions regarding humans. Being kind, was more critical than any AUC score could be.
Often machine learning bias hides deep under the covers, in seemingly benign data points that are actually dripping in historical bias.
To ensure our model did not harbor any unintentional bias, we limited the input values, removing anything that could even possibly influence results. The obvious data points, like gender and race, were excluded from the training data. And we also exclude less obvious data points that may or may not cause bias, such as age.
A case could be made that someone younger who have less work experience and thus be more challenging to staff. However, we were worried if we included age the model may discriminate from older workers. So we left age out. Better safe than sorry.
Our conservative approach to data inclusion left us with fewer input features than was ideal. More inputs could have given us a better performing model, perhaps. But we felt the risk of bias outweighed the risk of a bit less accurate model.
Implementing the staffing model is a way that positive, and not punitive, was incredibly important to us. We knew if we were not careful in our approach, we would breed distrust and fear with employees.
The last thing we wanted to be was for anyone to think that their career was in the hands of some black-box algorithm. Or even worse, being on the “list” implied they were going to be fired. That was as far from our intentions as it could be, but we could easily see how a worried employee could make those assumptions.
Thus, we framed the model as a career enabler in a couple of different ways. For employees on the “list,” we identified ways to provide them extra support and opportunity, including:
However, the most important thing we did was to present the list in an approachable way. People are not data points, and you are doing them a disservice by reducing them to a line in a spreadsheet. So instead of a lifeless list of names, we converted the list into virtual playing cards.
All the basics are covered, name, department, key skills. But we took it a step further and provided insight into the person behind the name. What are their interests out or work? What soft skills do they really excel at? What is their dream role? Basically, who is the real person? And most importantly, we shared pictures of each employee. Pictures resonant. When we started including pictures, engagement really went up. People identify with people. They are more willing and excited to help another human than a line in a spreadsheet.
This project and its unconventional approach to staffing yielded improved utilization, and thus improved profitability. Profitability immediately jumped 20% after implementing this approach. Anecdotal feedback from employees and staffing managers was overwhelmingly positive, with an appreciation for the transparent and thoughtful approach to a historically tricky process.
Parting Words of Wisdom