ML-powered Error Detection for Enhanced Fleet Maintenance Service

To enhance the efficiency of repair order quality reviews, our ML engineers developed an ML-powered system for error control in fleet management. The solution consists of two components: the ML model built based on gradient boosting algorithms that predict the likelihood of errors and an interpretability model designed to provide mechanics with actionable insights.

In the first stage of our project, we planned to build an ML solution that should test whether the mechanics’ comments retain enough structure and meaning and make sense for the maintenance service to perform. But later on, our ML engineers noticed that although this model successfully flagged high-risk records, it lacked transparency for technicians to understand and act on the predictions.

To address this, we incorporated an extra reasoning layer that helped to explain errors, prioritize issues, and suggest corrective actions. The latter streamlined the error resolution process and reduced the risk of incorrect billing.

Here is how the solution for error correction in fleet maintenance was built:

1. Data preparation

The project began with extracting historical repair order data from our partner’s databases. Using Python and Pandas, we processed and cleaned the data, removing inconsistencies such as duplicate records and anomalous values like negative prices.

Data quality improvements were crucial, as the lack of quality data was the primary challenge in this project. To be honest, in this particular case, we increased the model results more through data cleaning than by fine-tuning.

2. Model development

The Intelliarts team then trained an ML model using gradient boosting techniques, primarily XGBoost and LightGBM, to analyze repair orders. The model was designed to score each order based on error probability, helping prioritize quality review efforts. Evaluation metrics such as precision, recall, AUC, and Brier score were used to fine-tune performance and ensure high reliability.

Our next step was to integrate a reasoning module by using SHAP values and Z-score analysis for better usability. The idea was to highlight specific anomalies, for example, in parts or pricing deviating from historical data and suggest to the mechanic how to act next based on this information. To illustrate this, if a repair order listed the replacement of an engine at the $10 price, the system would flag it and suggest verification. Each flagged order is accompanied by a score and up to five potential error reasons, making it easier for mechanics to act upon the insights.

3. Model enhancement

99% accuracy was impossible to achieve in this case because of the human factor. Still, we managed to improve the model to reach 80% accuracy. What’s also important, our ML engineers added an extra business logic not to give a chance for the model to miss out on more expensive mistakes (receipts >$1000).

Another interesting moment is that initially, we planned to use ML for language processing only, i.e. the mechanics’ comments. As our team delved deeper into the business challenge, they suggested exploring other factors, such as pricing, parts, or labor, to improve the model performance. Hence, we refined the feature set and achieved a more robust solution. 

Iterative feature generation is a critical aspect of an ML engineer’s role. It involves identifying meaningful variables that may not seem representative at first but significantly enhance predictive capabilities.

In the process, the customer also mentioned that they had complex business rules documented but difficult to interpret. With our help, the ML model quickly learned these rules, often outperforming manually designed logic and uncovering underlying patterns. This way, our engineers proved that ML is not just about language models — it can also drive workflow optimization.

Since this was the first time the customer’s team had worked with machine learning, they were glad to expand automation opportunities as soon as they saw real ML potential.

4. Integration and deployment

The ML model was deployed via FastAPI and integrated into the customer’s workflow using Azure app containers. Our ML engineers integrated the solution directly into the mechanics’ CRM system, allowing real-time validation of repair orders. This way, technicians receive immediate feedback on potential errors, including probability scores and suggested corrections before finalizing an order.

By implementing this ML-driven system for detection of errors in fleet systems, the Intelliarts team helped improve operational efficiency and reduce costs. The ML model automated repair order quality reviews to lower the QA team’s workload, which resulted in estimated monthly savings of $30,000 – $40,000.

Another benefit of the ML solution was the real-time feedback it provided to mechanics. Since the model delivers human-readable explanations for its decisions, the QA team can quickly verify flagged issues and resolve any conflicts.

Overall, the key benefits of the ML solution include:

• Reduced manual workload: The ML model automates repair order reviews, reducing human intervention and improving efficiency.
• Faster error detection: The system significantly reduces the time needed to identify and correct issues and prevents billing mistakes.
• Cost-based accuracy assessment: The model assigns scores to repair orders and ensures that high-risk cases are still manually reviewed while automating low-risk ones.
• Failure explanations: Instead of vague alerts, the system provides technicians with precise reasons for detected errors, leading to faster corrections.
• Real-time QA checks: With the developed API, order reviews now happen instantly upon job completion, cutting down the review cycle from hours to seconds.
• Improved customer satisfaction: The model contributes to a better customer experience by minimizing errors and improving service quality.