Traditional process and rule systems
The traditional approach to optimization in process and decision support systems is to model the problem using a framework (e.g. business processes, business rules, or a combination of the two), integrate the model to the necessary data sources, deploy and execute the model using an engine that understands the model, observe and analyze the execution over time using integrated monitoring and analytics tools, identify bottlenecks and issues in the model, modify the model to address the issues, and redeploy it. This iterative process can be represented with Figure 1 below. For more on this refer to my article on BPM.
Figure 1. Process/rule systems without machine learning
In these systems, optimization is a post-execution and manual activity, and it largely depends on the availability of good analytics tools providing the right metrics and reports, and the expertise using them. The business analyst or manager in charge must also understand the model to identify the issues and recommend changes to the model to improve it. This optimization typically results in modifications to the process models and business rules, which in turn may require more technical resources for implementation and redeploying of the modified model. The changes in the model may mean changes to the integration with other applications and changes to its analytics. Bottom line: all this is easier said than done. While in theory this continuous improvement and optimization sounds good, it’s not all that practical.
Process and rule systems with machine learning
By use of machine learning, we are now able to move the optimization step up to the execution stage and perform some optimization automatically in real-time. Extending the problem model with the right data models and the training data sets, the system can use the appropriate algorithms to perform classifications or use regression algorithms and make predictions. This can result in improved and optimized process execution and better decision making in real-time and first time around. There is less need for post-execution analytics and optimization.
Furthermore, the system with machine learning is able to learn and improve on its optimization. The more instances of the problem it solves, it becomes smarter and provides increasingly improved executions on a continuous bases. Figure 2 below depicts this enhanced system with ML. Note that we still have the monitor/analyze step as it is still necessary to monitor the execution. After all, there may still be changes necessary to the processes and rules as well as to the learning model. Machine learning is not intended to eliminate the need for human monitoring and reporting.
Figure 2. Process/rule systems with machine learning
Another benefit of using machine learning is that it can help simplify the set of processes and rules defining the original problem model. In the traditional model without machine learning, there is a need for more complex rules and decision points. With machine learning, we delegate some of the rules and decisions to machine learning to figure out so we don’t have to encode every possible rule. This results in a cleaner and simpler model that is also easier to understand and maintain.
A example
Consider a customer-facing process around marketing and sales; e.g. for running a marketing campaign or a sales process consisting of stages, decision points, and decision points. This process typically involves customer touch points such as sending an email to a particular segment of customers and inviting them to the recipients to sign up for a webinar, download a piece of content, or schedule a meeting. We can use supervised learning for classification to accurately classify the target audience and accordingly create highly personalized messages. For example, a promotional email can include products based on past purchase behavior of a customer.
The next question to answer is when to send the email. Regression can be used to accurately calculate the best time to send the message. Without machine learning this simple marketing and sales process can become unnecessarily complicated and difficult to maintain. Even if we analyse the execution data and observe poor execution in terms of outcome (e.g. low rate of email open rate or CTR) it’s not clear how we can modify the model and process definition to improve it.
Conclusion
Traditional process and rule based systems can get complex if we’re to encode all the rules. Also improvements (if any) must be done after execution through monitoring and analysis. Use of machine learning in these systems can greatly improve their performance and accuracy by optimizing various variables in the system in real-time and during execution. These intelligent systems are able to learn and improve their predictions and decision making over time as more data becomes available. Machine learning can also help reduce the complexity of the process and rule models, making it easier to create and maintain them.