METHODOLOGY FOR TRANSITION PROBABILITIES DETERMINATION IN A MARKOV DECISION PROCESSES MODEL FOR QUALITY-ACCURACY MANAGEMENT

The main goal of the presented research is to define a methodology for determination of the transition probabilities in a Markov Decision Process on the example of optimization of the qualityaccuracy through optimization of its main measure (percent of scrap) in a Performance Measurement System. This research had two main driving forces. First, today’s urge for introduction of more robust, mathematically founded methods/tools in different enterprise areas, including PMSs. Second, since Markov Decision Processes are chosen as such tool, certain shortcomings of this approach had to be handled. Exactly the calculation of the transition probabilities is one of the weak points of the Markov Decision Processes. The proposed methodology for calculation of the transition probabilities is based on utilization of recorded historical data and they are calculated for each possible transition from a state after one run to a state after the following run of the influential factor (e.g. machine). The methodology encompasses several steps that include: collecting different data connected to the percent of scrap and their processing according to the needs of the methodology, determination of the limits of the states for every influential factor, classification of the data from real batches according to the determined states and calculation of the transition probabilities from one state to another state for every action. However, the implementation of the Markov Decision Process model with the proposed methodology for calculation of the transition probabilities resulted in optimal policy that showed significant differences in the percent of scrap, compared to the real situation when the optimization of the percent of scrap was done heuristically (5.2107% versus 13.5928%).