Question: How can you improve decision making?

Sreejita Ghosh answered on 31 Jan 2020:

Wrt machine learning, decision-making of algorithms is improved by making better cost functions. Think of cost functions as reward/penalty. The lesser the computed value of cost function, less is the penalty (desirable state). The cost function is designed as a statistical physics entity such as entropy (disorderness) or free energy.
Entropy or disorderness is maximum when samples from all classes are mixed together- this is an undesirable state. In a classification problem or a clustering problem, we try to do grouping/sorting out of samples of same classes together and keep them far away from samples of other classes. Thus, we try to decrease the disorderness/entropy.
Similarly wrt energy interpretation, when samples of the same class are grouped together by a classifier there is decrease in energy due to the system becoming more stable due to decrease variance.
There is also an information theory interpretation which provides yet another cost function design – and that is how much information is gained when a certain decision rule is followed. For desirable outcome we want a mathematical design which provides more information gain- greater ‘reward’ for the classifier.
Based on how we formulate our classification, regression, or clustering problem, we choose a statistical physics or information theory based interpretation, and then write it mathematically such that we can optimise this equation for the desired outcome- the most optimum decision making by our algorithm