![]() We see in the above example that the loss is 0.4797. it will try to reduce the loss from 0.479 to 0.0). A machine learning optimizer will attempt to minimize the loss (i.e. So that is how "wrong" or "far away" your prediction is from the true distribution. Q = np.array() # Predicted probabilityĬross_entropy_loss = -np.sum(p * np.log(q)) P = np.array() # True probability (one-hot) Here is the above example expressed in Python using Numpy: import numpy as np As it happens, the Python Numpy log() function computes the natural log (log base e). Note that it does not matter what logarithm base you use as long as you consistently use the same one. The sum is over the three classes A, B, and C. ![]() Where p(x) is the true probability distribution (one-hot) and q(x) is the predicted probability distribution. How close is the predicted distribution to the true distribution? That is what the cross-entropy loss determines. Now, suppose your machine learning algorithm predicts the following probability distribution: Pr(Class A) Pr(Class B) Pr(Class C) You can interpret the above true distribution to mean that the training instance has 0% probability of being class A, 100% probability of being class B, and 0% probability of being class C. The one-hot distribution for this training instance is therefore: Pr(Class A) Pr(Class B) Pr(Class C) Usually the "true" distribution (the one that your machine learning algorithm is trying to match) is expressed in terms of a one-hot distribution.įor example, suppose for a specific training instance, the true label is B (out of the possible labels A, B, and C). In the context of machine learning, it is a measure of error for categorical multi-class classification problems. Cross-entropy is commonly used to quantify the difference between two probability distributions. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |