About our Scores

About Black Crow’s Scores

Introduction

Black Crow generates real-time predictions of how likely a user is to take a specific action, typically a purchase, in the future. We make (and update) this prediction on every pageview a user logs. To make these predictions more consumable, we simplify them and translate them into a “score”. The higher the score for a pageview, the more likely the user is to make a purchase, or take whatever other action the model is trained on.

About scores

A score is generated by comparing a prediction to other predictions in a defined group (called the comparison group). There is a new score that is generated on every pageview. This score may be higher, lower or the same depending on what predictive actions the user has taken. We aim to have roughly the same number of predictions in each group, so these scores can be thought of as deciles, or quartiles, or n-tiles generally, depending on how many scores we bucket predictions into. This we call the bucket count.

Illustrating the bucket count

As an example, let’s say the comparison group is every pageview on your site in a given day, and the bucket count is 10 (deciles). If the prediction for a pageview was very high, let’s say in the 95th percentile, it will get a score of 10. If the prediction for the pageview was in the 43rd percentile, it would get a score of 5. If instead, the bucket count was 3, then the prediction in the 95th percentile would be scored a 3, and the prediction in the 43rd percentile scored a 2.

Illustrating the comparison group

The comparison group is the other component of a score that we vary. You might want a relatively even distribution of predictions that were made only on your cart page. To support this we’ll create a “cart” page score, which is generated by comparing a prediction only to other predictions made on the cart page, versus the example above, where predictions were compared to all other predictions made on the site. To continue the example above, the prediction that was in the 43rd percentile when compared to all predictions on the site might only be in the 16th percentile when compared to predictions on the cart page, and thus would be scored a 2 with a bucket count of 10, or a 1 with a bucket count of 3.

‍