Thursday, December 29, 2016

33. SCORING MODELS

OBJECTIVE
Define the priority of action concerning customers, employees, products, and so on.


DESCRIPTION
Scoring models help to decide which elements to act on as a priority based on the score that they obtain. For example, we can create a scoring model to prevent employees leaving the company in which the score depends on both the probability of leaving and the performance (we will act first on those employees who have a higher probability of leaving and are important to the company). Scoring models are also quite useful in marketing; for example, we can score customers based on their probability of responding positively to a telemarketing call and, based on our resources, call just the first “X” customers.

The model that I will propose concerns a scoring model of customers’ value based on the probability of purchasing a product and on the amount that they are likely to spend. This model is the result of two sub-models:

  • -          Purchase probability: We will use a logistic regression to estimate the purchase probability of a customer in the next period (see 26. RFM MODEL and 60. LOGISTIC REGRESSION);
  • -          Amount: We will use a linear regression to estimate the amount that each customer is likely to spend on his or her next purchase (see 38. LINEAR REGRESSION).

The first step is to choose the predictor variables. In our case I suggest using recency, first purchase, frequency, average amount, and maximum amount of year -2, but we could try additional or different variables. The target variable will be a binary variable that represents whether the client made a purchase during the following period (year -1). A logistic regression is run with the eventual transformation of variables and after verifying that all the necessary assumptions are met (see 36. INTRODUCTION TO REGRESSIONS and 60. LOGISTIC REGRESSION).

scoring model logistic regression

Coefficients of the Logistic and Linear Regressions

In the second part of the model, we can use for example only the average amount and the maximum amount of year -2, and the total amount spent in year -1 is used as the target variable. We run a multivariate linear regression with the eventual transformation of the variables, after verifying that all the necessary assumptions are met (see 36. INTRODUCTION TO REGRESSIONS, 38. LINEAR REGRESSION, and 39. OTHER REGRESSIONS). It is important to note that in this regression we will not use the whole customer database but select only those customers who realized a purchase in year -1.

The last step is to put together the two regressions to score customers based on both their purchase probability and the likely amount that they will spend. We use the regression coefficients for the estimates of each customer. In the linear regression, we directly sum the intercept and multiply the variables’ coefficients (Figure below) by the actual values of each customer to estimate the amount.[1] However, in the logistic regression we should use the exponential function to calculate the real odds of purchasing:

Probability = 1 / (1 + exp(- (intercept coefficient + variable 1 coefficient * variable 1 + variable n coefficient * variable n)))


Result Table with the Purchase Probability, Estimated Amount, and Final Score

Now that we have two more columns in our database, we just need to add a third one for the final score, which will be the purchase probability times the estimated amount (Figure above). With this indicator we can either rank our customers (to prioritize marketing and resource allocation for some customers) or use this indicator to estimate next-period revenues.


Download the Scoring Models Template




[1] Estimated amount = Intercept + Coefficient 1 * Variable 1 + Coefficient 2 * Variable 2.
Be aware that, if we have transformed some of the variables, we cannot simply multiply the coefficient but should make some additional calculations.

Tuesday, December 13, 2016

5. COMPETITIVE MAP

OBJECTIVE

Analyze the positioning of a company or a product in comparison with other companies or products by evaluating several attributes.


DESCRIPTION

A competitive map is a tool that compares a company or products with several competitors according to the most important attributes. We can also add to the map the relative importance of each attribute. From this map we can understand how a company is positioned in relation to several attributes and define the key issues for strategic decisions. For example, we can decide to focus on the communication and promotion about the quality of our product if we are well positioned and it is important for customers.

Example of Competitive Map

Competitive Map

The data for this map are usually gathered through surveys. If the selection of attributes is not clear or the number of attributes is large, we can ask a preselection question whereby interviewees rank the most important attributes. Then they will be asked about the importance and performance of the selected attributes for each company or product. I suggest dividing the question into two parts:
  • -          Importance: give a score from 1 to 5 for each attribute;
  • -          Performance: give a score from 1 to 5 for each combination of attribute and company (or product).




TEMPLATE

Tuesday, December 6, 2016

45. A/B TESTING

OBJECTIVE

Test two or more items/objects and identify the one with the best performance.


DESCRIPTION

A/B testing is part of a broader group of methods used for statistical hypothesis testing in which two data sets are compared. Having defined a probability threshold (significance level), we can determine statistically whether to reject the null hypothesis or not. Usually, the null hypothesis is that there is no significant difference between the two data sets.
A/B testing is a randomized experiment with two variants (two-sample hypothesis testing), but we can also add more samples. The difference from multivariate testing is that in A/B testing only one element varies, while in the other test different elements vary and we should test several combinations of elements. These tests are used in several sectors and for different business issues, but nowadays they are quite popular in online marketing and website design.

A/B Testing of Conversion Rate

Output of Conversion Rate A/B Testing

Usually the steps to follow are:
  • -        Identify the goals: for example “improving the conversion rate of our website”;
  • -       Generate hypotheses: for example “a bigger BUY button will convert more”;
  • -       Create variables: in our example the element to be modified is the BUY button and the variation website can be created with a double-size BUY button;
  • -       Run the experiment:

o   Establish a sample size: depending on the expected conversion rate, the margin of error that is acceptable, the confidence level, and the population, the minimum sample size can be calculated (see the template);
o   The two versions must be shown to visitors during the same period, and the visitors must be chosen randomly (we are interested in testing the effect of a larger button; if we do not choose visitors randomly or show the two versions during different periods, the results will probably be biased);

-          Analyze the results:
o   Significance: depending on the significance level chosen for the test (usually 90%, 95%, or 99%), we can be X% confident that the two versions convert differently;
o   Confidence intervals: depending on the confidence level chosen, there will be a probable range of conversion rates (we will be X% confident that the conversion rate ranges from X to Y);
o   Effect size: the effect size represents the difference between the two versions.

The proposed template provides a simple calculator for the necessary sample size and for testing the significance of conversion rate A/B testing. However, a considerable amount of information about A/B testing is available online.[1] The template of chapter 44. TEST OF PROPORTIONS shows the same test with more statistical detail as well as the calculation of the mean difference confidence interval, while the A/B testing template presents confidence intervals for each mean of the two samples.

In the proposed example, the data are obtained using a web analytics tool (for example Google Analytics), but they can come from any experiment that we decide to run.


TEMPLATE