Using the Apriori Algorithm to Discover Association Rules

For this tutorial, I am going to use the transaction Income dataset from the R package arules. This dataset comes from the website for the book The Elements of Statistical Learning. Chapter 14 has information about association rules. Here is a link and a citation for that book:

Click to access ESLII.pdf

Hastie, T., Tibshirani, R. & Friedman, J. (2001) The Elements of Statistical Learning. Springer-Verlag

I am going to use the Income dataset but if you type data() into R after loading the arules package, you will get this list that includes other dataset options to practice on:

Data sets in package ‘arules’:

Adult                    Adult Data Set
AdultUCI                 Adult Data Set
Epub                     The Epub Transactions Data Set
Groceries                The Groceries Transactions Data Set
Income                   The Income Data Set
IncomeESL                The Income Data Set
Mushroom                 The Mushroom Data Set as Transactions
SunBai                   The SunBai Weighted Transactions Data Set

The variables in the Income dataset include:
income, sex, marital status, age, education, occupation, years in bay area, dual incomes, number in household, number of children, householder status, type of home, ethnic classification, and language in home. This data has already been preprocessed. Missing values have been removed and the ordinal categorical variables have been divided at their medians.

rm(list = ls()) #clear the workspace
library(arules)
data(“Income”) #upload data
income<-Income
summary(income)

transactions as itemMatrix in sparse format with
6876 rows (elements/itemsets/transactions) and
50 columns (items) and a density of 0.28

The density value is the ratio of the dataset values that are present. When dealing with demographic data, an individual is only going to fall into a few columns in each row, so the sparse density makes sense.

Support measures the frequency in which an item (or itemset) appears in the data.

I am only going to look at demographic information that occurs at a of minimum of 5%. cex.names changes the size of the font

quartz()
itemFrequencyPlot(income, support = 0.05, cex.names = .5)

I can also sort the data by looking at the top 25 demographic variables.

quartz()
itemFrequencyPlot(income, topN=25)

The apriori algorithm can be used to discover association rules. According to Practical Machine Learning in R by Fred Nwanganga and Mike Chapple (a book I HIGHLY recommend), the apriori algorithm works by evaluating items based on whether or not they reach the predetermined support threshold. If an item does not reach the specified level of support it will be considered unimportant and is henceforth ignored by the algorithm. On the other hand, if the item reaches the support threshold, the item is then paired with a second item. This new itemset is evaluated at the support level. If the itemset does not reach the level of support, it is ignored. Items that reach the support threshold will be paired with a third item. This process continues until there are no more itemsets to evaluate.

The function apriori() can be used to apply the algorithm; the support and confidence can be set. The confidence is a measure of accuracy that can be calculated by dividing the support in which the antecedent and consequent both occur by the support in which only the antecedent occurs.

I will set the min length to two because I am not interested in single variables.

income.rules <- apriori(income, parameter = list(support = 0.4, confidence = 0.3, minlen=2))

summary(income.rules)

There are 113 association rules. 64 rules have a length of 2, 45 rules have a length of 3, and 4 rules have a length of 4. Lift is also described in this summary. In their book, Practical Machine Learning in R, Fred Nwanganga and Mike Chapple define the lift as the “increased or decreased likelihood of both the antecedent and the consequent occurring together compared to the typical rate of occurrence of the consequent alone” (p.374). The lift is calculated by dividing the confidence of the itemset containing both the antecedent and consequent by the support of the itemset containing the consequent.

inspect(income.rules)

The first 21 association rules are presented here. None of them are surprising, but understanding the output is important. Rule 20 can be interpreted as the home is listed as a house 60.65% of the time when ethnic classification is white. Also, when the ethnic classification is white, the type of home is 1.02 times more likely to be categorized as a house. This is found to be true in 40.62% of the dataset, 2,793 observations.

I am going to explore this data further by lift in descending order.

inspect(head(sort(income.rules, by = “lift”), n = 50))

Rule [28] {education=no college graduate,
language in home=english} => {income=$0-$40,000} Support: 0.4454625, Confidence: 0.7022008, Coverage: 0.6343805, Lift: 1.128115, Count: 3063

Rule 28 tells me that if the person is not a college graduate, and the language in the home is english, the income is between $0 and $40,000 70.22% of the time. If the person is not a college graduate, and the language in the home is english, it is 1.13 times more likely that the income is between $0 and $40,000. This pattern is found in 44.55% of the dataset, 3,063 observations.

The data can also be sorted by confidence.

inspect(head(sort(income.rules, by = “confidence”), n = 50))

The subset() function provides an avenue for specific subset searches in the rules.

nocollegegrad<-subset(income.rules, items %in% “education=no college graduate”)

inspect(head(sort(nocollegegrad, by = “lift”)))

By running ?arules::transactions in R, you can follow instructions on how to build your own dataset.

Here is an example:

#Create a list
my_list <- list(
c(“apple”,”cheese”,”dog food”,”kale”),
c(“apple”,”jalapenos”,”dog food”),
c(“cheese”,”wine”),
c(“apple”,”cat food”),
c(“cheese”,”wine”,”cat food”,”kale”),
c(“oat milk”,”wine”,”dog food”),
c(“oat milk”,”cheese”),
c(“oat milk”,”jalapenos”)
)

#Set the names of the transactions
names(my_list) <- paste(“Tr”, c(1:8), sep = “”)
my_list

#Create the transactions
trans <- transactions(my_list)
trans

summary(trans)