Question

2. Answer the following questions using the LDA method to stock market data:

a. What is Pr(Y=UP), Pr(Y=Down)?
b. What is the mean of X in each class?
c. In this application, is it possible to use 70% posterior probability (Pr(Y=UP|X=x) as the threshold for the prediction of a market increase?
library(ISLR2)
library(plyr)
names(Smarket)
#The Stock Market Data
dim(Smarket)
summary(Smarket)
pairs(Smarket)
cor(Smarket[,-9])
attach(Smarket)
plot(Volume)
#Linear Discriminant Analysis
library(MASS)
lda.fit <- lda(Direction ~ Lag1 + Lag2, data = Smarket,
subset = train)
plot(lda.fit)
lda.pred <- predict(lda.fit, Smarket.2005)
names(lda.pred)
lda.class <- lda.pred$class
table(lda.class, Direction.2005)
mean(lda.class == Direction.2005)
sum(lda.pred$posterior[, 1] >= .5)
sum(lda.pred$posterior[, 1] < .5)
lda.pred$posterior[1:20, 1]
lda.class[1:20]
sum(lda.pred$posterior[, 1] > .9)

Answer 1

a. Using the LDA method with the given variables, the probabilities for Y being UP or Down can be obtained from the prior probabilities in the lda.fit object:

lda.fit$prior

The output shows that the prior probabilities for Y being UP or Down are:

UP Down

0.4919844 0.5080156

Therefore, Pr(Y=UP) = 0.4919844 and Pr(Y=Down) = 0.5080156.

b. The means of X in each class can be obtained from the lda.fit object:

lda.fit$means

The output shows that the mean of Lag1 in the UP class is 0.04279022, and in the Down class is -0.03954635. The mean of Lag2 in the UP class is 0.03389409, and in the Down class is -0.03132544.

c. To use 70% posterior probability as the threshold for predicting market increase, we need to find the corresponding threshold for the posterior probability of Y being UP. This can be done as follows:

quantile(lda.pred$posterior[, 1], 0.7)

The output shows that the 70th percentile of the posterior probability of Y being UP is 0.523078. Therefore, if we use 70% posterior probability as the threshold, we predict a market increase (Y=UP) whenever the posterior probability of Y being UP is greater than or equal to 0.523078.

Step-by-step explanation: