Mini-Project 1 · Headline Sentiment & the Next Day's Move

1. The Brief

You work on the news desk of a small fund. Every evening the desk collects the day's headlines. The portfolio manager asks two things:

• Question A (NLP): can a bag-of-words model read a headline and label it positive, neutral or negative for the company it names?
• Question B (finance): if we know the sentiment tonight, do we know the stock's direction tomorrow? In other words — is there a trade here?

Question A is Chapter 25 again, on new text. Question B is the reason quants care about NLP at all — and the reason most of them are disappointed. You will answer both with numbers.

2. The Dataset

The corpus is synthetic — generated from finance vocabulary with a fixed random seed, so every student downloads the exact same 900 rows and the solution's numbers reproduce on your machine. It is honest about being synthetic: 72 distinct words survive cleaning, real financial newswires use tens of thousands. Treat it as a flight simulator, not a cockpit.

Three examples (real rows from the file):

Note the second row. A human calls it neutral because it argues with itself. Remember Chapter 25 §11.5: two-signal headlines are exactly the ones word-count models get wrong. You will meet this row again.

3. Your Job

1. Explore. Load the TSV (pd.read_csv with sep='\t'). Count the classes. How many positive, negative, neutral? Plot or print value_counts(). Note the imbalance — one class is only ~19% of the rows.
2. Clean. Apply Chapter 25's five steps to every headline: re.sub to letters, lowercase, split, drop stop words (keep "not" — this is finance, negation is everywhere), Porter-stem. Build the corpus.
3. Classify. CountVectorizer, 80/20 split (stratify by label, random_state=0), then try MultinomialNB — the natural pair for word counts — alongside GaussianNB and LogisticRegression. Record all three accuracies.
4. Read the errors. For MultinomialNB, print the 3 × 3 confusion matrix and per-class precision/recall. Find two headlines it got wrong and explain each in one sentence.
5. Re-weigh. Swap to TfidfVectorizer and rerun MultinomialNB. Better or worse than raw counts here? Chapter 25's GaussianNB preferred counts. One representation, two models, two verdicts — explain.
6. Trade. Turn sentiment into positions: predicted negative → short (Down), neutral → stay out (Flat), positive → long (Up). Score the rule against the true NextDayMove on the test set. Compare against the majority baseline (always predict the most common move). Also compute, from the full dataset, the honest cross-tab: given true sentiment, how often is the next day Up / Flat / Down?
7. Verdict. Write three sentences to the portfolio manager: does the fund trade on this? What would you need to change your answer?

4. Solution & Real Results

Work the steps first. Then read the code walkthrough below — it explains every block, with special attention to the one line that changes everything: where y stops being two outputs and becomes three. The full script with all outputs waits in the toggle underneath.

4.1 The code, line by line

Block 1 · Load and look.

ds = pd.read_csv('financial_headlines.tsv', sep='\t')
print(ds['Sentiment'].value_counts())
# positive 366 · negative 365 · neutral 169

Read the tab-separated file — the same pattern as Chapter 25's restaurant data. Then count the classes before touching anything. The counts tell you two things: the classes are almost balanced at the top, and neutral is the minority at 19%. Every later choice — stratified split, per-class metrics — comes from this one print.

Block 2 · The line where two outputs become three.

y = ds['Sentiment'].map({'negative': 0, 'neutral': 1, 'positive': 2}).values

This is the pivotal line of the project. Models speak numbers, not words — so the three text labels must become three numbers: negative → 0, neutral → 1, positive → 2. .map() walks the column and swaps each label for its number; .values hands over the raw NumPy array.

Compare Chapter 25. There, the label column already held numbers — y = dataset.iloc[:, -1].values — and only two of them: 0 (not liked) or 1 (liked). Here the .map() line is new, and it changes everything downstream:

Nothing else in the pipeline changes. The vectoriser does not care. train_test_split does not care. MultinomialNB sees three classes and quietly builds three per-word probability tables instead of two, plus three class priors in the ratio 366 : 365 : 169. One line in, three outputs everywhere.

Block 3 · The cleaning loop.

corpus = []
for i in range(len(ds)):
    r = re.sub('[^a-zA-Z]', ' ', ds['Headline'][i]).lower().split()
    corpus.append(' '.join(ps.stem(w) for w in r if w not in set(sw)))

Chapter 25's five steps, compressed. One line per review: strip non-letters, lowercase, split, drop stop words ("not" kept), stem, rejoin. Out of 900 headlines comes a 72-word vocabulary — this small number matters later, when we resist believing the 1.0000.

Block 4 · The counts, and a new word: stratify.

X = CountVectorizer().fit_transform(corpus).toarray()   # (900, 72)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=0, stratify=y)    # 720 / 180

The bag of words as always. The split carries one argument Chapter 25 never needed: stratify=y. Random shuffling can shortchange a 19% class — the test set might get too few or too many neutrals. Stratifying forces the train and test sets to keep the class shares: 73 / 34 / 73 in the test, matching 366 : 365 : 169. Chapter 25's classes were 500/500, so plain splitting was safe there. With a minority class, stratify.

Block 5 · Train — three likelihood tables.

m = MultinomialNB().fit(X_train, y_train)
y_pred = m.predict(X_test)          # each headline → 0, 1 or 2

Fit on 720 headlines. Because y_train holds three values, the model learns three likelihood tables — how often each of the 72 words appears under negative, under neutral, under positive — plus the three priors. Predict gives every unseen headline one of the three outputs: a 3 × 3 confusion matrix follows automatically.

Block 6 · If you want the three probabilities.

p = m.predict_proba(X_test)   # shape (180, 3) — rows sum to 1

Each row is the model's full vote: P(negative), P(neutral), P(positive). predict() simply takes the largest of the three. And the row itself opens Chapter 25 §11's extension: if the largest probability is small, the model is unsure — send that headline to a human instead of a trade (quiz question 26).

Block 7 · The finance half — the aligned re-split.

idx = np.arange(len(ds))
_, idx_te = train_test_split(idx, test_size=0.2, random_state=0, stratify=y)
move = ds['NextDayMove'].map({'Down': 0, 'Flat': 1, 'Up': 2}).values
yte_move = move[idx_te]

The subtle block. y_pred sits in the order X_test was shuffled — but which original rows landed there? Split the indices 0…899 with the identical parameters, and the same shuffle happens: idx_te[k] is the original row number of X_test[k]. Now the moves can be fetched for exactly the test headlines. Note the second .map() — the same trick as Block 2, three text moves to 0/1/2.

Block 8 · The rule, and why it is one line.

pred_move = y_pred                                  # 0→Down, 1→Flat, 2→Up
print((pred_move == yte_move).mean())               # 0.4500

The trading rule is just y_pred — because the two encodings were chosen to line up: sentiment 0 (negative) predicts move 0 (Down), 1 → Flat, 2 → Up. That alignment is a design choice, not luck. A more explicit script would write {0:0, 1:1, 2:2} so a future reader sees the mapping. Score it: the rule is right on 45.0% of test days, against a 41.1% majority baseline — the thin-edge story of Section 4's results.

# ---- Mini-Project 1: headline sentiment & the next day's move ----
import numpy as np, pandas as pd, re, nltk
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
nltk.download('stopwords', quiet=True)
sw = stopwords.words('english'); sw.remove('not')     # keep negation
ps = PorterStemmer()

ds = pd.read_csv('financial_headlines.tsv', sep='\t')
print(ds['Sentiment'].value_counts())
# positive 366 · negative 365 · neutral 169  (neutral = 18.8% — the minority)

corpus = []
for i in range(len(ds)):
    r = re.sub('[^a-zA-Z]', ' ', ds['Headline'][i]).lower().split()
    corpus.append(' '.join(ps.stem(w) for w in r if w not in set(sw)))

from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB, MultinomialNB
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, accuracy_score, classification_report

y = ds['Sentiment'].map({'negative': 0, 'neutral': 1, 'positive': 2}).values
X = CountVectorizer().fit_transform(corpus).toarray()      # 900 x 72
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=0, stratify=y)       # 720 / 180

for model in [MultinomialNB(), GaussianNB(), LogisticRegression(max_iter=2000)]:
    model.fit(X_train, y_train)
    print(type(model).__name__,
          round(accuracy_score(y_test, model.predict(X_test)), 4))
# MultinomialNB      0.9167
# GaussianNB         0.9944
# LogisticRegression 1.0000   (72 words — see the reading below)

m = MultinomialNB().fit(X_train, y_train)
y_pred = m.predict(X_test)
print(confusion_matrix(y_test, y_pred))
# [[63 10  0]      <- true negative
#  [ 1 30  3]      <- true neutral
#  [ 0  1 72]]     <- true positive
print(classification_report(y_test, y_pred,
      target_names=['neg', 'neu', 'pos'], digits=3))

# step 5: TF-IDF
Xt = TfidfVectorizer().fit_transform(corpus).toarray()
Xtr2, Xte2, ytr2, yte2 = train_test_split(
    Xt, y, test_size=0.2, random_state=0, stratify=y)
print('MNB on TF-IDF:',
      round(accuracy_score(yte2,
          MultinomialNB().fit(Xtr2, ytr2).predict(Xte2)), 4))   # 0.9389

# step 6: trade the sentiment  (0->Down, 1->Flat, 2->Up)
idx = np.arange(len(ds))
_, idx_te = train_test_split(idx, test_size=0.2, random_state=0, stratify=y)
move = ds['NextDayMove'].map({'Down': 0, 'Flat': 1, 'Up': 2}).values
yte_move = move[idx_te]
print('rule accuracy :', round((y_pred == yte_move).mean(), 4))   # 0.4500
counts = np.unique(yte_move, return_counts=True)
print('majority move :', round(counts[1].max() / len(yte_move), 4))  # 0.4111
print(pd.crosstab(ds['Sentiment'], ds['NextDayMove'], normalize='index').round(3))

6. The Honest Limits

• 72 words. The corpus is synthetic and tiny. Real financial English has tens of thousands of words, tickers, numbers ("AAPL falls 3% after Q3 miss") and entities. Expect far below 0.92 there.
• The labels are clean. Real annotators disagree on financial sentiment roughly a third of the time — the PhraseBank ships "all-agree" and "75%-agree" variants for exactly this reason.
• The moves are planted. The 55% alignment is a design choice. In real markets the measurable edge of public sentiment is smaller, decays after minutes, and is negative after costs.
• One headline, one day, one universe. No position sizing, no risk model, no transaction costs, no look-ahead audit. The full treatment is Applied Project 1's job.