Skip to main content
GACS will never ask for your seed phrase, private keys, or payment. free and ad-free.
GACS.app — Global Anti-Crime & Safety
Sign up / Log in
Back to lessonLab · Module 3 / 12
Lab

Embeddings & Representation Learning

Hands-on lab with starter code you can copy + run, and guided steps to produce the artifact for this module.

Setup

  1. Open a fresh terminal in a new folder, e.g. mkdir lab-3 && cd lab-3.
  2. Create a Python 3.11 venv: python -m venv .venv && source .venv/bin/activate.
  3. Install the dependencies noted on each starter snippet (the small pip install … line above the code).
  4. Copy each starter into a file with the suggested name, then run it and capture the output.

Starter code

01 · Sentence embeddings with cosine similarity
pip install sentence-transformers
save as: 03_01_lab.py
python
from sentence_transformers import SentenceTransformer
import torch.nn.functional as F
import torch

m = SentenceTransformer("BAAI/bge-small-en-v1.5")
sents = ["A cat sat on the mat.", "A feline rested on the rug.",
         "Quantum chromodynamics studies the strong force."]
emb = m.encode(sents, convert_to_tensor=True, normalize_embeddings=True)
sim = emb @ emb.T
print(sim.round(decimals=3))
Run with: python 03_01_lab.py
02 · Visualise embeddings with PCA
pip install scikit-learn matplotlib
save as: 03_02_lab.py
python
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt

xy = PCA(n_components=2).fit_transform(emb.cpu().numpy())
plt.scatter(xy[:,0], xy[:,1])
for i, s in enumerate(sents):
    plt.annotate(s[:18], xy[i])
plt.savefig("emb.png", dpi=120)
Run with: python 03_02_lab.py

Guided steps

Work through each step in order. Stop when your output matches the expected artifact — then push your work to your lab folder.

  1. 1
    Build a semantic search CLI

    Do — Embed 50 short docs with bge-small-en-v1.5, then accept a query from stdin and print the top-3 docs by cosine similarity.

    Expected artifact — Querying 'how does attention work' returns the doc about transformers as rank 1.

    Run free:
  2. 2
    Bias inspection

    Do — Compute embeddings for ['doctor', 'nurse', 'engineer', 'teacher'] and cluster against ['man','woman']. Plot the projections.

    Expected artifact — bias.png showing visible directional clustering — a finding to discuss in your report.md.

    Run free:
  3. 3
    Embedding drift

    Do — Re-embed the same 20 sentences using two different models. Compute pairwise correlation of the similarity matrices.

    Expected artifact — Spearman ρ > 0.7 between models, printed to console.

    Run free:

Authoritative sources

Independent primary sources used to check and corroborate the guidance on this page.