LSA with Randomized SVD

What I explored

This course project started with a practical question: how much useful topic structure survives when a large term-document matrix is compressed? I implemented randomized Singular Value Decomposition myself, used it for latent semantic analysis, and compared the approximation and runtime with classical SVD instead of treating the decomposition as a black box.

Dataset

20 Newsgroups — ~18,000 Usenet posts across 20 topic categories. The dataset was preprocessed with Snowball Stemming and custom stop-word filtering.

What I added around the algorithm

• Power iterations to tighten the randomized low-rank approximation.
• A sweep over cluster counts, compared with the Silhouette Score.
• A desktop view with Plotly charts and semantic search.
• A CLI path with Matplotlib output for running without the interface.
• Text cleanup with Snowball stemming and a custom stop-word list.

Under the hood

The rSVD Algorithm

Classical SVD scales as O(mn·min(m,n)) — prohibitive for large matrices. The randomized approach projects the matrix onto a lower-dimensional subspace first, then applies SVD to the smaller matrix, achieving near O(mnk) complexity for rank-k approximation. Power Iterations further refine the approximation quality.

Clustering Workflow

After dimensionality reduction, documents are clustered in the latent concept space. The Silhouette Score is computed across a range of k values to automatically identify the most cohesive grouping.

Built with

Language: Python 3
Libraries: NumPy, SciPy, Scikit-Learn, Matplotlib, Plotly, NLTK
Techniques: Randomized SVD, LSA, K-Means, Silhouette Score, TF-IDF, Snowball Stemming
Interfaces: GUI (Plotly + Tkinter), CLI (Matplotlib)