UDC 004.89
COMPARATIVE ANALYSIS OF SEMANTIC SEARCH METHODS FOR INTELLIGENT
RECOMMENDATION SYSTEMS ON A COLLECTION OF MOVIE DESCRIPTIONS
A. M. Gostin, PhD (in technical sciences), associate professor, CAD Department, RSREU, Ryazan, Russia;
orcid: 000-0001-6550-2982, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
S. V. Shoshnikov, student, RSREU, Ryazan, Russia;
orcid: 0009-0006-5863-6935, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
Four semantic search methods for recommendation systems are experimentally compared on a collection
of 2000 TMDB movie descriptions: TF-IDF, pre-trained SBERT, fine-tuned BERT with Triplet Loss, and
a hybrid combining neural and lexical signals. First, the effect of training set size (100 to 2000 items) on
fine-tuned BERT quality is investigated. The relationship turns out to be nonlinear: the main gains in Recall@
1 (54 %) and MAP (25,7 %) only materialize at 2000 training samples, while smaller sets produce
marginal improvements. At best configuration, fine-tuned BERT outperforms TF-IDF by a factor of 1,5 in
MRR (0,284 vs. 0,188). Pre-trained SBERT without any domain adaptation surpasses TF-IDF by 8,5 % in
MRR. An unexpected finding concerns the hybrid method: despite combining three signals, it scores below
standalone SBERT on Recall@10, suggesting that naive weight selection can degrade rather than improve
retrieval quality. The obtained results can be used in the design of recommendation systems where the quality
of semantic search over text descriptions determines the final relevance of recommendations. A software
system called SemanticBench with a graphical interface has been developed for conducting the experiments
and visualizing the results.
Key words: semantic search, recommendation systems, BERT, language model fine-tuning, Triplet Loss,
TF-IDF, information retrieval, search quality metrics, contrastive learning