Evaluation of Context-Oriented Search Architecture Reconstruction: Offline Validation of Quality and Performance

Fedor  Krasnov

ПРЕПРИНТ

Эта статья является препринтом и не была отрецензирована.
О результатах, изложенных в препринтах, не следует сообщать в СМИ как о проверенной информации.

Evaluation of Context-Oriented Search Architecture Reconstruction: Offline Validation of Quality and Performance

F. Krasnov

2026-01-01

https://doi.org/10.24108/preprints-3114198

In the segment of electronic sales of do-it-yourself (DIY) repair products, search quality significantly depends on correctly accounting for regional assortment restrictions and differences between client types. Traditional search solutions apply these restrictions at the post-search filtering stage, leading to additional computational costs, unstable response times, and inconsistencies between search, suggestions, and the catalog. This work proposes an architecture where the search query is first classified into the most probable product category, taking into account regional and user context. The obtained category is used to select a precomputed catalog index bucket $(\hat{c}, r, u)$, within which full-text search is then performed. This order shifts the assortment availability check to offline indexing, eliminating resource-intensive online filtering and ensuring predictable processing delays under high loads. The architecture's efficiency was evaluated offline by comparing users' actual purchases with the positions of the same products obtained through simulated reproduction of historical contexts. The results show that the context-oriented reconstruction of the search pipeline improves the quality of catalog ranking by nDCG@12 by 3 percentage points and simultaneously reduces service latency to around 2 ms, confirming the practical applicability of the proposed approach.

Ссылка для цитирования:

Krasnov F. 2026. Evaluation of Context-Oriented Search Architecture Reconstruction: Offline Validation of Quality and Performance. PREPRINTS.RU. https://doi.org/10.24108/preprints-3114198

Список литературы

1. Trappolini G. и др. Redefining Retrieval Evaluation in the Era of LLMs // ArXiv. 2025. Т. abs/2510.21440.

2. Qin Z. и др. KDD-2023 Workshop on Decision Intelligence and Analytics for Online Marketplaces // Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023. С. 5878–5879.

3. Joachims T. Optimizing search engines using clickthrough data // Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, USA: Association for Computing Machinery, 2002. С. 133–142.

4. Joachims T. Learning from User Interactions // Proceedings of the Eighth ACM International Conference on Web Search and Data Mining. New York, NY, USA: Association for Computing Machinery, 2015. С. 137–138.

5. Phophalia A. A survey on learning to rank (letor) approaches in information retrieval // 2011 Nirma University International Conference on Engineering. IEEE, 2011. С. 1–6.

6. Chavhan S., Raghuwanshi M., Dharmik R. Information retrieval using machine learning for ranking: a review // Journal of Physics: Conference Series. IOP Publishing, 2021. Т. 1913. С. 012150.

7. Craswell N. и др. An experimental comparison of click position-bias models // Web Search and Data Mining. 2008.

8. Hofmann K., Whiteson S., Rijke M. de. A probabilistic method for inferring preferences from clicks // International Conference on Information and Knowledge Management. 2011.

9. Dupret G. E., Piwowarski B. A user browsing model to predict search engine click data from past observations. // Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. 2008. С. 331–338.

10. Li L. и др. Counterfactual estimation and optimization of click metrics in search engines: A case study // Proceedings of the 24th International Conference on World Wide Web. 2015. С. 929–934.

11. Chuklin A., Markov I., De Rijke M. Click models for web search. Springer Nature, 2022.

12. Basalla M. и др. On latency of e-commerce platforms // Journal of Organizational Computing and Electronic Commerce. Taylor & Francis, 2021. Т. 31, № 1. С. 1–17.

13. Hwang S.-W. и др. Prediction and predictability for search query acceleration // ACM Transactions on the Web (TWEB). ACM New York, NY, USA, 2016. Т. 10, № 3. С. 1–28.

14. Kumar S. и др. 2P-BEnc: A two-phase information retrieval and ranking system based on the BERT encoder // Ain Shams Engineering Journal. Elsevier, 2026. Т. 17, № 1. С. 103853.

15. Cañamares R., Castells P., Moffat A. Offline evaluation options for recommender systems // Information Retrieval Journal. Springer, 2020. Т. 23, № 4. С. 387–410.

16. Ma X. и др. Knowing Before Seeing: Incorporating Post-retrieval Information into Pre-retrieval Query Intention Classification // International Conference on Knowledge Science, Engineering and Management. Springer, 2023. С. 3–15.

17. Jiang Z. и др. A large-scale evaluation for log parsing techniques: How far are we? // Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 2024. С. 223–234.

18. Huang Z. и др. Interaction-Data-guided Conditional Instrumental Variables for Debiasing Recommender Systems // arXiv preprint arXiv:2408.09651. 2024.

19. Clarke C. L., Diaz F., Arabzadeh N. Preference-based offline evaluation // Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 2023. С. 1248–1251.

20. Pang M. и др. Generative Retrieval and Alignment Model: A New Paradigm for E-commerce Retrieval // Companion Proceedings of the ACM on Web Conference 2025. 2025. С. 413–421.

ПРЕПРИНТ

ИНФОРМАЦИЯ

Использование куки-файлов