This article analyzes modern models and methods of intelligent processing of web documents based on artificial intelligence technologies. Algorithms for automatic processing, clustering, classification, and detection of unnamed events in textual, graphical, and structural data contained in web documents are considered. During the research, machine learning, deep learning, semantic analysis, and DOM tree-based models are studied, and their advantages and disadvantages are compared. In addition, the efficiency of K-means, DBSCAN, Hierarchical Clustering, and BiDist methods in intelligent processing of web documents is analyzed. As a result of the study, an effective model for automatic analysis of web data is proposed.
Models and Methods of Intelligent Processing of Web Documents Based on Artificial Intelligence Technologies
DOI:
Keywords:
Artificial intelligence
web document
DOM model
clustering
classification
Abstract
References
Han J., Kamber M., Pei J. Data Mining: Concepts and Techniques. Morgan Kaufmann, 2012.
Russell S., Norvig P. Artificial Intelligence: A Modern Approach. Pearson Education, 2021.
Manning C., Raghavan P. Introduction to Information Retrieval. Cambridge University Press, 2008.
Aggarwal C. Machine Learning for Text. Springer, 2018.
Witten I., Frank E. Data Mining: Practical Machine Learning Tools and Techniques. Elsevier, 2016.
Ester M., Kriegel H.P., Sander J. “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise”, 1996.
Blei D. “Probabilistic Topic Models”, Communications of the ACM, 2012.
Goodfellow I. Deep Learning. MIT Press, 2016.