Kazakh scientific publications dataset from Semantic Scholar (2000–2025)

Dataset 1: Kazakh academic abstracts corpus (kazakh_abstracts_10468.xlsx) This dataset contains a curated collection of 10,468 scientific article abstracts specifically focused on Kazakhstan and the Kazakh language. The data was programmatically collected from the Semantic Scholar API using targeted queries related to Kazakhstan Data structure: paperId: Unique identifier from Semantic Scholar. title: The title of the scientific paper. abstract_kk: The full text of the abstract in Kazakh. year: Publication year (ranging from 2000 to 2025). query: The search term used to retrieve the record. Dataset 2: Kazakh abstract pairs for duplicate detection (kazakh_abstractpairs_11851.xlsx) This dataset consists of 11,851 pairs of Kazakh-language academic abstracts, developed specifically for research in duplicate detection. The dataset provides a balanced mix of positive (duplicate/paraphrased) and negative (distinct) pairs. To ensure high quality and complexity, the authors utilized a hybrid approach combining real-world data with controlled synthetic augmentation: Near-duplicates: Generated using a custom paraphrasing engine that performs synonym replacement, sentence shuffling, and structural rephrasing based on Kazakh linguistics. Similarity levels: Pairs are categorized into "high", "medium", and "low" similarity based on TF-IDF and Cosine Similarity scores. Negative samples: Formed by pairing unrelated abstracts to provide "non-duplicate" labels for machine learning training. Data structure: abstract_a / abstract_b: The pair of texts to be compared. similarity_score: The computed cosine similarity value (0.0 to 1.0). label: Binary indicator (1 for duplicates/paraphrases, 0 for different texts). pair_type: Qualitative description of similarity (e.g., high_similarity, different_abstract). rephrase_level: The intensity of the transformation (very_light, light, medium, strong, or strong_shorten).