SCLog - Source code

SCLog - Universal Segmentation Tool for Log Parsing This is the source code of the SCLog tool, which implements the universal segmentation approach for log parsing, proposed in the paper "Towards Universal Segmentation for Log Parsing", ICPC 2026. If you use our tool, please kindly cite out paper:@inproceedings{le2026towards, title={Towards Universal Segmentation for Log Parsing}, author={Le, Van-Hoang and Bianculli, Domenico and Nguyen, Huy-Trung}, booktitle={2026 IEEE/ACM 34th International Conference on Program Comprehension (ICPC)}, year={2026}, organization={IEEE/ACM}}Requirements* python 3.11+Installationcd sclog pip install -e . To test the installation, run: pytest tests/test.py Expected output ============================== test session starts ============================== platform darwin -- Python 3.13.7, pytest-9.0.2, pluggy-1.6.0rootdir: /Users/vanhoang.le/Documents/GitLab/SVV/tools/s3log configfile: pyproject.tomlplugins: anyio-4.10.0 collected 2 items tests/test.py .. [100%] ============================== 2 passed in 0.03s ==============================UsageSCLog is used for segmentation of log messages. It takes a log line (content) as input and outputs a set of tokens (segments).Example:To segment a log line: from sclog import segmentation log_line = "audit(11382.749): item=1 name=/dev/pts/0 inode=2 dev=88:00" segments = segmentation(log_line) print(segments) # Output: ['audit', '(', '11382.749', ')', ':', ' ', 'item', '=', '1', ' ', 'name', '=', '/dev/pts/0', ' ', 'inode', '=', '2', ' ', 'dev', '=', '88:00']To compute context similarity between two log lines: from sclog import segmentation, context_similarity log1 = "audit(11382.749): item=1 name=/dev/pts/0 inode=2 dev=88:00" log2 = "audit(11382.486): item=0 name=/initrd inode=1021 dev=00:00" similarity = context_similarity(segmentation(log1), segmentation(log2)) print(similarity[0]) # Output: float value in [0, 1] representing the similarity score (in this case, 0.65)