SCOUT: Detecting Semantic Divergence Across Catalog Backends in Metadata Federation

Read full text Summarise with VIRAT
Authors: SCOUT
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

SCOUT: Semantic Inconsistency Detector for Catalog Federation Description SCOUT is an external, black-box testing framework that systematically detects semantic divergence in metadata catalog federation systems. It sends identical metadata operations to multiple federation platforms and catalog backends via REST APIs, then compares outcomes using formal oracle predicates to find where the unified API abstraction breaks. Key Features Three probe strategies: Semantic Space Exploration (S1), Invariant-Driven Stressing (S2), Path Differential Probing (S3) Three oracle predicates: cross-catalog consistency (pi_cross), cross-layer consistency (pi_layer), semantic invariants (pi_inv) Three federation platforms: Apache Gravitino, Databricks Unity Catalog OSS, Apache Polaris Five catalog backends (via Gravitino): Hive Metastore, Apache Iceberg, Apache Doris, Apache Paimon, MySQL Fully external -- no source code modification or internal instrumentation required Project Structure SCOUT/ # Detection framework (Python) ├── main.py # CLI entry point ├── config.py # Environment configuration ├── model/ # Data model (types, operations, traces, violations) ├── federation/ # Federation platform adapters (REST) │ ├── gravitino.py # Apache Gravitino adapter │ ├── unity_catalog.py # Unity Catalog OSS adapter │ └── polaris.py # Apache Polaris (Iceberg REST) adapter ├── backend/ # Direct backend inspectors (DB connections) │ ├── hive.py # HMS metastore_db via pymysql │ ├── iceberg.py # Iceberg JDBC catalog via SQL │ ├── mysql.py # MySQL information_schema │ └── postgres.py # PostgreSQL pg_catalog ├── probe/ # Probe generation strategies │ ├── strategy1_space.py # S1: Semantic Space Exploration │ ├── strategy2_invariant.py # S2: Invariant-Driven Stressing │ ├── strategy3_path.py # S3: Path Differential Probing │ └── strategy_random.py # Random baseline ├── oracle/ # Oracle predicates │ ├── pi_cross.py # Cross-catalog consistency │ ├── pi_layer.py # Cross-layer consistency │ ├── pi_inv.py # Semantic invariants │ └── pfs.py # Precision Fidelity Score ├── engine/ │ └── runner.py # Probe execution engine └── output/ └── reporter.py # JSON/CSV/Markdown reporting helm/ # Kubernetes Helm chart for experimental platform ├── Chart.yaml ├── values.yaml └── templates/ # Deployments for all services ├── gravitino/ # Gravitino federation server ├── hive/ # Hive Metastore ├── doris/ # Apache Doris ├── mysql/ # MySQL backend ├── postgresql/ # PostgreSQL backend ├── hdfs/ # HDFS NameNode + DataNode ├── polaris/ # Apache Polaris ├── unitycatalog/ # Unity Catalog OSS ├── spark/ # Apache Spark (workload driver) ├── trino/ # Trino (workload driver) ├── prometheus/ # Metrics collection └── grafana/ # Dashboards reproduce_all.sh # Run all paper experiments verify_results.py # Validate results against paper claimsrequirements.txtLICENSE Prerequisites Kubernetes cluster (tested on k8s 1.28+) Helm 3 for platform deployment Python 3.10+ with dependencies: pip install requests pymysql psycopg2-binary click Deploying the Experimental Platform # Deploy all services to a Kubernetes namespace kubectl create namespace metalake helm install scout-platform helm/ -n metalake # Verify all pods are running kubectl get pods -n metalake # Set up port-forwarding for local access kubectl port-forward -n metalake deploy/gravitino 8090:8090 & kubectl port-forward -n metalake deploy/mysql 3306:3306 & kubectl port-forward -n metalake deploy/unitycatalog 8080:8080 & kubectl port-forward -n metalake deploy/polaris 8181:8181 & kubectl port-forward -n metalake deploy/postgresql 5432:5432 & Running SCOUT # Strategy 1: Semantic Space Exploration (Gravitino, 5 catalog backends, 11 types) python -m SCOUT.main \ --platform gravitino \ --strategy semantic_space \ --catalogs catalog_hive,catalog_iceberg,catalog_doris,catalog_paimon,catalog_mysql \ --type-set ext_11 \ --output-dir results/s1/ # Strategy 2: Invariant-Driven Stressing python -m SCOUT.main \ --platform gravitino \ --strategy invariant \ --catalogs catalog_hive,catalog_iceberg,catalog_mysql \ --output-dir results/s2/ # Strategy 3: Path Differential Probing python -m SCOUT.main \ --platform gravitino \ --strategy path_diff \ --catalogs catalog_hive,catalog_iceberg,catalog_mysql \ --output-dir results/s3/ # Cross-platform: Unity Catalog OSS python -m SCOUT.main \ --platform unity_catalog \ --strategy semantic_space \ --catalogs unity \ --type-set ext_11 \ --skip-backend \ --output-dir results/uc/ # Cross-platform: Apache Polaris python -m SCOUT.main \ --platform polaris \ --strategy semantic_space \ --catalogs polaris_test \ --type-set ext_11 \ --skip-backend \ --output-dir results/polaris/ # All strategies at once python -m SCOUT.main \ --platform gravitino \ --strategy all \ --catalogs catalog_hive,catalog_iceberg,catalog_mysql \ --output-dir results/all/ CLI Options Option Description Default --platform gravitino, unity_catalog, polaris gravitino --strategy semantic_space, invariant, path_diff, random, all semantic_space --catalogs Comma-separated catalog names catalog_hive,catalog_iceberg,catalog_mysql --type-set core_3, core_6, ext_8, ext_11 core --output-dir Output directory results/ --skip-backend Skip direct backend inspection (pi_layer) false --n-probes Number of probes for random strategy 0 --seed Random seed 42 --timing Report execution time false --verbose Debug logging false Output Each run produces the following files in the output directory: File Description violations.json Full violation records with traces summary.csv One row per violation divergence_matrix.csv Operation x catalog acceptance matrix probe_outcomes.csv Per-probe accept/reject/partial classification pfs_summary.csv Precision Fidelity Score per catalog all_traces.json Raw trace data for all probes report.md Human-readable summary timing.json Execution timing (with --timing) Reproducing Paper Results # Run all experiments from the paper ./reproduce_all.sh --all # Or run individual scopes ./reproduce_all.sh --scope-a # Gravitino intra-federation (S1+S2+S3) ./reproduce_all.sh --scope-b # Unity Catalog + Polaris ./reproduce_all.sh --random # Random baseline comparison ./reproduce_all.sh --scalability # Type space + catalog count sweep # Validate results against paper claims python verify_results.py results/run_ Environment Configuration All endpoints and credentials are configurable via environment variables: Variable Description Default GRAVITINO_BASE_URL Gravitino REST API http://gravitino:8090/api/metalakes/metalake_demo UC_BASE_URL Unity Catalog REST API http://unitycatalog:8080/api/2.1/unity-catalog POLARIS_BASE_URL Polaris REST API http://polaris:8181/api MYSQL_HOST MySQL host mysql MYSQL_PORT MySQL port 3306 MYSQL_USER / MYSQL_PASSWORD MySQL credentials mysql / mysql POSTGRESQL_HOST PostgreSQL host postgresql POSTGRESQL_PORT PostgreSQL port 5432 POSTGRESQL_USER / POSTGRESQL_PASSWORD PostgreSQL credentials postgres / postgres License SCOUT/ -- MIT License helm/ -- Apache License 2.0 (derived from Apache Gravitino Playground)

Fields of Study & Subjects

Software Testing and Debugging Techniques Advanced Database Systems and Queries Software System Performance and Reliability

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
26 March 2026
Language
EN