DataTypical

When analyzing a dataset, we often ask: "Which instances are most important?" This seemingly simple question has no single answer. Importance depends on perspective, context, and purpose. DataTypical provides three distinct but complementary answers, each capturing a different notion of significance: Significance Type Philosophical Basis Question Answered Archetypal Objective Which instances occupy extreme positions? Prototypical Representative Which instances best represent the population? Stereotypical Subjective Which instances match my target criterion? These are not competing definitions but complementary perspectives that together provide a complete picture of dataset structure. Why Multiple Perspectives Matter Consider a drug discovery dataset with molecular descriptors and activity measurements: Archetypal compounds have extreme combinations of properties: Example: Highest lipophilicity combined with lowest polarity Value: Identify boundary cases, understand limits of chemical space Use: Novelty detection, coverage testing, outlier characterization Prototypical compounds are representative of the dataset: Example: Average across all properties, common structural motifs Value: Select diverse panel for screening, understand typical behavior Use: Dataset summarization, active learning, library curation Stereotypical compounds match a specific target: Example: Lowest IC50 (most potent) Value: Direct optimization, find best candidates Use: Hit identification, constraint satisfaction, target-based selection All three are "significant" but for different reasons. The choice depends on your scientific question, not on which metric produces the highest scores. The Explanation Problem After identifying that a sample is significant (high archetypal/prototypical/stereotypical rank), the natural question is: "WHY is this sample significant?" Shapley values provide a principled answer by attributing the significance score to individual features. Formative Instance Discovery The Discovery When shapley_mode=True, DataTypical doesn't just explain WHY instances are significant; it discovers a fundamentally new type of instance: Formative instances: Samples that CREATE the significance structure of your dataset. This is the core innovation in v0.6+, distinguishing between: Actual significance: Samples that ARE archetypal/prototypical/stereotypical Formative significance: Samples that CREATE the archetypal/prototypical/stereotypical structure The Core Distinction Perspective Question Measurement Meaning Actual Which samples ARE significant? Direct evaluation (scores) Positional property Formative Which samples CREATE structure? Shapley values (contributions) Causal property Why Formative Matters The Problem: Traditional significance metrics tell you WHERE samples are, not how IMPORTANT they are to the structure. They are the "hidden influencers" in your data.