PhD Research

PhD Thesis: Knowledge and Error Profile: A Context-based Meta-Analysis Framework for Synergistic Error Sampling in Active Learning

Active learning typically uses classification error as a unidimensional signal. This thesis proposed that a semantic interpretation of error as in cognitive science such as distinguishing between uncertainty (incomplete information), distortion (misrepresentation), and absence (lack of information), offers a richer basis for query selection.

Key Contributions

• Cross-disciplinary mapping of error taxonomy: Mapped a semantically rich taxonomy of error from cognitive science to active learning.
• Novel framework: Developed a meta-analysis framework that leverages the synergies between three semantic error types from cognitive science (uncertainty, distortion, absence) to select query batches for active learning. The framework also introduces a novel context-based error aggregation method and two novel re-ranking heuristics for making the final selection.
• Empirical validation: Showed KEP outperforms query by committee and random sampling. Demonstrated that optimal error synergy is dataset-dependent, there is no universal error synergy across datasets.
• Best practices: Based on the empirical evaluation, identified best practices for applying the KEP framework in practice.