Primary speaker:
Furquan Hassan, AI Special Projects Developer, Academic Research and Collaborative Technologies, ITS
Description:
The University of Toronto's AI Task Force has established a bold, human-centric roadmap for becoming an "AI-ready" institution. Central to this vision is the imperative to adopt generative AI safely while fiercely protecting academic integrity, data privacy and explainable decision-making. However, the probabilistic nature of modern large language models presents a unique administrative challenge: how can a university empirically guarantee that these fundamentally opaque systems continuously adhere to its strict ethical guardrails?
This presentation argues that AI observability is the indispensable technical layer required to operationalize U of T's digital trust frameworks. We will explore how advanced telemetry and continuous monitoring transition the Task Force's recommendations from theoretical policy documents into verifiable technical realities. Specifically, we will detail how to instrument the university's proposed "AI Kitchen" with the observability and evaluation tools necessary to rigorously benchmark new models for factual accuracy, safety guardrails and acceptable hallucination rates prior to their campus-wide deployment.
Furthermore, the presentation will demonstrate how observability platforms directly empower the university's AI Adoption Table. By replacing static assumptions with real-time empirical data regarding system latency, operational costs and statistical bias drift, the advisory committee can confidently govern AI use cases and adjust policies on the fly. Ultimately, by combining its robust governance structures with the continuous accountability of AI observability, U of T will actively dismantle the AI "black box" and secure its position as a global pioneer in responsible, trusted educational innovation.
