Loading...
Loading...
It may be waking up.
Faculty Twin
Keys live in environment variables on the server. This only shows whether each one is set.
The model that writes narration. Embeddings stay on Voyage whatever you pick here.
Pick from the list or type any model id the provider accepts.
Three kinds of voice. My clone and the other ElevenLabs voices cost ElevenLabs credits per character; the Microsoft voices are free. Students see a label that matches the voice: "AI voice made from my recordings." only for my clone, "AI voice (a stock voice, not mine)." for any other voice, and no voice label for captions only. Changing the voice means each answer's audio is made fresh the next time it plays.
The fallback voice is labeled "a stock voice, not mine" while it speaks. Only applies when an ElevenLabs voice is chosen.
Uploads go straight from this browser to private storage. A local processing worker on the computer that holds the private lecture archive picks them up, rebuilds the index, uploads it, and updates the status here.
| Course | Session | Kind | File | Status | Updated | Actions |
|---|
Every question is scored against every slide and Canvas page with cosine similarity (the score from rank(), the same Top score shown in Activity). Measured so far, across two 10-question checks: on-topic questions scored 0.543 to 0.712 and off-topic ones 0.377 to 0.509. Higher thresholds decline more real questions; lower ones let off-topic questions through. How the thresholds were chosen. After a change, run an eval to check it. Changes reach every server within 30 seconds.
How these numbers work. Run a new test: see Testing and scores.
| When | Covered | Question | Top score | Latency | Model |
|---|
Every instruction a model gets. Pick one, edit it, test the draft on a question, then save it with a note. Every saved version is kept, and any one can be restored.
These prompts change what students hear. A saved prompt is live for new questions within 30 seconds. The safety checks in code still run on every answer, whatever a prompt says: grounding in the slides and the injection check, word and character caps, slide id checks, PG language, access codes, and no student names. An answer that fails them falls back to the slide's speaker notes.
Saved with this version in the history.
Runs one question through the real path with the text in the editor. Nothing is saved, and the same safety checks apply. Counts against your question limits.
How students use the twin, what they ask about, and what it costs. Spend is an estimate: tokens and characters times the price table at the bottom, not a bill. Smoke checks, evals and model tests are left out of the student numbers unless you show them; their spend is always counted.
Scroll here or press Refresh to load analytics.
One call to a small model on the active provider groups the newest student questions (scrubbed text only, test traffic left out) into themes. At most 300 questions and 10 runs a day. Each run is kept in private storage.
Defaults were read from each provider's public price page on the date shown and are marked "verify". Edit a price and save; models you add here are used for the estimate. OpenRouter models not listed use OpenRouter's live price.
Grade the twin on real student questions. Pick the models that answer (each runs the real ask path, with no change to the live model) and the models that judge (each scores six dimensions from 1 to 5 and gives a pass or fail). How evals work.
Question text is de-identified and private: it lives only in the private bucket, shows only here behind the admin passcode, and never goes in git or a public link.
| Id | Category | Expected | Question | How Ben or the TA replied | Actions |
|---|
Scores eight invented answers whose right verdict is known (a grounded answer, an invented fact, a promised extension, a wrong decline, and so on). Don't trust a judge on real answers until it meets them.
| Judge | Cases met | Missed | When | From |
|---|
| Question | Category | Answering model | Top score (similarity, 0–1) | Outcome | Judges: verdict, then six scores (1–5, 5 best) |
|---|
Pass rates for every answering model under every judge, how lenient each judge is, how often judges agree, and mean scores per dimension. Pick one run, or all runs together. (This replaces "Model performance" under Analytics.)
Every eval question across runs, sorted by the share of judgements that failed it. Filter by run or by the model that wrote the answer; open a question to see each judge's verdict, scores and reason, and how much the judges agreed.
When a student reports a broken quiz, a broken submission, or an API key out of credits, the twin texts my cell with the course, the Canvas item, and a short scrubbed quote (no names, no ids). Texts go through Twilio. If Twilio is not set up or a text fails, the alert still shows here.
| When (ET) | Course | Type | Item | Reports | Delivery | Student said (scrubbed) |
|---|
The model fills in a small slide spec (bullets, a diagram, a Python example, or a comparison) and this page draws it with our own code, so it never writes markup. Every one is labeled "AI-drawn slide, not from my course". Use them for class prep or discussion, or, when you approve them and turn it on below, in answers.