forked from fasiha/ebisu.js
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
175 lines (174 loc) · 18.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
<head>
<meta charset="utf-8" />
<title>Ebisu.js</title>
<meta name="description" content="JavaScript port of the Ebisu public-domain library for Bayesian quiz scheduling." />
<meta name="twitter:card" value="summary">
<meta property="og:title" content="Ebisu.js" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://fasiha.github.io/ebisu.js/" />
<meta property="og:image" content="https://fasiha.github.io/ebisu.js/client/betarng.png" />
<meta property="og:description" content="Public-domain Bayesian quiz scheduling, JavaScript-style." />
<link rel="stylesheet" href="client/modest.css">
<link rel="stylesheet" href="client/custom.css">
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
code.sourceCode > span { display: inline-block; line-height: 1.25; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode { white-space: pre; position: relative; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
code.sourceCode { white-space: pre-wrap; }
code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ background-color: #f8f8f8; }
@media screen {
code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ef2929; } /* Alert */
code span.an { color: #8f5902; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #c4a000; } /* Attribute */
code span.bn { color: #0000cf; } /* BaseN */
code span.cf { color: #204a87; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4e9a06; } /* Char */
code span.cn { color: #000000; } /* Constant */
code span.co { color: #8f5902; font-style: italic; } /* Comment */
code span.cv { color: #8f5902; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #8f5902; font-weight: bold; font-style: italic; } /* Documentation */
code span.dt { color: #204a87; } /* DataType */
code span.dv { color: #0000cf; } /* DecVal */
code span.er { color: #a40000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #0000cf; } /* Float */
code span.fu { color: #000000; } /* Function */
code span.im { } /* Import */
code span.in { color: #8f5902; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #204a87; font-weight: bold; } /* Keyword */
code span.op { color: #ce5c00; font-weight: bold; } /* Operator */
code span.ot { color: #8f5902; } /* Other */
code span.pp { color: #8f5902; font-style: italic; } /* Preprocessor */
code span.sc { color: #000000; } /* SpecialChar */
code span.ss { color: #4e9a06; } /* SpecialString */
code span.st { color: #4e9a06; } /* String */
code span.va { color: #000000; } /* Variable */
code span.vs { color: #4e9a06; } /* VerbatimString */
code span.wa { color: #8f5902; font-weight: bold; font-style: italic; } /* Warning */
</style>
</head>
<h1 id="ebisujs">Ebisu.js</h1>
<p>This is a JavaScript port of the original Python implementation of <a href="https://github.com/fasiha/ebisu">Ebisu</a>, a public-domain library intended for use by quiz apps to intelligently handle scheduling. See <a href="https://github.com/fasiha/ebisu">Ebisu’s literate documentation</a> for <em>all</em> the details! This document just contains a quick guide to how things work.</p>
<p>Browse this library’s <a href="https://github.com/fasiha/ebisu.js">GitHub repo</a>. Read this <a href="https://fasiha.github.io/ebisu.js/">document in HTML</a> (cool interactive demos!).</p>
<h2 id="install">Install</h2>
<p><strong>Node.js</strong> First,</p>
<pre><code>$ npm install --save ebisu-js</code></pre>
<p>Then, in your code,</p>
<div class="sourceCode" id="cb2"><pre class="sourceCode js"><code class="sourceCode javascript"><span id="cb2-1"><a href="#cb2-1"></a><span class="kw">var</span> ebisu <span class="op">=</span> <span class="at">require</span>(<span class="st">'ebisu-js'</span>)<span class="op">;</span></span></code></pre></div>
<p><strong>Browser</strong> Two choices. For maximal compatibility, download the ES5-compatible <a href="https://raw.githubusercontent.com/fasiha/ebisu.js/gh-pages/dist/ebisu.min.js"><code>dist/ebisu.min.js</code></a> for the browser (13 KB uncompressed, 5 KB after gzip), then in your HTML:</p>
<div class="sourceCode" id="cb3"><pre class="sourceCode html"><code class="sourceCode html"><span id="cb3-1"><a href="#cb3-1"></a><span class="kw"><script</span><span class="ot"> type=</span><span class="st">"text/javascript"</span><span class="ot"> src=</span><span class="st">"ebisu.min.js"</span><span class="kw">></script></span></span></code></pre></div>
<p>If you want to target ES6-compatible browsers only, download and use <a href="https://raw.githubusercontent.com/fasiha/ebisu.js/gh-pages/dist/ebisu.min.es6.js"><code>dist/ebisu.min.es6.js</code></a>. This is 5 KB uncompressed, 2.5 KB after gzip.</p>
<h2 id="api-howto">API howto</h2>
<p>Let’s start working immediately with code and we’ll explain as we go.</p>
<p>First, in Node, e.g.,</p>
<div class="sourceCode" id="cb4"><pre class="sourceCode js"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1"></a><span class="kw">var</span> ebisu <span class="op">=</span> <span class="at">require</span>(<span class="st">'ebisu-js'</span>)<span class="op">;</span></span></code></pre></div>
<p>or if you’re developing in this repo,</p>
<div class="sourceCode" id="cb5"><pre class="sourceCode js"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1"></a><span class="kw">var</span> ebisu <span class="op">=</span> <span class="at">require</span>(<span class="st">'./index'</span>)<span class="op">;</span></span></code></pre></div>
<p>(The <code>ebisu</code> module is loaded in <a href="https://fasiha.github.io/ebisu.js">this webpage</a>. Pop open your JavaScript console to try it out.)</p>
<h3 id="memory-model">Memory model</h3>
<p>Now, it’s important to know that Ebisu is a very narrowly-scoped library: it aims to answer just two questions:</p>
<ul>
<li>given a set of facts that a student is learning, which is the most (or least) likely to be forgotten?</li>
<li>After the student is quizzed on one of these facts, how does the result get incorporated into Ebisu’s model of that fact’s memory strength?</li>
</ul>
<p>Ebisu doesn’t concern itself with what these facts are, what they mean, nor does it handle <em>storing</em> the results of reviews. The external quiz app, at a minimum, stores a probability <em>model</em> with each fact’s memory strength, and it is this <em>model</em> that Ebisu transforms into predictions about recall probability or into <em>new</em> models after a quiz occurs.</p>
<p>Create a <em>default</em> model to assign newly-learned facts:</p>
<div class="sourceCode" id="cb6"><pre class="sourceCode js"><code class="sourceCode javascript"><span id="cb6-1"><a href="#cb6-1"></a><span class="kw">var</span> defaultModel <span class="op">=</span> <span class="va">ebisu</span>.<span class="at">defaultModel</span>(<span class="dv">24</span>)<span class="op">;</span></span>
<span id="cb6-2"><a href="#cb6-2"></a><span class="co">// Also ok: `ebisu.defaultModel(24, 4)` or even `ebisu.defaultModel(24, 4, 4)`.</span></span>
<span id="cb6-3"><a href="#cb6-3"></a><span class="va">console</span>.<span class="at">log</span>(defaultModel)<span class="op">;</span></span></code></pre></div>
<p>This returns a three-element array of numbers: we’ll call them <code>[a, b, t]</code>.</p>
These three numbers describe the probability distribution on a fact’s recall probability. Specifically, they say that, <code>24</code> hours after review, we believe this fact’s recall probability will have a <code>Beta(a, b)</code> distribution, whose histogram looks like this, for a few thousand samples:
<div id="betarng-choo">
</div>
<div id="betarng-render">
</div>
<p>In the interactive graph above, that <strong>fourth</strong> slider above lets you say how much time has <em>actually</em> elapsed since this fact was last reviewed. If you move it to be <em>more</em> or <em>less</em> than 24 hours, you’ll see the bulk of the histogram move <em>left</em> or <em>right</em>, since the less time elapsed, the more likely the student remembers this fact.</p>
<p>You can also move the sliders for <code>a</code> and <code>b</code>. Move the two time sliders back to 24 hours and notice that when <code>a = b</code>, the distribution is centered around 0.5. In this case, <code>t</code> is a half-life, i.e., the length of time it takes for recall probability to drop to 50%. If this <code>a = b</code> is high, the histogram tightly clusters around 0.5. For small <code>a = b</code>, the histogram is very diffuse around 0.5.</p>
<blockquote>
<p>We use the <a href="https://en.wikipedia.org/wiki/Beta_distribution">Beta distribution</a>, and not some other probability distribution on numbers between 0 and 1, for <a href="https://en.wikipedia.org/wiki/Conjugate_prior">statistical reasons</a> that are indicated in depth in the <a href="https://fasiha.github.io/ebisu/#bernoulli-quizzes">Ebisu math</a> writeup.</p>
</blockquote>
<p>This should give you some insight into what those three numbers, <code>[4, 4, 24]</code> mean, and why you might want to customize them—you might want the half-life to be just two hours instead of a whole day, in which case you’d set <code>defaultModel</code> to <code>ebisu.defaultModel(2)</code>.</p>
<h3 id="predict-current-recall-probability-ebisupredictrecall">Predict current recall probability: <code>ebisu.predictRecall</code></h3>
<p>Given a set of models for facts that the student has learned, you can ask Ebisu to predict each fact’s recall probability by passing in its model and the currently elapsed time since that fact was last reviewed or quizzed via <code>ebisu.predictRecall</code>:</p>
<div class="sourceCode" id="cb7"><pre class="sourceCode js"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1"></a><span class="kw">var</span> model <span class="op">=</span> defaultModel<span class="op">;</span></span>
<span id="cb7-2"><a href="#cb7-2"></a><span class="kw">var</span> elapsed <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb7-3"><a href="#cb7-3"></a><span class="kw">var</span> predictedRecall <span class="op">=</span> <span class="va">ebisu</span>.<span class="at">predictRecall</span>(model<span class="op">,</span> elapsed<span class="op">,</span> <span class="kw">true</span>)<span class="op">;</span></span>
<span id="cb7-4"><a href="#cb7-4"></a><span class="va">console</span>.<span class="at">log</span>(predictedRecall)<span class="op">;</span></span></code></pre></div>
This function efficiently calculates the <em>mean</em> of the histogram of recall probabilities in the interactive demo above (it uses math, not histograms!). Below you can see what this function would return for different models.
<div id="predict-choo">
</div>
<div id="predict-render">
</div>
<p>A quiz app can call this function on each fact to find which fact is most in danger of being forgotten—that’s the one with the lowest predicted recall probability.</p>
<blockquote>
<p>N.B. In your app, you should omit the third argument, i.e., use <code>predictRecall(model, elapsed)</code>, which skips a final exponential and saves some runtime. (See the full API below.)</p>
<p>If your quiz app starts having thousands of facts, and it becomes computationally-burdensome to evaluate this function over and over again, you can build a look-up table containing a range of elapsed times and their predicted recall probabilities, then linearly-interpolate into it.</p>
</blockquote>
<h3 id="update-a-recall-probability-model-given-a-quiz-result-ebisuupdaterecall">Update a recall probability model given a quiz result: <code>ebisu.updateRecall</code></h3>
<p>Suppose your quiz app has chosen a fact to review, and tests the student. Out of a <code>total</code> number of trials, the student gets <code>successes</code> of them correct.</p>
<blockquote>
<p>Version 1 of Ebisu required <code>total=1</code>, i.e., binary quizzes. Version 2 relaxed this so <code>total</code> can be an integer greater than one, which models the case where each trial is a statistically-independent review of the fact under test. Note that this doesn’t mean you just ask the same fact multiple times!, since then the trials become highly dependent. Having <code>total</code> greater than 1 may make sense if, for example, the student is reviewing a verb conjugation, and conjugates the same verb in different sentences.</p>
</blockquote>
<div class="sourceCode" id="cb8"><pre class="sourceCode js"><code class="sourceCode javascript"><span id="cb8-1"><a href="#cb8-1"></a><span class="kw">var</span> model <span class="op">=</span> defaultModel<span class="op">;</span></span>
<span id="cb8-2"><a href="#cb8-2"></a><span class="kw">var</span> successes <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb8-3"><a href="#cb8-3"></a><span class="kw">var</span> total <span class="op">=</span> <span class="dv">1</span><span class="op">;</span></span>
<span id="cb8-4"><a href="#cb8-4"></a><span class="kw">var</span> elapsed <span class="op">=</span> <span class="dv">10</span><span class="op">;</span></span>
<span id="cb8-5"><a href="#cb8-5"></a><span class="kw">var</span> newModel <span class="op">=</span> <span class="va">ebisu</span>.<span class="at">updateRecall</span>(model<span class="op">,</span> successes<span class="op">,</span> total<span class="op">,</span> elapsed)<span class="op">;</span></span>
<span id="cb8-6"><a href="#cb8-6"></a><span class="va">console</span>.<span class="at">log</span>(newModel)<span class="op">;</span></span></code></pre></div>
<p>The new model is a new 3-array with a new <code>[a, b, t]</code>. The Bayesian update magic happens inside here: see here for <a href="https://fasiha.github.io/ebisu/#updating-the-posterior-with-quiz-results">the gory math details</a>.</p>
<h3 id="api-summary">API summary</h3>
<p>That’s it! That’s the entire API:</p>
<ul>
<li><code>ebisu.defaultModel(t, [a, [b]]) -> model</code> if you can’t bother to create a 3-array.</li>
<li><code>ebisu.predictRecall(model, elapsed, exact = false) -> number</code> predicts the current recall probability given a model and the time elapsed since the last review or quiz. If <code>exact</code>, then the returned value is actually a real probability. If <code>exact</code> is falsey, a final exponential is skipped and the returned value is the log-probability: this is the default because it makes things a bit faster.</li>
<li><code>ebisu.updateRecall(model, successes, total, elapsed) -> model</code> to update the model after a quiz session with <code>successes</code> out of <code>total</code> statistically-independent trials exercising the fact, and time after its last review.</li>
</ul>
<p>As a bonus, you can find the half-life (time for recall probability to decay to 50%), or actually, any percentile-life (time for recall probability to decay to any percentile):</p>
<ul>
<li><code>ebisu.modelToPercentileDecay(model, percentile = 0.5, coarse = false, tolerance = 1e-4) -> number</code>, where, if <code>coarse</code> is falsey (the default), the returned value is accurate to within <code>tolerance</code> (i.e., if the true half-life is 1 week, the returned value will be between 0.9999 and 1.0001). If <code>coarse</code> is truthy, the returned value is only roughly within a factor of two of the actual value.</li>
</ul>
<h2 id="building">Building</h2>
<p>We use <code>tape</code> for tests: run <code>npm test</code>. This consumes <code>test.json</code>, which came from the <a href="https://fasiha.github.io/ebisu/">Ebisu Python reference implementation</a>.</p>
<p>The version of this repo matches the Python reference’s version up to minor rev (i.e., Python Ebisu 1.0.x will match Ebisu.js 1.0.y).</p>
<p>We use Browserify followed by Google Closure Compiler to minify Ebisu for the browser (and the interactive components of the website). <code>Makefile</code> coordinates the builds—I prefer <code>make</code> to npm scripts because Google Closure Compiler takes a few seconds to run, and <code>make</code> easily ensures it’s only run when it needs to.</p>
<h2 id="acknowledgements">Acknowledgements</h2>
<p>I use <a href="https://github.com/substack/gamma.js">gamma.js</a>, one of substack’s very lightweight and very useful modules.</p>
<p>I’m super-grateful for, and happily acknowledge, the hard work of Athan Reines and collaborators on <a href="https://github.com/stdlib-js/stdlib">Stdlib.js</a>, which promises to be the math library JavaScript so badly needs. It is used here only for visualization purposes but I can recommend it.</p>
<p>The interactive website uses <a href="https://choo.io">Choo</a>, which is, as advertised, quite cute.</p>
<p>It’s generated from Markdown via <a href="http://pandoc.org">Pandoc</a>, and styled with John Otander’s <a href="http://markdowncss.github.io/modest/">Modest CSS</a>.</p>
<p>The plots are rendered using <a href="https://github.com/plotly/plotly.js/">Plotly.js</a>.</p>
<script type="text/javascript" src="client/plotly.min.js"></script>
<script type="text/javascript" src="client/interactive.min.js" async></script>
<script type="text/javascript" src="dist/ebisu.min.js" async></script>