diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 3046befe..236359e5 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -33,7 +33,7 @@ jobs:
     strategy:
       matrix:
         node-version: [18]
-        python-version: ["3.10"]
+        python-version: ["3.10", "3.11"]
     defaults:
       run:
         shell: bash -l {0}
diff --git a/docs/documentation/_sources/components.md.txt b/docs/documentation/_sources/components.md.txt
index 7cf577f6..f2366d7c 100644
--- a/docs/documentation/_sources/components.md.txt
+++ b/docs/documentation/_sources/components.md.txt
@@ -460,11 +460,27 @@ The UI supports multiple options for analysis, including:
 
 For a walkthrough of how to use sequence salience to debug LLMs, check out the
 Responsible Generative AI Toolkit at
-https://ai.google.dev/responsible/model_behavior.
+https://ai.google.dev/responsible/model_behavior and for more on design of the
+system see our paper at https://arxiv.org/abs/2404.07498.
+
+If you find this useful in your work, please cite Sequence Salience as:
+
+```
+@article{tenney2024interactive,
+  title={Interactive Prompt Debugging with Sequence Salience},
+  author={Tenney, Ian and Mullins, Ryan and Du, Bin and Pandya, Shree and Kahng, Minsuk and Dixon, Lucas},
+  journal={arXiv preprint arXiv:2404.07498},
+  year={2024}
+}
+```
 
 **Code:**
 
-*   LIT-for-Gemma Colab: [`lit_gemma.ipynb`](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb)
+Currently, this works out-of-the-box with Gemma, Llama 2, Mistral, and GPT-2,
+using either KerasNLP or Transformers.
+
+*   LIT-for-Gemma Colab:
+    [`lit_gemma.ipynb`](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb)
 *   Demo binary:
     [`lm_salience_demo.py`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/lm_salience_demo.py)
 *   KerasNLP model wrappers:
@@ -472,8 +488,6 @@ https://ai.google.dev/responsible/model_behavior.
 *   Transformers model wrappers:
     [`pretrained_lms.py`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/models/pretrained_lms.py)
 
-Currently, this works out-of-the-box
-with Gemma models (using Keras) as well as with GPT-2.
 
 
 ## Salience Clustering
diff --git a/docs/documentation/components.html b/docs/documentation/components.html
index 48454b7e..bfbf62a6 100644
--- a/docs/documentation/components.html
+++ b/docs/documentation/components.html
@@ -637,10 +637,23 @@ <h2>Sequence Salience<a class="headerlink" href="#sequence-salience" title="Link
 </ul>
 <p>For a walkthrough of how to use sequence salience to debug LLMs, check out the
 Responsible Generative AI Toolkit at
-<a class="reference external" href="https://ai.google.dev/responsible/model_behavior">https://ai.google.dev/responsible/model_behavior</a>.</p>
+<a class="reference external" href="https://ai.google.dev/responsible/model_behavior">https://ai.google.dev/responsible/model_behavior</a> and for more on design of the
+system see our paper at <a class="reference external" href="https://arxiv.org/abs/2404.07498">https://arxiv.org/abs/2404.07498</a>.</p>
+<p>If you find this useful in your work, please cite Sequence Salience as:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@article</span><span class="p">{</span><span class="n">tenney2024interactive</span><span class="p">,</span>
+  <span class="n">title</span><span class="o">=</span><span class="p">{</span><span class="n">Interactive</span> <span class="n">Prompt</span> <span class="n">Debugging</span> <span class="k">with</span> <span class="n">Sequence</span> <span class="n">Salience</span><span class="p">},</span>
+  <span class="n">author</span><span class="o">=</span><span class="p">{</span><span class="n">Tenney</span><span class="p">,</span> <span class="n">Ian</span> <span class="ow">and</span> <span class="n">Mullins</span><span class="p">,</span> <span class="n">Ryan</span> <span class="ow">and</span> <span class="n">Du</span><span class="p">,</span> <span class="n">Bin</span> <span class="ow">and</span> <span class="n">Pandya</span><span class="p">,</span> <span class="n">Shree</span> <span class="ow">and</span> <span class="n">Kahng</span><span class="p">,</span> <span class="n">Minsuk</span> <span class="ow">and</span> <span class="n">Dixon</span><span class="p">,</span> <span class="n">Lucas</span><span class="p">},</span>
+  <span class="n">journal</span><span class="o">=</span><span class="p">{</span><span class="n">arXiv</span> <span class="n">preprint</span> <span class="n">arXiv</span><span class="p">:</span><span class="mf">2404.07498</span><span class="p">},</span>
+  <span class="n">year</span><span class="o">=</span><span class="p">{</span><span class="mi">2024</span><span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+</div>
 <p><strong>Code:</strong></p>
+<p>Currently, this works out-of-the-box with Gemma, Llama 2, Mistral, and GPT-2,
+using either KerasNLP or Transformers.</p>
 <ul class="simple">
-<li><p>LIT-for-Gemma Colab: <a class="reference external" href="https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb"><code class="docutils literal notranslate"><span class="pre">lit_gemma.ipynb</span></code></a></p></li>
+<li><p>LIT-for-Gemma Colab:
+<a class="reference external" href="https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb"><code class="docutils literal notranslate"><span class="pre">lit_gemma.ipynb</span></code></a></p></li>
 <li><p>Demo binary:
 <a class="reference external" href="https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/lm_salience_demo.py"><code class="docutils literal notranslate"><span class="pre">lm_salience_demo.py</span></code></a></p></li>
 <li><p>KerasNLP model wrappers:
@@ -648,8 +661,6 @@ <h2>Sequence Salience<a class="headerlink" href="#sequence-salience" title="Link
 <li><p>Transformers model wrappers:
 <a class="reference external" href="https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/models/pretrained_lms.py"><code class="docutils literal notranslate"><span class="pre">pretrained_lms.py</span></code></a></p></li>
 </ul>
-<p>Currently, this works out-of-the-box
-with Gemma models (using Keras) as well as with GPT-2.</p>
 </section>
 <section id="salience-clustering">
 <h2>Salience Clustering<a class="headerlink" href="#salience-clustering" title="Link to this heading">#</a></h2>
diff --git a/docs/documentation/searchindex.js b/docs/documentation/searchindex.js
index a50bc241..75ece15f 100644
--- a/docs/documentation/searchindex.js
+++ b/docs/documentation/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["api", "components", "demos", "docker", "faq", "frontend_development", "getting_started", "glossary", "includes/highlight_demos", "index", "ui_guide"], "filenames": ["api.md", "components.md", "demos.md", "docker.md", "faq.md", "frontend_development.md", "getting_started.md", "glossary.md", "includes/highlight_demos.md", "index.md", "ui_guide.md"], "titles": ["LIT Python API", "Components and Features", "Demos", "Running LIT in a Docker container", "Frequently Asked Questions", "Frontend Developer Guide", "Getting Started with LIT", "Glossary", "&lt;no title&gt;", "Learning Interpretability Tool (LIT)", "UI Guide"], "terms": {"i": [0, 1, 3, 5, 6, 7, 10], "modular": 0, "compris": [0, 10], "collect": [0, 4, 5, 10], "backend": [0, 4, 5, 7, 10], "written": [0, 4], "frontend": [0, 4, 7, 9], "modul": [0, 1, 2, 3, 4, 7, 9], "typescript": [0, 5], "most": [0, 1, 4, 5, 6, 7, 10], "user": [0, 1, 3, 4, 5, 6, 9], "develop": [0, 1, 4, 9, 10], "against": [0, 1, 2, 4, 10], "which": [0, 1, 2, 3, 4, 5, 6, 7, 10], "document": [0, 1, 4, 5, 6], "below": [0, 1, 2, 3, 5, 10], "allow": [0, 1, 4, 5, 10], "extend": [0, 5], "counterfactu": [0, 2, 6, 7, 9, 10], "The": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10], "ar": [0, 1, 3, 4, 5, 6, 7, 10], "provid": [0, 1, 3, 4, 5, 6, 7, 10], "librari": [0, 1, 5], "can": [0, 1, 3, 4, 5, 6, 10], "through": [0, 1, 4, 10], "own": [0, 1, 9], "demo": [0, 1, 3, 4, 5, 7, 9], "binari": [0, 2, 6, 7, 10], "via": [0, 1, 4], "colab": [0, 1, 4], "also": [0, 1, 3, 4, 5, 6, 10], "regular": [0, 4, 7], "class": [0, 1, 4, 5, 6, 7, 10], "without": [0, 1, 4, 5, 10], "start": [0, 1, 3, 4, 5, 9], "see": [0, 1, 2, 4, 5, 6, 7, 10], "detail": [0, 1, 3, 4, 5, 6, 9], "serv": [0, 1, 3, 4, 5, 6, 7], "each": [0, 1, 2, 3, 5, 6, 10], "implement": [0, 1, 3, 4, 5, 6, 7], "minim": [0, 1, 3, 7, 10], "reli": [0, 5], "spec": [0, 1, 5], "detect": 0, "field": [0, 1, 4, 10], "verifi": [0, 1, 3, 10], "stateless": 0, "cach": [0, 4, 5, 6], "layer": [0, 1, 2, 4, 10], "predict": [0, 4, 5, 7, 9, 10], "thi": [0, 1, 2, 3, 4, 5, 6, 7, 10], "simplifi": [0, 5], "interact": [0, 1, 2, 4, 5, 7, 9, 10], "larg": [0, 1], "like": [0, 1, 4, 5, 7, 10], "bert": [0, 1, 4, 10], "t5": [0, 1, 4, 10], "state": [0, 4, 7, 10], "singl": [0, 1, 2, 4, 5, 6, 7, 10], "page": [0, 1, 5, 6, 9, 10], "app": [0, 3, 4, 5, 10], "built": [0, 2, 3, 5, 6, 7, 10], "1": [0, 1, 4, 5, 6, 10], "mobx": [0, 5], "manag": [0, 1, 4, 5], "It": [0, 1, 5, 10], "consist": [0, 1, 5, 7, 10], "core": [0, 5], "framework": [0, 4, 5, 6, 7, 9], "set": [0, 1, 2, 3, 4, 5, 6, 7], "share": [0, 4, 5], "servic": [0, 5, 7], "persist": [0, 1, 10], "independ": 0, "render": [0, 1, 5, 6, 7], "visual": [0, 1, 2, 4, 5, 7, 9, 10], "support": [0, 2, 4, 5, 6, 9, 10], "For": [0, 1, 2, 4, 5, 6, 10], "guid": [0, 1, 4, 6, 7, 9], "To": [0, 1, 6, 10], "run": [0, 1, 4, 5, 7, 9, 10], "your": [0, 1, 4, 5, 9, 10], "you": [0, 1, 3, 4, 5, 6, 9, 10], "creat": [0, 1, 3, 4, 5, 7, 10], "py": [0, 1, 2, 3, 4, 5, 6, 7], "script": [0, 3, 4, 6], "pass": [0, 1, 3, 4, 5, 6, 10], "def": [0, 1, 3], "main": [0, 1, 3, 5, 9], "_": 0, "mulitinlidata": 0, "mnli_match": 0, "multinlidata": 0, "path": [0, 1, 3, 4, 5, 6], "dev_match": 0, "tsv": [0, 6], "mnli_mismatch": 0, "dev_mismatch": 0, "nlimodel": 0, "model_foo": 0, "foo": [0, 5, 6], "file": [0, 3, 4, 5, 6], "model_bar": 0, "bar": [0, 1, 6, 7], "lit_demo": [0, 6], "lit_nlp": [0, 3, 5, 6], "dev_serv": [0, 3, 5, 6], "port": [0, 3, 5, 6], "4321": [0, 6], "__name__": 0, "__main__": 0, "conceptu": [0, 5], "just": [0, 4, 5, 6, 10], "list": [0, 1, 3, 4, 6, 10], "function": [0, 1, 2, 3, 10], "take": [0, 1, 4, 5, 6, 10], "return": [0, 1, 3, 5, 10], "metadata": [0, 1, 5], "describ": [0, 1, 3, 5, 6, 10], "themselv": 0, "other": [0, 1, 2, 4, 5, 6, 7, 10], "pre": [0, 1, 2, 4, 5, 6, 10], "out": [0, 1, 2, 5, 6, 9, 10], "http": [0, 1, 2, 3, 5, 6, 7, 9], "github": [0, 1, 2, 3, 4, 6, 9], "com": 0, "pair": [0, 1, 2, 3, 6, 9], "code": [0, 1, 2, 3, 4, 5, 6, 9], "tree": 0, "ensur": [0, 1, 3, 5], "match": [0, 4], "valu": [0, 1, 3, 4, 5, 10], "veri": [0, 1, 2, 5], "help": [0, 1, 3, 4, 10], "dure": [0, 5], "new": [0, 1, 4, 5, 10], "wrapper": [0, 1, 2, 4, 7], "correct": [0, 10], "behavior": [0, 1, 5], "At": [0, 10], "startup": 0, "flag": [0, 2, 3, 4, 5, 6, 10], "enabl": [0, 1, 6, 10], "There": [0, 1, 4, 7, 10], "three": [0, 2, 4, 5, 10], "mode": [0, 1, 2, 5, 10], "first": [0, 4, 5, 10], "sampl": [0, 1, 4], "5": [0, 1, 2, 5, 10], "all": [0, 1, 3, 4, 5, 7, 10], "from": [0, 1, 2, 3, 5, 6, 7, 10], "addition": [0, 1, 10], "call": [0, 1, 3, 5, 10], "directli": [0, 1, 4, 5, 6, 10], "associ": [0, 2, 5, 9, 10], "inform": [0, 4, 5, 10], "follow": [0, 3, 5, 10], "": [0, 1, 2, 3, 4, 5, 10], "should": [0, 1, 3, 4, 5, 6], "flat": 0, "dict": [0, 1], "self": [0, 1, 5], "_exampl": 0, "oper": [0, 4], "load": [0, 2, 5, 6, 10], "includ": [0, 1, 2, 3, 4, 5, 7, 10], "therefor": [0, 3], "care": 0, "size": [0, 1, 2], "fit": [0, 4], "memori": [0, 1, 4], "displai": [0, 1, 5, 10], "browser": [0, 4, 5], "note": [0, 1, 3, 5, 6, 10], "faq": [0, 1, 9], "limit": [0, 4], "subclass": [0, 5], "usual": [0, 1, 7], "few": [0, 1, 4, 5, 6, 7], "line": [0, 1, 3, 6, 10], "complet": [0, 1, 3, 10], "multinli": [0, 2], "loader": [0, 1], "nli_label": 0, "entail": 0, "neutral": 0, "contradict": 0, "__init__": [0, 1], "str": [0, 1, 3], "read": [0, 4, 5], "eval": [0, 2], "distribut": [0, 1], "glue": [0, 1, 3], "benchmark": 0, "df": 0, "panda": 0, "read_csv": 0, "sep": 0, "t": [0, 1, 3, 4, 5, 7, 10], "store": [0, 4, 5, 10], "conform": 0, "premis": [0, 2], "row": [0, 1, 10], "sentence1": 0, "hypothesi": [0, 1, 2], "sentence2": 0, "label": [0, 2, 10], "gold_label": 0, "genr": 0, "iterrow": 0, "lit_typ": [0, 1], "textseg": [0, 1], "categorylabel": [0, 1], "vocab": [0, 1], "we": [0, 1, 3, 4, 5, 6, 7, 10], "addit": [0, 1, 3, 4, 5, 6, 10], "don": [0, 1, 4, 5], "have": [0, 1, 3, 5, 10], "four": 0, "string": [0, 1, 4, 5], "semant": [0, 1], "tell": 0, "bit": [0, 5], "about": [0, 1, 5, 10], "how": [0, 1, 3, 5, 6, 10], "them": [0, 1, 4, 5, 10], "treat": [0, 1, 4], "natur": [0, 2, 9], "languag": [0, 1, 7, 9, 10], "text": [0, 1, 2, 4], "categor": [0, 1, 2, 4, 10], "featur": [0, 2, 4, 5, 6, 7, 9, 10], "fix": [0, 1], "known": [0, 7, 10], "possibl": [0, 1, 5], "unknown": 0, "open": [0, 1, 9], "tensorflow": [0, 2], "simpli": [0, 1, 3], "wrap": [0, 1, 4], "freeli": 0, "add": [0, 1, 3, 4, 5, 10], "abov": [0, 1, 3, 4, 5, 6, 10], "mai": [0, 1, 4, 5, 6, 7, 10], "awar": 0, "recogn": [0, 1], "slice": [0, 1, 2, 4, 7, 9], "bin": 0, "etc": [0, 3, 4], "thei": [0, 1, 4, 5, 10], "similar": [0, 3, 10], "tf": [0, 1], "scope": 0, "aim": [0, 5], "quick": [0, 1, 2, 10], "iter": 0, "step": [0, 3, 5], "end": [0, 3, 10], "same": [0, 1, 5], "datapoint": [0, 1, 4, 7, 9], "n": 0, "seed": 0, "42": 0, "random": [0, 1, 10], "remap": 0, "field_map": 0, "renam": 0, "both": [0, 1, 5], "latter": 0, "shortcut": 0, "one": [0, 1, 3, 4, 5, 6, 7, 10], "anoth": [0, 1, 5, 7, 10], "expect": [0, 4], "2": [0, 1, 5], "produc": [0, 1, 3, 10], "method": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10], "input_spec": [0, 1], "necessari": [0, 3, 5], "output_spec": [0, 1], "ani": [0, 1, 3, 4, 5, 10], "sequenc": [0, 3, 9], "satisfi": 0, "yield": 0, "parallel": 0, "might": [0, 1, 3, 10], "look": [0, 1, 3, 4, 10], "someth": [0, 1, 5, 7], "infer": [0, 1, 2, 4, 5, 6, 10], "model_path": 0, "kw": 0, "so": [0, 1, 3, 4, 10], "re": [0, 4, 5, 7], "readi": 0, "_model": 0, "_load_my_model": 0, "pred": 0, "stream": 0, "convert_dict_input": 0, "d": [0, 3, 10], "preprocess": 0, "predict_exampl": 0, "parent": [0, 1, 5], "keyword": 0, "where": [0, 1, 2, 3, 5, 6, 10], "gold": [0, 1, 2], "when": [0, 1, 4, 5, 7, 10], "comput": [0, 1, 4, 5, 9], "proba": [0, 1], "multiclasspr": [0, 1, 10], "unlik": [0, 1], "incomplet": 0, "ll": [0, 1, 4, 5], "need": [0, 1, 3, 4, 5, 6, 7], "accordingli": [0, 1], "post": [0, 1, 2], "process": [0, 1, 2, 5, 9], "token": [0, 2, 4, 9, 10], "mani": [0, 1, 3, 4, 5, 6, 7, 10], "deep": 0, "learn": [0, 1, 7, 10], "batch": 0, "thu": [0, 5], "batchedmodel": 0, "simpl": [0, 1, 5, 6, 10], "must": [0, 1, 5, 10], "predict_minibatch": 0, "convert": [0, 1], "jsondict": 0, "object": [0, 4], "appropri": [0, 1, 4], "represent": [0, 1, 2], "typic": [0, 1], "map": [0, 1, 2, 3, 5], "align": [0, 1], "tensor": 0, "befor": [0, 3, 4, 6, 10], "want": [0, 1, 3, 4, 5, 6, 9, 10], "overrid": [0, 3, 5], "max_minibatch_s": 0, "determin": [0, 1, 5, 10], "If": [0, 1, 4, 5, 6, 9, 10], "remot": [0, 7], "consid": 0, "batchedremotemodel": 0, "base": [0, 2, 3, 5, 9, 10], "request": [0, 4, 5], "thread": 0, "pool": 0, "defin": [0, 1, 3, 5, 10], "black": [0, 1], "box": [0, 1, 10], "intern": [0, 1, 5], "richer": 0, "view": [0, 1, 4, 10], "correspond": [0, 1], "hidden": [0, 5], "activ": [0, 1, 10], "gradient": [0, 2, 10], "word": [0, 1, 2, 10], "embed": [0, 2, 4, 9], "attent": [0, 2, 9], "sever": [0, 1, 3, 4], "two": [0, 1, 2, 3, 5, 10], "differ": [0, 1, 2, 4, 5, 10], "vector": [0, 1], "easili": [0, 1, 3, 4, 6, 10], "output_emb": 0, "cl": 0, "top": [0, 1, 5], "mean_word_emb": 0, "mean": [0, 1, 10], "than": [0, 1, 4, 5, 10], "segment": [0, 2], "premise_token": 0, "hypothesis_token": 0, "here": [0, 3, 5], "refer": [0, 1, 2, 3, 5, 7, 10], "premise_grad": 0, "tokengradi": [0, 1], "hypothesis_grad": 0, "similarli": [0, 1, 10], "full": [0, 1, 4, 5, 7, 10], "baz": 0, "spam": 0, "egg": 0, "attention_layer0": 0, "attentionhead": [0, 1], "attention_layer1": 0, "attention_layer2": 0, "sinc": [0, 1, 4, 5], "dictionari": [0, 5], "dataclass": 0, "popul": 0, "loop": 0, "would": [0, 1], "entri": [0, 5, 10], "becaus": [0, 1, 3, 5], "often": [0, 1, 7], "tightli": 0, "coupl": [0, 4], "intermedi": 0, "level": [0, 1, 2, 9, 10], "expos": 0, "automat": [0, 1, 5, 10], "projector": [0, 2, 4, 9], "salienc": [0, 2, 6, 7, 9], "multi": [0, 2, 4, 10], "head": [0, 1, 10], "wai": [0, 1, 2, 4, 5, 10], "By": [0, 1, 10], "default": [0, 1, 4, 5, 6, 9, 10], "requir": [0, 1, 3, 5, 10], "howev": [0, 4, 5], "fals": [0, 1, 10], "wish": 0, "accept": [0, 1], "And": [0, 3, 5], "logic": [0, 1, 5], "bypass": 0, "input_token": [0, 1], "ex": 0, "get": [0, 1, 3, 5, 7, 9, 10], "rest": [0, 1], "though": [0, 1, 5, 6, 10], "omit": 0, "entir": [0, 1, 2, 10], "aid": 0, "umap": [0, 1, 2, 7, 10], "plug": [0, 1], "ins": 0, "some": [0, 1, 2, 5, 6, 10], "3": [0, 1, 5, 10], "local": [0, 1, 4], "gradientnorm": 0, "score": [0, 9, 10], "integr": [0, 2, 9], "tokenembed": [0, 1], "well": [0, 1, 2, 4, 5, 6, 10], "target": [0, 2, 10], "pin": [0, 1, 4, 10], "involv": [0, 1, 3, 4, 6], "lit_model": 0, "lit_dataset": 0, "model_output": 0, "none": [0, 1, 5, 10], "config": 0, "runtim": [0, 1, 4], "threshold": [0, 2, 10], "classif": [0, 4, 9, 10], "unconstrain": 0, "up": [0, 1, 4, 5, 10], "correctli": [0, 10], "particular": [0, 1, 5, 10], "while": [0, 1, 3, 4, 5, 10], "aggreg": [0, 9], "summari": 0, "respons": [0, 1, 2, 5], "what": [0, 1, 3, 5, 10], "A": [0, 1, 3, 5, 7, 10], "over": [0, 1, 7, 10], "relev": 0, "find_field": 0, "find": [0, 1, 2, 3, 10], "grad_field": 0, "util": [0, 10], "find_spec_kei": 0, "f": 0, "tokens_field": 0, "pytyp": 0, "disabl": [0, 5], "attribut": [0, 5, 9], "error": [0, 2, 6, 7, 10], "assert": 0, "isinst": 0, "given": [0, 1, 5, 10], "log": [0, 1, 3, 10], "info": [0, 3], "found": [0, 3, 5, 7, 9, 10], "len": 0, "0": [0, 1, 2, 5, 6, 10], "pylint": 0, "g": [0, 1, 3, 4, 7, 10], "explicit": [0, 4, 5], "length": [0, 1, 4, 10], "test": [0, 1, 2, 5], "do": [0, 1, 4, 5], "work": [0, 1, 2, 4, 5], "dtype": [0, 1], "tokensali": 0, "ha": [0, 1, 4, 5, 10], "overhead": 0, "benefit": 0, "flexibl": [0, 1], "specifi": [0, 1, 2, 4, 5], "complex": [0, 1, 5], "depend": [0, 2, 3, 4, 5, 10], "between": [0, 1, 2, 5, 10], "multipl": [0, 2, 3, 5, 10], "simplemetr": 0, "unpack": 0, "is_compat": 0, "name": [0, 1, 2, 3, 4, 5, 7, 10], "regressionmetr": 0, "standard": [0, 1, 4, 10], "regress": [0, 9, 10], "field_spec": 0, "littyp": 0, "bool": 0, "true": [0, 2, 3, 5, 10], "regressionscor": [0, 1], "float": [0, 1], "label_spec": 0, "scalar": [0, 4], "pred_spec": 0, "del": 0, "mse": 0, "sklearn_metr": 0, "mean_squared_error": 0, "pearsonr": 0, "scipy_stat": 0, "spearmanr": 0, "kei": [0, 1], "compar": [0, 1, 2, 4, 9], "These": [0, 1, 3, 4, 5, 10], "onli": [0, 1, 3, 4, 5, 10], "techniqu": [0, 1, 2], "back": [0, 1, 5, 7, 10], "translat": [0, 1, 2, 4, 5, 10], "feedback": [0, 5, 10], "adversari": [0, 1, 2], "attack": [0, 1], "generate_al": 0, "arg": [0, 3, 5], "current": [0, 1, 4, 5, 10], "belong": [0, 1, 10], "conveni": [0, 4, 5, 10], "As": [0, 1, 10], "argument": [0, 3], "substitut": [0, 1], "replac": [0, 1, 10], "foreign": 0, "sourc": [0, 1, 3, 6, 9, 10], "order": [0, 1, 4, 5, 10], "paraphras": [0, 1, 10], "googl": [0, 1, 2], "cloud": [0, 2], "perform": [0, 1, 5, 10], "those": [0, 1, 3, 4, 10], "project": [0, 1, 5, 7, 10], "doc": [0, 3, 5, 10], "setup": [0, 10], "Then": [0, 1, 3, 6], "download": [0, 10], "applic": [0, 1, 7, 9, 10], "credenti": 0, "google_application_credenti": 0, "environ": [0, 3, 4], "variabl": [0, 3, 5], "point": [0, 1, 4, 5, 10], "With": [0, 5, 10], "make": [0, 1, 4, 5, 10], "backtranl": 0, "constructor": [0, 4, 5, 6], "time": [0, 4, 10], "number": [0, 1, 2, 4, 5, 10], "lime": [0, 2, 10], "pivot": 0, "dsl": 0, "auto": 0, "form": 0, "config_spec": 0, "sparsemultilabel": [0, 1], "bg": 0, "de": 0, "el": 0, "en": [0, 2], "e": [0, 1, 3, 4, 7, 10], "fr": 0, "hi": [0, 2, 10], "ru": 0, "sw": 0, "th": 0, "tr": 0, "ur": 0, "vi": 0, "zh": 0, "give": [0, 1, 6, 10], "its": [0, 1, 5, 10], "interpreter_control": 0, "slider": [0, 1, 10], "numer": [0, 1, 2, 4, 10], "min_val": 0, "max_val": 0, "boolean": [0, 5], "booleanlittyp": 0, "checkbox": [0, 1, 10], "dropdown": [0, 10], "seri": 0, "comma": [0, 10], "separ": [0, 1, 3, 4, 5, 10], "pars": [0, 1, 3, 5], "suppli": 0, "singlefieldmatch": 0, "act": [0, 1], "multifieldmatch": 0, "except": [0, 7], "select": [0, 2, 4, 5, 9], "control": [0, 1, 4, 5, 7, 10], "choos": [0, 1], "perturb": [0, 1, 2], "hotflip": [0, 1, 2, 10], "around": [0, 5], "web": [0, 4, 5, 7, 10], "record": 0, "serial": 0, "json": 0, "commun": [0, 1], "client": [0, 7, 9, 10], "introduc": [0, 5, 10], "understand": [0, 1, 2, 10], "shape": [0, 1], "being": [0, 1, 5, 10], "specif": [0, 1, 2, 5, 7, 10], "properti": [0, 5], "whose": 0, "annot": [0, 1], "hierarch": 0, "inherit": [0, 7], "canon": 0, "kind": 0, "v": [0, 10], "meta": 0, "context": 0, "implic": 0, "per": [0, 1, 2, 5, 10], "everi": [0, 1, 5, 10], "init_spec": 0, "instanc": [0, 2, 5, 7, 9], "empti": [0, 3], "noth": 0, "show": [0, 1, 2, 5, 10], "section": [0, 1, 10], "global": [0, 1, 5, 7], "subset": [0, 1, 2, 10], "paramet": [0, 3, 10], "alwai": [0, 1, 3, 7], "meta_spec": 0, "essenti": 0, "ideal": [0, 1], "wherea": 0, "slight": 0, "variat": 0, "tradit": 0, "metricresult": 0, "higher": [0, 1], "lower": [0, 5, 10], "closer": 0, "zero": 0, "better": [0, 1, 10], "encapsul": 0, "impli": 0, "index": [0, 5, 10], "element": [0, 1, 4, 7], "contain": [0, 1, 4, 5, 7, 10], "po": 0, "sequencetag": [0, 1], "item": 0, "indic": [0, 1, 10], "transit": [0, 1], "zip": 0, "strict": 0, "10": [0, 1, 5], "pseudo": 0, "ground": [0, 1, 10], "truth": [0, 1, 10], "repres": [0, 1], "appear": [0, 1, 10], "assum": [0, 3, 10], "pattern": [0, 1, 5], "manipul": [0, 5, 10], "could": [0, 1, 7, 10], "former": [0, 7, 10], "swap": 0, "mask": [0, 1, 2], "feed": [0, 1, 7], "fill": [0, 10], "plai": [0, 6, 9], "critic": 0, "role": 0, "reliabl": 0, "interoper": 0, "robust": [0, 1, 10], "univers": 0, "is_compatible_with_dataset": 0, "wordreplac": 0, "doe": [0, 1, 3, 4, 10], "curv": 0, "gradientdotinput": [0, 1], "shoulddisplaymodul": 0, "suffici": [0, 1], "caus": [0, 5, 10], "jitter": [0, 2, 10], "disappear": 0, "reorder": [0, 5], "resiz": [0, 4, 7], "switch": [0, 1, 2, 10], "heterogen": 0, "findspeckei": 0, "identifi": 0, "interest": [0, 1, 10], "respect": [0, 1, 10], "import": [0, 1, 2, 5, 10], "build": [0, 1, 5, 9], "litmetadata": 0, "litapp": [0, 5], "raw": [0, 1], "jupyt": [0, 6], "repl": 0, "encourag": [0, 1], "explicitli": [0, 1, 5, 10], "avoid": [0, 5], "chase": 0, "red": [0, 10], "her": 0, "mnli": 0, "buffet": 0, "la": 0, "cart": 0, "travel": 0, "task": [0, 1, 2, 5, 6, 10], "notic": 0, "967": 0, "024": 0, "009": 0, "classificationresult": 0, "human": [0, 1], "readabl": 0, "classification_result": 0, "predicted_class": 0, "integ": 0, "predefin": [0, 1], "rang": [0, 1, 2, 10], "overridden": 0, "seen": 0, "summar": [0, 2, 10], "tabl": [0, 1, 4, 5], "bracket": 0, "syntax": 0, "num_token": [0, 1], "numpi": [0, 1], "arrai": [0, 1], "insid": [0, 3, 10], "descript": [0, 3, 5], "untoken": 0, "generatedtext": [0, 1], "seq2seq": 0, "url": [0, 1, 4, 5, 9], "generatedurl": 0, "may": 0, "real": 0, "inappropri": 0, "searchqueri": 0, "search": [0, 1, 10], "queri": [0, 1, 2, 4, 5], "opaqu": 0, "ignor": 0, "referencetext": [0, 1], "mt": 0, "tupl": [0, 1], "generatedtextcandid": [0, 1], "candid": [0, 1, 2, 10], "beam": [0, 1, 2], "tokentopkpr": 0, "32768": 0, "32767": 0, "int": 0, "imagebyt": [0, 1], "imag": [0, 4, 9], "base64": [0, 1], "encod": [0, 1, 4, 10], "jpegbyt": 0, "pngbyte": 0, "referencescor": [0, 1], "vocabulari": 0, "multiclass": [0, 2], "probabl": [0, 1, 2, 4, 10], "num_label": 0, "non": [0, 1, 2, 4, 10], "exclus": [0, 1], "sparsemultilabelpr": [0, 1], "spars": 0, "tag": [0, 1, 3, 5, 10], "spanlabel": [0, 1], "span": [0, 10], "j": [0, 1, 5], "edgelabel": [0, 1], "edg": [0, 2], "structur": [0, 5, 9, 10], "corefer": 0, "srl": [0, 1], "arxiv": 0, "org": [0, 9], "ab": 0, "1905": 0, "06316": 0, "multisegmentannot": [0, 1], "byte": [0, 1], "annotationclust": 0, "emb_dim": [0, 1], "imagegradi": [0, 1], "pixel": [0, 9], "image_height": [0, 1], "image_width": [0, 1], "color_channel": [0, 1], "group": [0, 1, 5, 7, 10], "num_head": [0, 1], "plain": [0, 1, 2], "further": 0, "stringlittyp": 0, "urllittyp": 0, "collis": 0, "protect": 0, "page_titl": 0, "titl": [0, 5, 9], "canonical_url": [0, 10], "shortlink": 0, "copi": [0, 4, 5, 10], "link": [0, 6, 10], "default_layout": 0, "demo_mod": [0, 3, 4], "kiosk": 0, "save": [0, 1, 4, 5, 10], "untrust": 0, "inline_doc": 0, "markdown": 0, "panel": [0, 5, 7, 10], "onboard_start_doc": 0, "onboard": 0, "splash": 0, "screen": [0, 5, 10], "onboard_end_doc": 0, "last": [0, 2, 3], "server_flag": [0, 5], "litcanonicallayout": [0, 5], "lm_layout": 0, "upper": [0, 5, 10], "embeddingsmodul": 0, "datatablemodul": [0, 5], "datapointeditormodul": [0, 5], "languagemodelpredictionmodul": 0, "confusionmatrixmodul": 0, "generatormodul": 0, "lm": 0, "get_flag": 0, "lm_demo": [0, 2, 3, 5], "achiev": [0, 2], "major": [0, 10], "content": [0, 4, 10], "area": [0, 10], "left": [0, 1, 2, 5, 10], "right": [0, 1, 2, 5, 10], "shown": [0, 1, 3, 10], "initi": [0, 1, 10], "set_default": [0, 3, 5], "my_layout_nam": 0, "fly": 0, "param": [0, 4, 5], "preced": 0, "comprehens": 0, "remov": [0, 1, 5, 10], "chang": [0, 1, 3, 5, 10], "experiment": [0, 1, 4, 5, 10], "desir": [0, 2, 5, 10], "altern": [0, 1], "connect": [0, 1, 4], "after": [0, 1, 5, 10], "instal": [0, 3, 9], "pip": [0, 6], "litwidget": [0, 6], "height": [0, 5, 10], "cell": [0, 1, 4, 6, 10], "widget": [0, 4, 7], "tab": [0, 1, 4, 5, 10], "open_in_new_tab": 0, "certain": 0, "stop": 0, "shut": 0, "down": [0, 1, 10], "free": 0, "resourc": 0, "plan": 0, "doesn": [0, 7, 10], "still": [0, 1], "standalon": [0, 4], "handi": [0, 1], "reload": [0, 1, 5], "click": [0, 1, 2, 5, 10], "sst2data": 0, "print": 0, "sentenc": [0, 1, 2, 10], "glue_model": [0, 4], "sst2model": 0, "cls_emb": 0, "Or": 0, "lime_explain": 0, "lit_components_exampl": [0, 6], "ipynb": [0, 1, 6], "happi": 0, "coincid": 0, "tool": [0, 1, 5, 7, 10], "relat": [0, 1, 10], "solv": 0, "case": [0, 1, 2, 3, 4, 5, 6, 10], "question": [0, 1], "qa": 0, "trivial": 0, "practic": [0, 1, 5], "endpoint": [0, 1, 5, 7], "lit": [1, 2, 5, 7, 10], "agnost": 1, "compat": [1, 5, 10], "python": [1, 3, 4, 5, 6, 7, 9, 10], "In": [1, 4, 5, 6, 10], "ve": 1, "tf2": [1, 2], "jax": 1, "pytorch": 1, "us": [1, 2, 3, 5, 7, 9, 10], "custom": [1, 2, 4, 7, 9, 10], "c": [1, 10], "clif": 1, "rpc": 1, "aren": [1, 10], "constraint": [1, 2], "beyond": 1, "impos": 1, "platform": 1, "coexist": 1, "underli": [1, 4], "hardwar": 1, "gpu": [1, 4, 6], "scale": [1, 9], "dataset": [1, 2, 5, 6, 7, 9, 10], "kera": [1, 2, 4], "easi": [1, 5, 10], "access": [1, 4, 5, 6, 9, 10], "interpret": [1, 2, 6, 7, 10], "great": [1, 10], "place": [1, 5, 10], "small": [1, 5, 6], "experi": 1, "academ": 1, "more": [1, 3, 4, 5, 6, 7, 10], "due": 1, "graph": [1, 10], "session": 1, "cannot": 1, "invoc": 1, "weight": [1, 5], "export": [1, 5, 10], "savedmodel": [1, 2], "an": [1, 3, 4, 5, 7, 9, 10], "eager": 1, "lead": [1, 5, 10], "much": [1, 5, 10], "simpler": 1, "servomat": 1, "usag": [1, 4, 9], "remain": 1, "server": [1, 3, 4, 5, 6, 7, 9, 10], "stub": [1, 5], "handl": [1, 2, 4, 5, 7, 10], "format": [1, 2, 4, 5, 10], "convers": 1, "purpos": 1, "interfac": [1, 5], "remote_model": [1, 4], "best": [1, 2, 5], "isn": [1, 10], "situat": 1, "staticpredict": 1, "lookup": [1, 10], "quickli": [1, 10], "brows": 1, "retain": 1, "rich": 1, "almost": 1, "tfrecord": [1, 6], "capacitor": 1, "sstabl": 1, "even": 1, "sql": 1, "api": [1, 3, 4, 5, 6, 7, 9, 10], "our": [1, 3, 4, 5, 6, 9, 10], "tfd": 1, "ui": [1, 5, 6, 7, 9], "workflow": [1, 9], "extens": [1, 5, 9], "system": [1, 3, 5, 7, 9, 10], "modal": [1, 4], "common": [1, 2, 5], "nlp": [1, 3, 6, 9], "domain": 1, "ad": [1, 4, 5, 9, 10], "explor": [1, 2, 5, 7, 10], "side": [1, 2, 10], "avail": [1, 4, 5, 6, 10], "matric": 1, "basic": [1, 9], "sentiment": 1, "simple_tf2_demo": 1, "evalu": [1, 2, 4, 10], "neg": [1, 10], "design": [1, 4, 9], "null_idx": 1, "commonli": [1, 7, 10], "precis": [1, 10], "recal": 1, "f1": 1, "auc": 1, "aucpr": 1, "comment": [1, 3], "toxic": [1, 10], "posit": [1, 2, 10], "margin": [1, 2, 10], "bia": 1, "space": [1, 2, 10], "bucket": [1, 10], "facet": [1, 5, 7, 10], "scatterplot": 1, "st": 1, "b": [1, 10], "textual": 1, "glue_demo": [1, 2, 3, 6], "image_demo": [1, 2, 3], "quit": 1, "matur": [1, 5], "highlight": [1, 2, 10], "diff": [1, 2], "decod": [1, 2, 10], "emit": 1, "varieti": [1, 2, 4, 6], "part": [1, 2, 7, 10], "speech": 1, "entiti": 1, "ner": 1, "mix": 1, "privileg": 1, "otherwis": [1, 5], "kernel": 1, "shap": 1, "valid": [1, 3], "check": [1, 2, 6, 9, 10], "mark": 1, "option": [1, 3, 5], "least": 1, "multilabel": 1, "penguin": [1, 4], "stat": 1, "penguin_demo": [1, 2], "That": 1, "awai": [1, 2], "move": [1, 5], "individu": [1, 2, 4, 5, 10], "color": [1, 5, 10], "break": 1, "exist": [1, 4, 9, 10], "open_imag": 1, "classifi": [1, 2, 10], "comparison": [1, 5, 7, 10], "onc": [1, 10], "io": [1, 2, 3, 6, 7, 9], "html": [1, 2, 5, 7], "navig": [1, 6, 10], "explan": [1, 10], "proport": 1, "l2": [1, 2], "propto": 1, "nabla_": 1, "x_i": 1, "hat": 1, "y": [1, 10], "_2": 1, "result": [1, 2, 5, 6, 10], "choic": 1, "argmax": 1, "product": [1, 4], "cdot": 1, "grad": [1, 2], "direct": [1, 3], "influenc": [1, 10], "suggest": [1, 4, 10], "stronger": 1, "wa": [1, 5, 10], "grad_for": 1, "normal": [1, 5, 10], "contribut": 1, "along": [1, 4, 7, 10], "sundararajan": 1, "et": [1, 2], "al": [1, 2], "2017": 1, "algorithm": 1, "instrument": 1, "modifi": [1, 5, 10], "plu": 1, "grad_target_field_kei": 1, "convent": [1, 5], "interpol": 1, "token_emb": 1, "input_text": 1, "token_grad": 1, "concret": 1, "contact": 1, "team": [1, 2], "assist": 1, "drop": [1, 10], "train": [1, 2, 10], "linear": 1, "reconstruct": 1, "origin": [1, 5, 10], "trade": 1, "off": [1, 5, 10], "slow": [1, 4], "noisi": 1, "longer": [1, 10], "ablat": 1, "compens": 1, "increas": 1, "explain": [1, 3, 10], "movi": [1, 10], "terribl": [1, 10], "ask": [1, 10], "receiv": 1, "okai": 1, "present": 1, "long": [1, 3, 4, 10], "deriv": [1, 5], "impact": 1, "prompt": [1, 2], "causal": 1, "granular": [1, 2], "sub": [1, 10], "paragraph": [1, 2], "refin": 1, "densiti": 1, "shot": 1, "eaxmpl": 1, "chain": 1, "thought": [1, 5], "walkthrough": [1, 2], "debug": [1, 2, 7], "llm": [1, 2], "ai": [1, 2], "toolkit": [1, 2, 5], "dev": [1, 2, 7], "model_behavior": [1, 2], "gemma": 1, "lit_gemma": 1, "lm_salience_demo": [1, 2], "kerasnlp": [1, 2], "instrumented_keras_lm": 1, "pretrained_lm": 1, "gpt": 1, "ebert": 1, "2022": 1, "k": 1, "bag": 1, "appli": 1, "button": [1, 5, 10], "6": [1, 5], "paper": [1, 9], "seek": 1, "column": [1, 10], "pleas": [1, 5, 9], "tutori": [1, 2], "analyz": [1, 10], "actual": [1, 5, 10], "been": [1, 3, 5, 10], "grad_target": 1, "channel": 1, "2d": [1, 10], "abl": [1, 4], "mobilenet": [1, 4], "blur": 1, "xrai": 1, "imagesali": 1, "layer_0": 1, "align_in": 1, "align_out": 1, "layer_1": 1, "layer_2": 1, "target_token": 1, "version": 1, "latent": [1, 2], "pca": [1, 2, 7, 10], "pan": [1, 10], "zoom": 1, "rotat": [1, 10], "shift": [1, 5, 10], "colormap": 1, "menu": [1, 2, 4, 10], "bleu": [1, 10], "whole": [1, 4], "try": 1, "power": [1, 5], "header": [1, 7, 10], "intersect": 1, "drag": [1, 10], "decis": 1, "boundari": 1, "respond": [1, 4, 5], "either": [1, 5, 10], "calcul": [1, 10], "optim": 1, "cost": 1, "ratio": 1, "rel": 1, "penalti": 1, "equal": 1, "costli": 1, "term": [1, 7, 10], "twice": 1, "overal": [1, 5, 10], "fair": [1, 2], "One": [1, 5], "demograph": [1, 2], "pariti": [1, 2], "attempt": 1, "percentag": 1, "accuraci": [1, 10], "opportun": 1, "among": 1, "effect": [1, 5], "minimum": 1, "maximum": [1, 10], "chart": 1, "averag": 1, "chosen": [1, 5], "done": [1, 5, 10], "across": [1, 5, 10], "inspir": 1, "prior": [1, 10], "toolbar": [1, 5, 9], "megaplot": [1, 4], "100k": [1, 4], "mous": 1, "scroll": 1, "reset": [1, 10], "futur": [1, 10], "releas": 1, "contrast": 1, "high": [1, 9, 10], "concept": 1, "gender": 1, "race": [1, 5], "akin": 1, "cav": 1, "intuit": 1, "measur": [1, 10], "sensit": [1, 10], "mention": [1, 2], "rather": [1, 5], "begin": 1, "low": 1, "15": [1, 4, 9], "selector": 1, "actor": 1, "actress": 1, "next": [1, 10], "final": [1, 5], "newli": [1, 4, 10], "split": 1, "second": [1, 5, 10], "9": [1, 5], "blue": [1, 10], "baselin": 1, "technic": 1, "null": [1, 5, 10], "pitfal": 1, "potenti": 1, "meaningless": 1, "randomli": [1, 10], "meaning": 1, "guard": 1, "whether": [1, 5], "possibli": 1, "remaind": 1, "reject": 1, "insignific": 1, "p": [1, 3], "greater": [1, 5], "05": 1, "happen": 1, "warn": 1, "100": [1, 10], "uniqu": 1, "accommod": 1, "cross": [1, 4], "approach": [1, 5], "too": 1, "min": 1, "upcom": 1, "pictur": 1, "insight": 1, "answer": [1, 10], "my": 1, "behav": 1, "under": [1, 3, 6, 10], "delet": [1, 10], "systemat": 1, "editor": 1, "enter": [1, 4], "duplic": [1, 5, 7, 10], "keep": [1, 4, 10], "track": [1, 10], "relationship": 1, "cycl": [1, 10], "scrambl": [1, 10], "regex": [1, 10], "hand": [1, 7, 10], "shelf": 1, "scrambler": [1, 10], "ebrahimi": 1, "tri": 1, "special": 1, "get_embedding_t": 1, "flip": 1, "maintain": [2, 7], "host": [2, 3, 4, 7, 9], "launcher": 2, "type": [2, 5, 6, 9, 10], "publicli": 2, "visibl": [2, 5], "exampl": [2, 3, 5, 7, 9, 10], "analysi": [2, 9], "sst": 2, "input": [2, 4, 5, 9, 10], "huggingfac": 2, "widest": 2, "output": [2, 4, 5, 6, 9], "metric": [2, 6, 7], "plot": [2, 10], "confid": 2, "ones": 2, "cluster": [2, 9, 10], "gener": [2, 4, 6, 7, 9], "tip": [2, 9], "studi": 2, "public": [2, 3], "websit": 2, "xnli_demo": 2, "14": 2, "jp": 2, "fine": 2, "tune": 2, "whitespac": 2, "delimit": 2, "wordpiec": 2, "stsb_dev": 2, "unrel": 2, "wide": 2, "scatter": [2, 10], "filter": [2, 10], "2b": 2, "7b": 2, "dot": 2, "depth": 2, "t5_demo": [2, 3], "hypothes": 2, "veer": 2, "roug": [2, 10], "machin": [2, 3, 4, 7], "cnndm": 2, "wmt": [2, 4], "corpora": 2, "coref_demo": [2, 3], "ontonot": [2, 10], "winogend": [2, 10], "schema": 2, "ruding": 2, "2018": 2, "profess": 2, "stratifi": 2, "quantifi": 2, "pronoun": [2, 10], "bureau": 2, "labor": 2, "statist": 2, "partial": 2, "setter": 2, "imagenet": 2, "deploi": 3, "onto": 3, "face": 3, "container": 3, "prefer": [3, 5], "engin": [3, 5], "dockerfil": 3, "directori": [3, 5, 6], "front": 3, "gunicorn": 3, "invok": [3, 5], "get_wsgi_app": 3, "wsgi": [3, 4, 7], "gunicorn_config": 3, "shell": 3, "command": [3, 6], "rememb": 3, "trail": 3, "know": [3, 10], "now": 3, "rm": 3, "5432": [3, 5], "launch": [3, 10], "demo_nam": 3, "demo_port": 3, "children": 3, "nest": 3, "coref": 3, "is_ev": 3, "is_eval_demo": 3, "Be": 3, "sure": [3, 4, 5], "2345": 3, "bring": 3, "togeth": [3, 10], "combin": 3, "background": [3, 10], "execut": 3, "meet": 3, "locat": 3, "litservertyp": 3, "server_typ": 3, "extern": [3, 4], "unus": 3, "sy": 3, "argv": 3, "known_onli": 3, "rebuild": 3, "your_server_script_path_her": 3, "consum": 3, "absl": 3, "stricli": 3, "recommend": [3, 4, 5, 6], "consol": 3, "suppos": 3, "come": 3, "soon": 3, "submit": 4, "bug": 4, "issu": [4, 5], "modern": 4, "ml": 4, "good": [4, 10], "tabular": [4, 9], "palmer": 4, "unicod": 4, "english": 4, "latin": [4, 5], "modif": 4, "xnli": 4, "lingual": 4, "nli": 4, "multilingu": 4, "comfort": 4, "10k": 4, "speed": 4, "caveat": [4, 5], "lot": 4, "larger": [4, 5], "warm_start": [4, 5, 6], "nativ": 4, "resolut": 4, "smaller": [4, 6], "anywai": 4, "256x256": 4, "thing": [4, 5, 7], "webgl": 4, "scattergl": 4, "dive": [4, 10], "older": [4, 10], "thousand": 4, "preload": 4, "wait": [4, 5, 10], "visit": 4, "bewar": 4, "node": [4, 5], "favorit": 4, "lightweight": 4, "alreadi": [4, 10], "ephemer": 4, "window": [4, 7], "manual": [4, 5, 10], "data_dir": [4, 5], "disk": [4, 5], "abil": 4, "edit": [4, 10], "write": [4, 6], "anyon": 4, "address": [4, 10], "restrict": 4, "configur": [4, 5, 6, 9, 10], "thin": [4, 7], "middlewar": 4, "docker": [4, 9], "data_": 4, "fieldnam": 4, "total": 4, "within": [4, 10], "data0": 4, "data1": 4, "data2": 4, "data0_": 4, "csv": [4, 10], "push": 4, "writabl": 4, "notebook": [4, 9], "ui_stat": 4, "primari": [4, 10], "bulk": 4, "pipelin": [4, 7], "littl": 4, "difficult": 4, "refactor": 4, "primarili": 4, "offici": 4, "facilit": 4, "reus": 4, "fact": 4, "exactli": 4, "quickstart_sst_demo": 4, "gotcha": 5, "central": 5, "piec": [5, 10], "tech": 5, "compon": [5, 7, 9, 10], "observ": 5, "orient": 5, "templat": 5, "declar": 5, "isol": 5, "center": 5, "data": [5, 6, 9], "scalabl": 5, "highli": 5, "fairli": 5, "digest": 5, "heavier": 5, "angular": 5, "roughli": [5, 7], "divid": [5, 10], "itself": 5, "coordin": 5, "footer": 5, "lai": 5, "variou": 5, "go": 5, "later": 5, "bundl": 5, "attach": 5, "dom": 5, "kick": 5, "singleton": [5, 7], "arraign": 5, "minimalist": [5, 10], "bottom": [5, 10], "analyt": [5, 10], "put": [5, 10], "exploratori": [5, 10], "inspect": [5, 10], "v1": [5, 10], "appreci": [5, 10], "custom_layout": 5, "updat": [5, 10], "factor": 5, "model": [5, 7, 9], "modules_servic": 5, "fetch": 5, "static": 5, "helper": [5, 7], "outlin": [5, 10], "dummi": 5, "pig": 5, "customel": 5, "demotextmodul": 5, "selectionserviceindex": 5, "shouldreact": 5, "duplicateformodelcomparison": 5, "4": 5, "privat": 5, "readonli": 5, "colorservic": 5, "getservic": 5, "piglatin": 5, "firstupd": 5, "reactimmedi": 5, "selectionservic": [5, 7], "primaryselectedinputdata": 5, "7": [5, 10], "gettransl": 5, "async": 5, "indexedinput": 5, "const": 5, "promis": 5, "apiservic": 5, "getpiglatin": 5, "8": 5, "await": 5, "loadlatest": 5, "renderimpl": 5, "getdatapointcolor": 5, "div": 5, "stylemap": 5, "checkmodul": 5, "modelspec": 5, "modelsmap": 5, "datasetspec": 5, "11": 5, "12": 5, "htmlelementtagnamemap": 5, "illustr": 5, "definit": 5, "css": 5, "shared_styl": 5, "super": 5, "inject": 5, "easier": 5, "mock": 5, "appstat": 5, "u": [5, 10], "decor": 5, "unit": 5, "reactiv": 5, "mobxlitel": 5, "trigger": 5, "excel": 5, "asynchron": 5, "leverag": 5, "machineri": 5, "lifecycl": 5, "reaction": 5, "whatev": [5, 10], "immedi": 5, "whenev": 5, "react": 5, "won": 5, "condit": 5, "rapidli": 5, "supersed": 5, "recent": 5, "rerend": 5, "seem": 5, "pure": 5, "worth": 5, "sometim": [5, 7], "inadequ": 5, "advanc": 5, "suit": 5, "anim": 5, "imper": 5, "draw": 5, "canva": 5, "fortun": 5, "bridg": 5, "queryselector": 5, "shadow": 5, "root": 5, "mess": 5, "reconcili": 5, "shadowroot": 5, "drawcanva": 5, "faceting_control": 5, "construct": 5, "programmat": 5, "member": 5, "oppos": [5, 10], "destroi": 5, "recreat": 5, "brought": 5, "lose": 5, "previous": [5, 10], "held": 5, "snippet": 5, "examplemodul": 5, "facetingcontrol": 5, "createel": 5, "facetschang": 5, "event": 5, "customev": 5, "contextnam": 5, "addeventlisten": 5, "eventlisten": 5, "clang": 5, "instead": [5, 10], "id": [5, 10], "dosometh": 5, "litel": 5, "reactiveel": 5, "annotated_text_modul": 5, "annotated_text_vi": 5, "On": [5, 10], "regist": 5, "properli": 5, "clean": 5, "leav": 5, "yarn": 5, "termin": 5, "cd": 5, "watch": 5, "m": [5, 6], "example_nam": 5, "localhost": [5, 6], "ctrl": 5, "cmd": 5, "r": 5, "hard": 5, "refresh": 5, "pick": 5, "hot": 5, "subsequ": 5, "conjunct": 5, "stale": 5, "effort": 5, "potato": [5, 7], "theme": 5, "custom_modul": 5, "env": 5, "asset": 5, "client_root": 5, "potato_demo": 5, "parent_dir": 5, "o": 5, "join": 5, "pathlib": 5, "__file__": 5, "absolut": 5, "enum": 5, "litmodulenam": 5, "potato_layout": 5, "classificationmodul": 5, "spud": 5, "tastic": 5, "jump": [6, 9], "short": 6, "video": [6, 10], "rout": 6, "foo_data": 6, "foodataset": 6, "bar_data": 6, "bardataset": 6, "my_model": 6, "mymodel": 6, "wealth": 6, "overview": [6, 9], "lit_sentiment_classifi": 6, "403": 6, "cooki": 6, "site": [6, 9], "outsid": [6, 9], "blaze": 6, "quickstart": 6, "alsologtostderr": 6, "cpu": 6, "overload": 7, "codebas": 7, "folder": 7, "fledg": 7, "job": 7, "fulli": 7, "capit": 7, "accompani": 7, "emoji": 7, "pronounc": 7, "ell": 7, "ey": 7, "tee": 7, "formerli": 7, "polym": 7, "coincident": 7, "cours": 7, "neural": 7, "network": 7, "composit": 7, "strictli": 7, "speak": 7, "litmodul": [7, 9], "noun": 7, "verb": 7, "notabl": 7, "layout": [7, 9], "maxim": 7, "gui": 7, "welcom": 9, "cite": 9, "demonstr": 9, "misc": 9, "tenney2020languag": 9, "author": 9, "ian": 9, "tennei": 9, "jame": 9, "wexler": 9, "jasmijn": 9, "bast": 9, "tolga": 9, "bolukbasi": 9, "andi": 9, "coenen": 9, "sebastian": 9, "gehrmann": 9, "ellen": 9, "jiang": 9, "mahima": 9, "pushkarna": 9, "carei": 9, "radebaugh": 9, "emili": 9, "reif": 9, "ann": 9, "yuan": 9, "booktitl": 9, "proceed": 9, "2020": 9, "confer": 9, "empir": 9, "year": 9, "publish": 9, "linguist": 9, "107": 9, "118": 9, "www": 9, "aclweb": 9, "anthologi": 9, "emnlp": 9, "stand": 9, "alon": 9, "multimod": 9, "journei": 9, "tcav": 9, "architectur": 9, "style": 9, "glossari": 9, "privaci": 9, "secur": 9, "tour": 10, "live": 10, "draggabl": 10, "alloc": 10, "vertic": 10, "horizont": 10, "adopt": 10, "organiz": 10, "scheme": 10, "reflect": 10, "focu": 10, "arrow": 10, "darker": 10, "dialog": 10, "declutt": 10, "someon": 10, "els": 10, "clear": 10, "deselect": 10, "legend": 10, "unpin": 10, "indetermin": 10, "progress": 10, "pend": 10, "fail": 10, "failur": 10, "until": 10, "x": 10, "icon": 10, "messag": 10, "star": 10, "screenshot": 10, "toggl": 10, "dimens": 10, "hover": 10, "lasso": 10, "unselect": 10, "consecut": 10, "light": 10, "sort": 10, "were": 10, "prefix": 10, "hide": 10, "exce": 10, "truncat": 10, "ellipsi": 10, "entireti": 10, "collaps": 10, "film": 10, "again": 10, "unstar": 10, "neighbor": 10, "immut": 10, "simplic": 10, "blank": 10, "creation": 10, "altogeth": 10, "highest": 10, "opposit": 10, "hello": 10, "bad": 10, "exact": 10, "broken": 10, "made": 10, "versu": 10, "ax": 10, "agreement": 10, "disagr": 10, "clickabl": 10, "disagre": 10, "laid": 10, "axi": 10, "methodologi": 10, "capabl": 10, "wise": 10, "autorun": 10, "uncheck": 10, "valuabl": 10, "expens": 10, "attend": 10, "opac": 10, "magnitud": 10, "negat": 10, "stanford": 10, "treebank": 10, "56": 10, "surprisingli": 10, "But": 10, "truli": 10, "review": 10, "ultim": 10, "depress": 10, "era": 10, "gangster": 10, "strongli": 10, "worst": 10, "elicit": 10, "mildli": 10, "incorrect": 10, "occup": 10, "technician": 10, "particip": 10, "male": 10, "domin": 10, "25": 10, "femal": 10, "bl": 10, "agre": 10, "stereotyp": 10, "resolv": 10, "83": 10, "37": 10, "cnn": 10, "dm": 10, "middl": 10, "erron": 10, "constitu": 10, "alastair": 10, "cook": 10, "captain": 10, "dig": 10, "deeper": 10, "28": 10, "arriv": 10, "searcher": 10, "fast": 10, "approxim": 10, "nearest": 10, "corpu": 10, "retriev": 10, "34": 10, "16": 10, "occurr": 10, "strong": 10, "toward": 10, "phrase": 10}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"lit": [0, 3, 4, 6, 9], "python": 0, "api": 0, "design": 0, "overview": [0, 5], "ad": 0, "model": [0, 1, 2, 4, 6, 10], "data": [0, 1, 2, 4, 10], "valid": 0, "dataset": [0, 4], "transform": [0, 1], "more": 0, "output": [0, 1, 10], "option": [0, 10], "input": [0, 1], "interpret": [0, 9], "compon": [0, 1, 4, 6], "metric": [0, 1, 10], "gener": [0, 1, 10], "backtransl": 0, "configur": 0, "ui": [0, 4, 10], "type": [0, 1, 4], "system": 0, "convent": 0, "compat": 0, "check": 0, "an": [0, 6], "In": 0, "depth": 0, "exampl": [0, 1, 4, 6], "avail": 0, "server": 0, "custom": [0, 3, 5, 6], "layout": [0, 5, 10], "access": 0, "notebook": [0, 6], "us": [0, 4, 6], "outsid": [0, 4], "featur": 1, "framework": 1, "support": 1, "huggingfac": 1, "tf1": 1, "x": 1, "estim": 1, "remot": 1, "host": [1, 6], "static": 1, "predict": [1, 2], "load": [1, 4], "classif": [1, 2], "regress": [1, 2], "score": [1, 2], "multi": 1, "label": 1, "seq2seq": [1, 2], "span": 1, "structur": [1, 2], "multipl": 1, "segment": 1, "tabular": [1, 2], "imag": [1, 2, 3], "token": 1, "base": 1, "salienc": [1, 10], "gradient": 1, "norm": 1, "dot": 1, "integr": [1, 3, 4], "lime": 1, "target": 1, "select": [1, 10], "sequenc": [1, 2], "cluster": 1, "attribut": 1, "pixel": 1, "attent": [1, 10], "embed": [1, 10], "projector": [1, 10], "aggreg": 1, "analysi": [1, 10], "confus": [1, 10], "matrix": [1, 10], "scalar": [1, 10], "plot": 1, "binari": 1, "threshold": 1, "partial": 1, "depend": 1, "dive": 1, "tcav": 1, "statist": 1, "signific": 1, "sort": 1, "cosin": 1, "similar": [1, 2], "counterfactu": 1, "manual": 1, "edit": 1, "demo": [2, 6], "do": [2, 6], "NOT": [2, 6], "remov": [2, 6], "section": 2, "header": 2, "sentiment": [2, 10], "nli": 2, "glue": 2, "multilingu": 2, "xnli": 2, "textual": 2, "st": 2, "b": 2, "stsb": 2, "gemma": 2, "t5": 2, "languag": [2, 4], "lm": 2, "bert": 2, "gpt": 2, "2": 2, "gender": [2, 10], "bia": [2, 10], "corefer": [2, 10], "coref": 2, "multimod": 2, "penguin": 2, "mobilenet": 2, "run": [3, 6], "docker": 3, "contain": 3, "basic": 3, "usag": 3, "instanc": 3, "default": 3, "build": 3, "your": [3, 6], "own": 3, "frequent": 4, "ask": 4, "question": 4, "scale": 4, "size": 4, "larg": 4, "privaci": 4, "secur": 4, "i": 4, "have": 4, "proprietari": 4, "my": 4, "team": 4, "workflow": 4, "send": 4, "from": 4, "anoth": 4, "tool": [4, 9], "download": 4, "export": 4, "train": 4, "frontend": 5, "develop": 5, "guid": [5, 10], "high": 5, "level": 5, "applic": 5, "architectur": 5, "bootstrap": 5, "initi": 5, "modul": [5, 10], "litmodul": 5, "setup": 5, "function": 5, "escap": 5, "hatch": 5, "state": 5, "child": 5, "element": 5, "style": 5, "tip": 5, "open": 5, "sourc": 5, "client": 5, "get": 6, "start": 6, "instal": 6, "colab": 6, "stand": 6, "alon": 6, "standalon": 6, "exist": 6, "glossari": 7, "learn": 9, "research": 9, "datapoint": 10, "toolbar": 10, "top": 10, "bar": 10, "global": 10, "set": 10, "url": 10, "share": 10, "main": 10, "statu": 10, "compar": 10, "slice": 10, "detail": 10, "tabl": 10, "editor": 10, "map": 10, "user": 10, "journei": 10, "debug": 10, "text": 10}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 60}, "alltitles": {"LIT Python API": [[0, "lit-python-api"]], "Design Overview": [[0, "design-overview"]], "Adding Models and Data": [[0, "adding-models-and-data"]], "Validating Models and Data": [[0, "validating-models-and-data"]], "Datasets": [[0, "datasets"]], "Transformations": [[0, "transformations"]], "Models": [[0, "models"]], "Adding more outputs": [[0, "adding-more-outputs"]], "Optional inputs": [[0, "optional-inputs"]], "Interpretation Components": [[0, "interpretation-components"]], "Metrics": [[0, "metrics"], [1, "metrics"]], "Generators": [[0, "generators"], [1, "generators"]], "Backtranslator Generator": [[0, "backtranslator-generator"]], "Configuration UI": [[0, "configuration-ui"]], "Type System": [[0, "type-system"]], "Conventions": [[0, "conventions"]], "Compatibility Checks": [[0, "compatibility-checks"]], "An In-Depth Example": [[0, "an-in-depth-example"]], "Available types": [[0, "available-types"]], "Server Configuration": [[0, "server-configuration"]], "Customizing the Layout": [[0, "customizing-the-layout"]], "Accessing the LIT UI in Notebooks": [[0, "accessing-the-lit-ui-in-notebooks"]], "Using LIT components outside of LIT": [[0, "using-lit-components-outside-of-lit"]], "Components and Features": [[1, "components-and-features"]], "Framework and Model Support": [[1, "framework-and-model-support"]], "HuggingFace Transformers": [[1, "huggingface-transformers"]], "TF1.x Estimator": [[1, "tf1-x-estimator"]], "Remote or hosted models": [[1, "remote-or-hosted-models"]], "Static predictions": [[1, "static-predictions"]], "Data loading": [[1, "data-loading"]], "Input and Output Types": [[1, "input-and-output-types"]], "Classification": [[1, "classification"]], "Regression / Scoring": [[1, "regression-scoring"]], "Multi-label classification": [[1, "multi-label-classification"]], "Seq2Seq / Generation": [[1, "seq2seq-generation"]], "Span Labeling and Structured Prediction": [[1, "span-labeling-and-structured-prediction"]], "Multiple input segments": [[1, "multiple-input-segments"]], "Tabular data": [[1, "tabular-data"]], "Images": [[1, "images"]], "Token-based Salience": [[1, "token-based-salience"]], "Gradient Norm": [[1, "gradient-norm"]], "Gradient-dot-Input": [[1, "gradient-dot-input"]], "Integrated Gradients": [[1, "integrated-gradients"]], "LIME": [[1, "lime"]], "Target Selection on Classification Output": [[1, "target-selection-on-classification-output"]], "Sequence Salience": [[1, "sequence-salience"]], "Salience Clustering": [[1, "salience-clustering"]], "Tabular Feature Attribution": [[1, "tabular-feature-attribution"]], "Pixel-based Salience": [[1, "pixel-based-salience"]], "Attention": [[1, "attention"], [10, "attention"]], "Embedding Projector": [[1, "embedding-projector"], [10, "embedding-projector"]], "Aggregate Analysis": [[1, "aggregate-analysis"]], "Confusion Matrix": [[1, "confusion-matrix"], [10, "confusion-matrix"]], "Scalar Plots": [[1, "scalar-plots"]], "Binary Classification Thresholds": [[1, "binary-classification-thresholds"]], "Partial Dependence Plots": [[1, "partial-dependence-plots"]], "Dive": [[1, "dive"]], "TCAV": [[1, "tcav"]], "Example": [[1, "example"]], "Statistical Significance": [[1, "statistical-significance"]], "Sorting by Cosine Similarity": [[1, "sorting-by-cosine-similarity"]], "Counterfactual Analysis": [[1, "counterfactual-analysis"]], "Manual Editing": [[1, "manual-editing"]], "Demos": [[2, "demos"]], "Classification <!-- DO NOT REMOVE {#classification .demo-section-header} -->": [[2, "classification"]], "Sentiment and NLI <!-- DO NOT REMOVE {#glue .demo-header} -->": [[2, "sentiment-and-nli"]], "Multilingual (XNLI) <!-- DO NOT REMOVE {#xnli .demo-header} -->": [[2, "multilingual-xnli"]], "Regression / Scoring <!-- DO NOT REMOVE {#regression-scoring .demo-section-header} -->": [[2, "regression-scoring"]], "Textual Similarity (STS-B) <!-- DO NOT REMOVE {#stsb .demo-header} -->": [[2, "textual-similarity-sts-b"]], "Sequence-to-Sequence <!-- DO NOT REMOVE {#seq2seq .demo-section-header} -->": [[2, "sequence-to-sequence"]], "Gemma <!-- DO NOT REMOVE {#gemma .demo-header} -->": [[2, "gemma"]], "T5 <!-- DO NOT REMOVE {#t5 .demo-header} -->": [[2, "t5"]], "Language Modeling <!-- DO NOT REMOVE {#lm .demo-section-header} -->": [[2, "language-modeling"]], "BERT and GPT-2 <!-- DO NOT REMOVE {#bert .demo-header} -->": [[2, "bert-and-gpt-2"]], "Structured Prediction <!-- DO NOT REMOVE {#structured .demo-section-header} -->": [[2, "structured-prediction"]], "Gender Bias in Coreference <!-- DO NOT REMOVE {#coref .demo-header} -->": [[2, "gender-bias-in-coreference"]], "Multimodal <!-- DO NOT REMOVE {#multimodal .demo-section-header} -->": [[2, "multimodal"]], "Tabular Data: Penguin Classification <!-- DO NOT REMOVE {#penguin .demo-header} -->": [[2, "tabular-data-penguin-classification"]], "Image Classification with MobileNet <!-- DO NOT REMOVE {#mobilenet .demo-header} -->": [[2, "image-classification-with-mobilenet"]], "Running LIT in a Docker container": [[3, "running-lit-in-a-docker-container"]], "Basic Usage": [[3, "basic-usage"]], "Integrating Custom LIT Instances with the Default Docker Image": [[3, "integrating-custom-lit-instances-with-the-default-docker-image"]], "Building Your Own Image": [[3, "building-your-own-image"]], "Frequently Asked Questions": [[4, "frequently-asked-questions"]], "Model and Data Types": [[4, "model-and-data-types"]], "Languages": [[4, "languages"]], "Scale": [[4, "scale"]], "Dataset Size": [[4, "dataset-size"]], "Large Models": [[4, "large-models"]], "Privacy and Security": [[4, "privacy-and-security"]], "I have proprietary data. Is LIT secure for my team to use?": [[4, "i-have-proprietary-data-is-lit-secure-for-my-team-to-use"]], "Workflow and Integrations": [[4, "workflow-and-integrations"]], "Sending examples from another tool": [[4, "sending-examples-from-another-tool"]], "Downloading or exporting data": [[4, "downloading-or-exporting-data"]], "Loading data from the UI": [[4, "loading-data-from-the-ui"]], "Using components outside the LIT UI": [[4, "using-components-outside-the-lit-ui"]], "Training models with LIT": [[4, "training-models-with-lit"]], "Frontend Developer Guide": [[5, "frontend-developer-guide"]], "High Level Overview": [[5, "high-level-overview"]], "Application Architecture": [[5, "application-architecture"]], "Bootstrapping": [[5, "bootstrapping"]], "Layout": [[5, "layout"]], "Initialization": [[5, "initialization"]], "Modules (LitModule)": [[5, "modules-litmodule"]], "Setup": [[5, "setup"]], "Functionality": [[5, "functionality"]], "Escape Hatches": [[5, "escape-hatches"]], "Stateful Child Elements": [[5, "stateful-child-elements"]], "Style Guide": [[5, "style-guide"]], "Development Tips (open-source)": [[5, "development-tips-open-source"]], "Custom Client / Modules": [[5, "custom-client-modules"]], "Getting Started with LIT": [[6, "getting-started-with-lit"]], "Installation": [[6, "installation"]], "Hosted demos": [[6, "hosted-demos"]], "LIT with your model <!-- DO NOT REMOVE {#custom-demos} -->": [[6, "lit-with-your-model"]], "Using LIT in notebooks <!-- DO NOT REMOVE {#colab} -->": [[6, "using-lit-in-notebooks"]], "Stand-alone components <!-- DO NOT REMOVE {#standalone} -->": [[6, "stand-alone-components"]], "Run an existing example <!-- DO NOT REMOVE {#running-lit} -->": [[6, "run-an-existing-example"]], "Glossary": [[7, "glossary"]], "Learning Interpretability Tool (LIT)": [[9, "learning-interpretability-tool-lit"]], "Research": [[9, "research"]], "UI Guide": [[10, "ui-guide"]], "General Layout": [[10, "general-layout"]], "Layout Options": [[10, "layout-options"]], "Datapoint Selections": [[10, "datapoint-selections"]], "Toolbars": [[10, "toolbars"]], "Top Bar": [[10, "top-bar"]], "Global Settings": [[10, "global-settings"]], "URL Sharing": [[10, "url-sharing"]], "Main Toolbar": [[10, "main-toolbar"]], "Status Bar": [[10, "status-bar"]], "Comparing Models": [[10, "comparing-models"]], "Comparing Datapoints": [[10, "comparing-datapoints"]], "Slices": [[10, "slices"]], "Module Details": [[10, "module-details"]], "Data Table": [[10, "data-table"]], "Datapoint Editor": [[10, "datapoint-editor"]], "Datapoint Generator": [[10, "datapoint-generator"]], "Metrics Table": [[10, "metrics-table"]], "Scalars": [[10, "scalars"]], "Model Output": [[10, "model-output"]], "Salience Maps": [[10, "salience-maps"]], "User Journeys": [[10, "user-journeys"]], "Sentiment Analysis": [[10, "sentiment-analysis"]], "Gender Bias in Coreference": [[10, "gender-bias-in-coreference"]], "Debugging Text Generation": [[10, "debugging-text-generation"]]}, "indexentries": {}})
\ No newline at end of file
+Search.setIndex({"docnames": ["api", "components", "demos", "docker", "faq", "frontend_development", "getting_started", "glossary", "includes/highlight_demos", "index", "ui_guide"], "filenames": ["api.md", "components.md", "demos.md", "docker.md", "faq.md", "frontend_development.md", "getting_started.md", "glossary.md", "includes/highlight_demos.md", "index.md", "ui_guide.md"], "titles": ["LIT Python API", "Components and Features", "Demos", "Running LIT in a Docker container", "Frequently Asked Questions", "Frontend Developer Guide", "Getting Started with LIT", "Glossary", "&lt;no title&gt;", "Learning Interpretability Tool (LIT)", "UI Guide"], "terms": {"i": [0, 1, 3, 5, 6, 7, 10], "modular": 0, "compris": [0, 10], "collect": [0, 4, 5, 10], "backend": [0, 4, 5, 7, 10], "written": [0, 4], "frontend": [0, 4, 7, 9], "modul": [0, 1, 2, 3, 4, 7, 9], "typescript": [0, 5], "most": [0, 1, 4, 5, 6, 7, 10], "user": [0, 1, 3, 4, 5, 6, 9], "develop": [0, 1, 4, 9, 10], "against": [0, 1, 2, 4, 10], "which": [0, 1, 2, 3, 4, 5, 6, 7, 10], "document": [0, 1, 4, 5, 6], "below": [0, 1, 2, 3, 5, 10], "allow": [0, 1, 4, 5, 10], "extend": [0, 5], "counterfactu": [0, 2, 6, 7, 9, 10], "The": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10], "ar": [0, 1, 3, 4, 5, 6, 7, 10], "provid": [0, 1, 3, 4, 5, 6, 7, 10], "librari": [0, 1, 5], "can": [0, 1, 3, 4, 5, 6, 10], "through": [0, 1, 4, 10], "own": [0, 1, 9], "demo": [0, 1, 3, 4, 5, 7, 9], "binari": [0, 2, 6, 7, 10], "via": [0, 1, 4], "colab": [0, 1, 4], "also": [0, 1, 3, 4, 5, 6, 10], "regular": [0, 4, 7], "class": [0, 1, 4, 5, 6, 7, 10], "without": [0, 1, 4, 5, 10], "start": [0, 1, 3, 4, 5, 9], "see": [0, 1, 2, 4, 5, 6, 7, 10], "detail": [0, 1, 3, 4, 5, 6, 9], "serv": [0, 1, 3, 4, 5, 6, 7], "each": [0, 1, 2, 3, 5, 6, 10], "implement": [0, 1, 3, 4, 5, 6, 7], "minim": [0, 1, 3, 7, 10], "reli": [0, 5], "spec": [0, 1, 5], "detect": 0, "field": [0, 1, 4, 10], "verifi": [0, 1, 3, 10], "stateless": 0, "cach": [0, 4, 5, 6], "layer": [0, 1, 2, 4, 10], "predict": [0, 4, 5, 7, 9, 10], "thi": [0, 1, 2, 3, 4, 5, 6, 7, 10], "simplifi": [0, 5], "interact": [0, 1, 2, 4, 5, 7, 9, 10], "larg": [0, 1], "like": [0, 1, 4, 5, 7, 10], "bert": [0, 1, 4, 10], "t5": [0, 1, 4, 10], "state": [0, 4, 7, 10], "singl": [0, 1, 2, 4, 5, 6, 7, 10], "page": [0, 1, 5, 6, 9, 10], "app": [0, 3, 4, 5, 10], "built": [0, 2, 3, 5, 6, 7, 10], "1": [0, 1, 4, 5, 6, 10], "mobx": [0, 5], "manag": [0, 1, 4, 5], "It": [0, 1, 5, 10], "consist": [0, 1, 5, 7, 10], "core": [0, 5], "framework": [0, 4, 5, 6, 7, 9], "set": [0, 1, 2, 3, 4, 5, 6, 7], "share": [0, 4, 5], "servic": [0, 5, 7], "persist": [0, 1, 10], "independ": 0, "render": [0, 1, 5, 6, 7], "visual": [0, 1, 2, 4, 5, 7, 9, 10], "support": [0, 2, 4, 5, 6, 9, 10], "For": [0, 1, 2, 4, 5, 6, 10], "guid": [0, 1, 4, 6, 7, 9], "To": [0, 1, 6, 10], "run": [0, 1, 4, 5, 7, 9, 10], "your": [0, 1, 4, 5, 9, 10], "you": [0, 1, 3, 4, 5, 6, 9, 10], "creat": [0, 1, 3, 4, 5, 7, 10], "py": [0, 1, 2, 3, 4, 5, 6, 7], "script": [0, 3, 4, 6], "pass": [0, 1, 3, 4, 5, 6, 10], "def": [0, 1, 3], "main": [0, 1, 3, 5, 9], "_": 0, "mulitinlidata": 0, "mnli_match": 0, "multinlidata": 0, "path": [0, 1, 3, 4, 5, 6], "dev_match": 0, "tsv": [0, 6], "mnli_mismatch": 0, "dev_mismatch": 0, "nlimodel": 0, "model_foo": 0, "foo": [0, 5, 6], "file": [0, 3, 4, 5, 6], "model_bar": 0, "bar": [0, 1, 6, 7], "lit_demo": [0, 6], "lit_nlp": [0, 3, 5, 6], "dev_serv": [0, 3, 5, 6], "port": [0, 3, 5, 6], "4321": [0, 6], "__name__": 0, "__main__": 0, "conceptu": [0, 5], "just": [0, 4, 5, 6, 10], "list": [0, 1, 3, 4, 6, 10], "function": [0, 1, 2, 3, 10], "take": [0, 1, 4, 5, 6, 10], "return": [0, 1, 3, 5, 10], "metadata": [0, 1, 5], "describ": [0, 1, 3, 5, 6, 10], "themselv": 0, "other": [0, 1, 2, 4, 5, 6, 7, 10], "pre": [0, 1, 2, 4, 5, 6, 10], "out": [0, 1, 2, 5, 6, 9, 10], "http": [0, 1, 2, 3, 5, 6, 7, 9], "github": [0, 1, 2, 3, 4, 6, 9], "com": 0, "pair": [0, 1, 2, 3, 6, 9], "code": [0, 1, 2, 3, 4, 5, 6, 9], "tree": 0, "ensur": [0, 1, 3, 5], "match": [0, 4], "valu": [0, 1, 3, 4, 5, 10], "veri": [0, 1, 2, 5], "help": [0, 1, 3, 4, 10], "dure": [0, 5], "new": [0, 1, 4, 5, 10], "wrapper": [0, 1, 2, 4, 7], "correct": [0, 10], "behavior": [0, 1, 5], "At": [0, 10], "startup": 0, "flag": [0, 2, 3, 4, 5, 6, 10], "enabl": [0, 1, 6, 10], "There": [0, 1, 4, 7, 10], "three": [0, 2, 4, 5, 10], "mode": [0, 1, 2, 5, 10], "first": [0, 4, 5, 10], "sampl": [0, 1, 4], "5": [0, 1, 2, 5, 10], "all": [0, 1, 3, 4, 5, 7, 10], "from": [0, 1, 2, 3, 5, 6, 7, 10], "addition": [0, 1, 10], "call": [0, 1, 3, 5, 10], "directli": [0, 1, 4, 5, 6, 10], "associ": [0, 2, 5, 9, 10], "inform": [0, 4, 5, 10], "follow": [0, 3, 5, 10], "": [0, 1, 2, 3, 4, 5, 10], "should": [0, 1, 3, 4, 5, 6], "flat": 0, "dict": [0, 1], "self": [0, 1, 5], "_exampl": 0, "oper": [0, 4], "load": [0, 2, 5, 6, 10], "includ": [0, 1, 2, 3, 4, 5, 7, 10], "therefor": [0, 3], "care": 0, "size": [0, 1, 2], "fit": [0, 4], "memori": [0, 1, 4], "displai": [0, 1, 5, 10], "browser": [0, 4, 5], "note": [0, 1, 3, 5, 6, 10], "faq": [0, 1, 9], "limit": [0, 4], "subclass": [0, 5], "usual": [0, 1, 7], "few": [0, 1, 4, 5, 6, 7], "line": [0, 1, 3, 6, 10], "complet": [0, 1, 3, 10], "multinli": [0, 2], "loader": [0, 1], "nli_label": 0, "entail": 0, "neutral": 0, "contradict": 0, "__init__": [0, 1], "str": [0, 1, 3], "read": [0, 4, 5], "eval": [0, 2], "distribut": [0, 1], "glue": [0, 1, 3], "benchmark": 0, "df": 0, "panda": 0, "read_csv": 0, "sep": 0, "t": [0, 1, 3, 4, 5, 7, 10], "store": [0, 4, 5, 10], "conform": 0, "premis": [0, 2], "row": [0, 1, 10], "sentence1": 0, "hypothesi": [0, 1, 2], "sentence2": 0, "label": [0, 2, 10], "gold_label": 0, "genr": 0, "iterrow": 0, "lit_typ": [0, 1], "textseg": [0, 1], "categorylabel": [0, 1], "vocab": [0, 1], "we": [0, 1, 3, 4, 5, 6, 7, 10], "addit": [0, 1, 3, 4, 5, 6, 10], "don": [0, 1, 4, 5], "have": [0, 1, 3, 5, 10], "four": 0, "string": [0, 1, 4, 5], "semant": [0, 1], "tell": 0, "bit": [0, 5], "about": [0, 1, 5, 10], "how": [0, 1, 3, 5, 6, 10], "them": [0, 1, 4, 5, 10], "treat": [0, 1, 4], "natur": [0, 2, 9], "languag": [0, 1, 7, 9, 10], "text": [0, 1, 2, 4], "categor": [0, 1, 2, 4, 10], "featur": [0, 2, 4, 5, 6, 7, 9, 10], "fix": [0, 1], "known": [0, 7, 10], "possibl": [0, 1, 5], "unknown": 0, "open": [0, 1, 9], "tensorflow": [0, 2], "simpli": [0, 1, 3], "wrap": [0, 1, 4], "freeli": 0, "add": [0, 1, 3, 4, 5, 10], "abov": [0, 1, 3, 4, 5, 6, 10], "mai": [0, 1, 4, 5, 6, 7, 10], "awar": 0, "recogn": [0, 1], "slice": [0, 1, 2, 4, 7, 9], "bin": [0, 1], "etc": [0, 3, 4], "thei": [0, 1, 4, 5, 10], "similar": [0, 3, 10], "tf": [0, 1], "scope": 0, "aim": [0, 5], "quick": [0, 1, 2, 10], "iter": 0, "step": [0, 3, 5], "end": [0, 3, 10], "same": [0, 1, 5], "datapoint": [0, 1, 4, 7, 9], "n": 0, "seed": 0, "42": 0, "random": [0, 1, 10], "remap": 0, "field_map": 0, "renam": 0, "both": [0, 1, 5], "latter": 0, "shortcut": 0, "one": [0, 1, 3, 4, 5, 6, 7, 10], "anoth": [0, 1, 5, 7, 10], "expect": [0, 4], "2": [0, 1, 5], "produc": [0, 1, 3, 10], "method": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10], "input_spec": [0, 1], "necessari": [0, 3, 5], "output_spec": [0, 1], "ani": [0, 1, 3, 4, 5, 10], "sequenc": [0, 3, 9], "satisfi": 0, "yield": 0, "parallel": 0, "might": [0, 1, 3, 10], "look": [0, 1, 3, 4, 10], "someth": [0, 1, 5, 7], "infer": [0, 1, 2, 4, 5, 6, 10], "model_path": 0, "kw": 0, "so": [0, 1, 3, 4, 10], "re": [0, 4, 5, 7], "readi": 0, "_model": 0, "_load_my_model": 0, "pred": 0, "stream": 0, "convert_dict_input": 0, "d": [0, 3, 10], "preprocess": 0, "predict_exampl": 0, "parent": [0, 1, 5], "keyword": 0, "where": [0, 1, 2, 3, 5, 6, 10], "gold": [0, 1, 2], "when": [0, 1, 4, 5, 7, 10], "comput": [0, 1, 4, 5, 9], "proba": [0, 1], "multiclasspr": [0, 1, 10], "unlik": [0, 1], "incomplet": 0, "ll": [0, 1, 4, 5], "need": [0, 1, 3, 4, 5, 6, 7], "accordingli": [0, 1], "post": [0, 1, 2], "process": [0, 1, 2, 5, 9], "token": [0, 2, 4, 9, 10], "mani": [0, 1, 3, 4, 5, 6, 7, 10], "deep": 0, "learn": [0, 1, 7, 10], "batch": 0, "thu": [0, 5], "batchedmodel": 0, "simpl": [0, 1, 5, 6, 10], "must": [0, 1, 5, 10], "predict_minibatch": 0, "convert": [0, 1], "jsondict": 0, "object": [0, 4], "appropri": [0, 1, 4], "represent": [0, 1, 2], "typic": [0, 1], "map": [0, 1, 2, 3, 5], "align": [0, 1], "tensor": 0, "befor": [0, 3, 4, 6, 10], "want": [0, 1, 3, 4, 5, 6, 9, 10], "overrid": [0, 3, 5], "max_minibatch_s": 0, "determin": [0, 1, 5, 10], "If": [0, 1, 4, 5, 6, 9, 10], "remot": [0, 7], "consid": 0, "batchedremotemodel": 0, "base": [0, 2, 3, 5, 9, 10], "request": [0, 4, 5], "thread": 0, "pool": 0, "defin": [0, 1, 3, 5, 10], "black": [0, 1], "box": [0, 1, 10], "intern": [0, 1, 5], "richer": 0, "view": [0, 1, 4, 10], "correspond": [0, 1], "hidden": [0, 5], "activ": [0, 1, 10], "gradient": [0, 2, 10], "word": [0, 1, 2, 10], "embed": [0, 2, 4, 9], "attent": [0, 2, 9], "sever": [0, 1, 3, 4], "two": [0, 1, 2, 3, 5, 10], "differ": [0, 1, 2, 4, 5, 10], "vector": [0, 1], "easili": [0, 1, 3, 4, 6, 10], "output_emb": 0, "cl": 0, "top": [0, 1, 5], "mean_word_emb": 0, "mean": [0, 1, 10], "than": [0, 1, 4, 5, 10], "segment": [0, 2], "premise_token": 0, "hypothesis_token": 0, "here": [0, 3, 5], "refer": [0, 1, 2, 3, 5, 7, 10], "premise_grad": 0, "tokengradi": [0, 1], "hypothesis_grad": 0, "similarli": [0, 1, 10], "full": [0, 1, 4, 5, 7, 10], "baz": 0, "spam": 0, "egg": 0, "attention_layer0": 0, "attentionhead": [0, 1], "attention_layer1": 0, "attention_layer2": 0, "sinc": [0, 1, 4, 5], "dictionari": [0, 5], "dataclass": 0, "popul": 0, "loop": 0, "would": [0, 1], "entri": [0, 5, 10], "becaus": [0, 1, 3, 5], "often": [0, 1, 7], "tightli": 0, "coupl": [0, 4], "intermedi": 0, "level": [0, 1, 2, 9, 10], "expos": 0, "automat": [0, 1, 5, 10], "projector": [0, 2, 4, 9], "salienc": [0, 2, 6, 7, 9], "multi": [0, 2, 4, 10], "head": [0, 1, 10], "wai": [0, 1, 2, 4, 5, 10], "By": [0, 1, 10], "default": [0, 1, 4, 5, 6, 9, 10], "requir": [0, 1, 3, 5, 10], "howev": [0, 4, 5], "fals": [0, 1, 10], "wish": 0, "accept": [0, 1], "And": [0, 3, 5], "logic": [0, 1, 5], "bypass": 0, "input_token": [0, 1], "ex": 0, "get": [0, 1, 3, 5, 7, 9, 10], "rest": [0, 1], "though": [0, 1, 5, 6, 10], "omit": 0, "entir": [0, 1, 2, 10], "aid": 0, "umap": [0, 1, 2, 7, 10], "plug": [0, 1], "ins": 0, "some": [0, 1, 2, 5, 6, 10], "3": [0, 1, 5, 10], "local": [0, 1, 4], "gradientnorm": 0, "score": [0, 9, 10], "integr": [0, 2, 9], "tokenembed": [0, 1], "well": [0, 1, 2, 4, 5, 6, 10], "target": [0, 2, 10], "pin": [0, 1, 4, 10], "involv": [0, 1, 3, 4, 6], "lit_model": 0, "lit_dataset": 0, "model_output": 0, "none": [0, 1, 5, 10], "config": 0, "runtim": [0, 1, 4], "threshold": [0, 2, 10], "classif": [0, 4, 9, 10], "unconstrain": 0, "up": [0, 1, 4, 5, 10], "correctli": [0, 10], "particular": [0, 1, 5, 10], "while": [0, 1, 3, 4, 5, 10], "aggreg": [0, 9], "summari": 0, "respons": [0, 1, 2, 5], "what": [0, 1, 3, 5, 10], "A": [0, 1, 3, 5, 7, 10], "over": [0, 1, 7, 10], "relev": 0, "find_field": 0, "find": [0, 1, 2, 3, 10], "grad_field": 0, "util": [0, 10], "find_spec_kei": 0, "f": 0, "tokens_field": 0, "pytyp": 0, "disabl": [0, 5], "attribut": [0, 5, 9], "error": [0, 2, 6, 7, 10], "assert": 0, "isinst": 0, "given": [0, 1, 5, 10], "log": [0, 1, 3, 10], "info": [0, 3], "found": [0, 3, 5, 7, 9, 10], "len": 0, "0": [0, 1, 2, 5, 6, 10], "pylint": 0, "g": [0, 1, 3, 4, 7, 10], "explicit": [0, 4, 5], "length": [0, 1, 4, 10], "test": [0, 1, 2, 5], "do": [0, 1, 4, 5], "work": [0, 1, 2, 4, 5], "dtype": [0, 1], "tokensali": 0, "ha": [0, 1, 4, 5, 10], "overhead": 0, "benefit": 0, "flexibl": [0, 1], "specifi": [0, 1, 2, 4, 5], "complex": [0, 1, 5], "depend": [0, 2, 3, 4, 5, 10], "between": [0, 1, 2, 5, 10], "multipl": [0, 2, 3, 5, 10], "simplemetr": 0, "unpack": 0, "is_compat": 0, "name": [0, 1, 2, 3, 4, 5, 7, 10], "regressionmetr": 0, "standard": [0, 1, 4, 10], "regress": [0, 9, 10], "field_spec": 0, "littyp": 0, "bool": 0, "true": [0, 2, 3, 5, 10], "regressionscor": [0, 1], "float": [0, 1], "label_spec": 0, "scalar": [0, 4], "pred_spec": 0, "del": 0, "mse": 0, "sklearn_metr": 0, "mean_squared_error": 0, "pearsonr": 0, "scipy_stat": 0, "spearmanr": 0, "kei": [0, 1], "compar": [0, 1, 2, 4, 9], "These": [0, 1, 3, 4, 5, 10], "onli": [0, 1, 3, 4, 5, 10], "techniqu": [0, 1, 2], "back": [0, 1, 5, 7, 10], "translat": [0, 1, 2, 4, 5, 10], "feedback": [0, 5, 10], "adversari": [0, 1, 2], "attack": [0, 1], "generate_al": 0, "arg": [0, 3, 5], "current": [0, 1, 4, 5, 10], "belong": [0, 1, 10], "conveni": [0, 4, 5, 10], "As": [0, 1, 10], "argument": [0, 3], "substitut": [0, 1], "replac": [0, 1, 10], "foreign": 0, "sourc": [0, 1, 3, 6, 9, 10], "order": [0, 1, 4, 5, 10], "paraphras": [0, 1, 10], "googl": [0, 1, 2], "cloud": [0, 2], "perform": [0, 1, 5, 10], "those": [0, 1, 3, 4, 10], "project": [0, 1, 5, 7, 10], "doc": [0, 3, 5, 10], "setup": [0, 10], "Then": [0, 1, 3, 6], "download": [0, 10], "applic": [0, 1, 7, 9, 10], "credenti": 0, "google_application_credenti": 0, "environ": [0, 3, 4], "variabl": [0, 3, 5], "point": [0, 1, 4, 5, 10], "With": [0, 5, 10], "make": [0, 1, 4, 5, 10], "backtranl": 0, "constructor": [0, 4, 5, 6], "time": [0, 4, 10], "number": [0, 1, 2, 4, 5, 10], "lime": [0, 2, 10], "pivot": 0, "dsl": 0, "auto": 0, "form": 0, "config_spec": 0, "sparsemultilabel": [0, 1], "bg": 0, "de": 0, "el": 0, "en": [0, 2], "e": [0, 1, 3, 4, 7, 10], "fr": 0, "hi": [0, 2, 10], "ru": 0, "sw": 0, "th": 0, "tr": 0, "ur": 0, "vi": 0, "zh": 0, "give": [0, 1, 6, 10], "its": [0, 1, 5, 10], "interpreter_control": 0, "slider": [0, 1, 10], "numer": [0, 1, 2, 4, 10], "min_val": 0, "max_val": 0, "boolean": [0, 5], "booleanlittyp": 0, "checkbox": [0, 1, 10], "dropdown": [0, 10], "seri": 0, "comma": [0, 10], "separ": [0, 1, 3, 4, 5, 10], "pars": [0, 1, 3, 5], "suppli": 0, "singlefieldmatch": 0, "act": [0, 1], "multifieldmatch": 0, "except": [0, 7], "select": [0, 2, 4, 5, 9], "control": [0, 1, 4, 5, 7, 10], "choos": [0, 1], "perturb": [0, 1, 2], "hotflip": [0, 1, 2, 10], "around": [0, 5], "web": [0, 4, 5, 7, 10], "record": 0, "serial": 0, "json": 0, "commun": [0, 1], "client": [0, 7, 9, 10], "introduc": [0, 5, 10], "understand": [0, 1, 2, 10], "shape": [0, 1], "being": [0, 1, 5, 10], "specif": [0, 1, 2, 5, 7, 10], "properti": [0, 5], "whose": 0, "annot": [0, 1], "hierarch": 0, "inherit": [0, 7], "canon": 0, "kind": 0, "v": [0, 10], "meta": 0, "context": 0, "implic": 0, "per": [0, 1, 2, 5, 10], "everi": [0, 1, 5, 10], "init_spec": 0, "instanc": [0, 2, 5, 7, 9], "empti": [0, 3], "noth": 0, "show": [0, 1, 2, 5, 10], "section": [0, 1, 10], "global": [0, 1, 5, 7], "subset": [0, 1, 2, 10], "paramet": [0, 3, 10], "alwai": [0, 1, 3, 7], "meta_spec": 0, "essenti": 0, "ideal": [0, 1], "wherea": 0, "slight": 0, "variat": 0, "tradit": 0, "metricresult": 0, "higher": [0, 1], "lower": [0, 5, 10], "closer": 0, "zero": 0, "better": [0, 1, 10], "encapsul": 0, "impli": 0, "index": [0, 5, 10], "element": [0, 1, 4, 7], "contain": [0, 1, 4, 5, 7, 10], "po": 0, "sequencetag": [0, 1], "item": 0, "indic": [0, 1, 10], "transit": [0, 1], "zip": 0, "strict": 0, "10": [0, 1, 5], "pseudo": 0, "ground": [0, 1, 10], "truth": [0, 1, 10], "repres": [0, 1], "appear": [0, 1, 10], "assum": [0, 3, 10], "pattern": [0, 1, 5], "manipul": [0, 5, 10], "could": [0, 1, 7, 10], "former": [0, 7, 10], "swap": 0, "mask": [0, 1, 2], "feed": [0, 1, 7], "fill": [0, 10], "plai": [0, 6, 9], "critic": 0, "role": 0, "reliabl": 0, "interoper": 0, "robust": [0, 1, 10], "univers": 0, "is_compatible_with_dataset": 0, "wordreplac": 0, "doe": [0, 1, 3, 4, 10], "curv": 0, "gradientdotinput": [0, 1], "shoulddisplaymodul": 0, "suffici": [0, 1], "caus": [0, 5, 10], "jitter": [0, 2, 10], "disappear": 0, "reorder": [0, 5], "resiz": [0, 4, 7], "switch": [0, 1, 2, 10], "heterogen": 0, "findspeckei": 0, "identifi": 0, "interest": [0, 1, 10], "respect": [0, 1, 10], "import": [0, 1, 2, 5, 10], "build": [0, 1, 5, 9], "litmetadata": 0, "litapp": [0, 5], "raw": [0, 1], "jupyt": [0, 6], "repl": 0, "encourag": [0, 1], "explicitli": [0, 1, 5, 10], "avoid": [0, 5], "chase": 0, "red": [0, 10], "her": 0, "mnli": 0, "buffet": 0, "la": 0, "cart": 0, "travel": 0, "task": [0, 1, 2, 5, 6, 10], "notic": 0, "967": 0, "024": 0, "009": 0, "classificationresult": 0, "human": [0, 1], "readabl": 0, "classification_result": 0, "predicted_class": 0, "integ": 0, "predefin": [0, 1], "rang": [0, 1, 2, 10], "overridden": 0, "seen": 0, "summar": [0, 2, 10], "tabl": [0, 1, 4, 5], "bracket": 0, "syntax": 0, "num_token": [0, 1], "numpi": [0, 1], "arrai": [0, 1], "insid": [0, 3, 10], "descript": [0, 3, 5], "untoken": 0, "generatedtext": [0, 1], "seq2seq": 0, "url": [0, 1, 4, 5, 9], "generatedurl": 0, "may": 0, "real": 0, "inappropri": 0, "searchqueri": 0, "search": [0, 1, 10], "queri": [0, 1, 2, 4, 5], "opaqu": 0, "ignor": 0, "referencetext": [0, 1], "mt": 0, "tupl": [0, 1], "generatedtextcandid": [0, 1], "candid": [0, 1, 2, 10], "beam": [0, 1, 2], "tokentopkpr": 0, "32768": 0, "32767": 0, "int": 0, "imagebyt": [0, 1], "imag": [0, 4, 9], "base64": [0, 1], "encod": [0, 1, 4, 10], "jpegbyt": 0, "pngbyte": 0, "referencescor": [0, 1], "vocabulari": 0, "multiclass": [0, 2], "probabl": [0, 1, 2, 4, 10], "num_label": 0, "non": [0, 1, 2, 4, 10], "exclus": [0, 1], "sparsemultilabelpr": [0, 1], "spars": 0, "tag": [0, 1, 3, 5, 10], "spanlabel": [0, 1], "span": [0, 10], "j": [0, 1, 5], "edgelabel": [0, 1], "edg": [0, 2], "structur": [0, 5, 9, 10], "corefer": 0, "srl": [0, 1], "arxiv": [0, 1], "org": [0, 1, 9], "ab": [0, 1], "1905": 0, "06316": 0, "multisegmentannot": [0, 1], "byte": [0, 1], "annotationclust": 0, "emb_dim": [0, 1], "imagegradi": [0, 1], "pixel": [0, 9], "image_height": [0, 1], "image_width": [0, 1], "color_channel": [0, 1], "group": [0, 1, 5, 7, 10], "num_head": [0, 1], "plain": [0, 1, 2], "further": 0, "stringlittyp": 0, "urllittyp": 0, "collis": 0, "protect": 0, "page_titl": 0, "titl": [0, 1, 5, 9], "canonical_url": [0, 10], "shortlink": 0, "copi": [0, 4, 5, 10], "link": [0, 6, 10], "default_layout": 0, "demo_mod": [0, 3, 4], "kiosk": 0, "save": [0, 1, 4, 5, 10], "untrust": 0, "inline_doc": 0, "markdown": 0, "panel": [0, 5, 7, 10], "onboard_start_doc": 0, "onboard": 0, "splash": 0, "screen": [0, 5, 10], "onboard_end_doc": 0, "last": [0, 2, 3], "server_flag": [0, 5], "litcanonicallayout": [0, 5], "lm_layout": 0, "upper": [0, 5, 10], "embeddingsmodul": 0, "datatablemodul": [0, 5], "datapointeditormodul": [0, 5], "languagemodelpredictionmodul": 0, "confusionmatrixmodul": 0, "generatormodul": 0, "lm": 0, "get_flag": 0, "lm_demo": [0, 2, 3, 5], "achiev": [0, 2], "major": [0, 10], "content": [0, 4, 10], "area": [0, 10], "left": [0, 1, 2, 5, 10], "right": [0, 1, 2, 5, 10], "shown": [0, 1, 3, 10], "initi": [0, 1, 10], "set_default": [0, 3, 5], "my_layout_nam": 0, "fly": 0, "param": [0, 4, 5], "preced": 0, "comprehens": 0, "remov": [0, 1, 5, 10], "chang": [0, 1, 3, 5, 10], "experiment": [0, 1, 4, 5, 10], "desir": [0, 2, 5, 10], "altern": [0, 1], "connect": [0, 1, 4], "after": [0, 1, 5, 10], "instal": [0, 3, 9], "pip": [0, 6], "litwidget": [0, 6], "height": [0, 5, 10], "cell": [0, 1, 4, 6, 10], "widget": [0, 4, 7], "tab": [0, 1, 4, 5, 10], "open_in_new_tab": 0, "certain": 0, "stop": 0, "shut": 0, "down": [0, 1, 10], "free": 0, "resourc": 0, "plan": 0, "doesn": [0, 7, 10], "still": [0, 1], "standalon": [0, 4], "handi": [0, 1], "reload": [0, 1, 5], "click": [0, 1, 2, 5, 10], "sst2data": 0, "print": 0, "sentenc": [0, 1, 2, 10], "glue_model": [0, 4], "sst2model": 0, "cls_emb": 0, "Or": 0, "lime_explain": 0, "lit_components_exampl": [0, 6], "ipynb": [0, 1, 6], "happi": 0, "coincid": 0, "tool": [0, 1, 5, 7, 10], "relat": [0, 1, 10], "solv": 0, "case": [0, 1, 2, 3, 4, 5, 6, 10], "question": [0, 1], "qa": 0, "trivial": 0, "practic": [0, 1, 5], "endpoint": [0, 1, 5, 7], "lit": [1, 2, 5, 7, 10], "agnost": 1, "compat": [1, 5, 10], "python": [1, 3, 4, 5, 6, 7, 9, 10], "In": [1, 4, 5, 6, 10], "ve": 1, "tf2": [1, 2], "jax": 1, "pytorch": 1, "us": [1, 2, 3, 5, 7, 9, 10], "custom": [1, 2, 4, 7, 9, 10], "c": [1, 10], "clif": 1, "rpc": 1, "aren": [1, 10], "constraint": [1, 2], "beyond": 1, "impos": 1, "platform": 1, "coexist": 1, "underli": [1, 4], "hardwar": 1, "gpu": [1, 4, 6], "scale": [1, 9], "dataset": [1, 2, 5, 6, 7, 9, 10], "kera": [1, 2, 4], "easi": [1, 5, 10], "access": [1, 4, 5, 6, 9, 10], "interpret": [1, 2, 6, 7, 10], "great": [1, 10], "place": [1, 5, 10], "small": [1, 5, 6], "experi": 1, "academ": 1, "more": [1, 3, 4, 5, 6, 7, 10], "due": 1, "graph": [1, 10], "session": 1, "cannot": 1, "invoc": 1, "weight": [1, 5], "export": [1, 5, 10], "savedmodel": [1, 2], "an": [1, 3, 4, 5, 7, 9, 10], "eager": 1, "lead": [1, 5, 10], "much": [1, 5, 10], "simpler": 1, "servomat": 1, "usag": [1, 4, 9], "remain": 1, "server": [1, 3, 4, 5, 6, 7, 9, 10], "stub": [1, 5], "handl": [1, 2, 4, 5, 7, 10], "format": [1, 2, 4, 5, 10], "convers": 1, "purpos": 1, "interfac": [1, 5], "remote_model": [1, 4], "best": [1, 2, 5], "isn": [1, 10], "situat": 1, "staticpredict": 1, "lookup": [1, 10], "quickli": [1, 10], "brows": 1, "retain": 1, "rich": 1, "almost": 1, "tfrecord": [1, 6], "capacitor": 1, "sstabl": 1, "even": 1, "sql": 1, "api": [1, 3, 4, 5, 6, 7, 9, 10], "our": [1, 3, 4, 5, 6, 9, 10], "tfd": 1, "ui": [1, 5, 6, 7, 9], "workflow": [1, 9], "extens": [1, 5, 9], "system": [1, 3, 5, 7, 9, 10], "modal": [1, 4], "common": [1, 2, 5], "nlp": [1, 3, 6, 9], "domain": 1, "ad": [1, 4, 5, 9, 10], "explor": [1, 2, 5, 7, 10], "side": [1, 2, 10], "avail": [1, 4, 5, 6, 10], "matric": 1, "basic": [1, 9], "sentiment": 1, "simple_tf2_demo": 1, "evalu": [1, 2, 4, 10], "neg": [1, 10], "design": [1, 4, 9], "null_idx": 1, "commonli": [1, 7, 10], "precis": [1, 10], "recal": 1, "f1": 1, "auc": 1, "aucpr": 1, "comment": [1, 3], "toxic": [1, 10], "posit": [1, 2, 10], "margin": [1, 2, 10], "bia": 1, "space": [1, 2, 10], "bucket": [1, 10], "facet": [1, 5, 7, 10], "scatterplot": 1, "st": 1, "b": [1, 10], "textual": 1, "glue_demo": [1, 2, 3, 6], "image_demo": [1, 2, 3], "quit": 1, "matur": [1, 5], "highlight": [1, 2, 10], "diff": [1, 2], "decod": [1, 2, 10], "emit": 1, "varieti": [1, 2, 4, 6], "part": [1, 2, 7, 10], "speech": 1, "entiti": 1, "ner": 1, "mix": 1, "privileg": 1, "otherwis": [1, 5], "kernel": 1, "shap": 1, "valid": [1, 3], "check": [1, 2, 6, 9, 10], "mark": 1, "option": [1, 3, 5], "least": 1, "multilabel": 1, "penguin": [1, 4], "stat": 1, "penguin_demo": [1, 2], "That": 1, "awai": [1, 2], "move": [1, 5], "individu": [1, 2, 4, 5, 10], "color": [1, 5, 10], "break": 1, "exist": [1, 4, 9, 10], "open_imag": 1, "classifi": [1, 2, 10], "comparison": [1, 5, 7, 10], "onc": [1, 10], "io": [1, 2, 3, 6, 7, 9], "html": [1, 2, 5, 7], "navig": [1, 6, 10], "explan": [1, 10], "proport": 1, "l2": [1, 2], "propto": 1, "nabla_": 1, "x_i": 1, "hat": 1, "y": [1, 10], "_2": 1, "result": [1, 2, 5, 6, 10], "choic": 1, "argmax": 1, "product": [1, 4], "cdot": 1, "grad": [1, 2], "direct": [1, 3], "influenc": [1, 10], "suggest": [1, 4, 10], "stronger": 1, "wa": [1, 5, 10], "grad_for": 1, "normal": [1, 5, 10], "contribut": 1, "along": [1, 4, 7, 10], "sundararajan": 1, "et": [1, 2], "al": [1, 2], "2017": 1, "algorithm": 1, "instrument": 1, "modifi": [1, 5, 10], "plu": 1, "grad_target_field_kei": 1, "convent": [1, 5], "interpol": 1, "token_emb": 1, "input_text": 1, "token_grad": 1, "concret": 1, "contact": 1, "team": [1, 2], "assist": 1, "drop": [1, 10], "train": [1, 2, 10], "linear": 1, "reconstruct": 1, "origin": [1, 5, 10], "trade": 1, "off": [1, 5, 10], "slow": [1, 4], "noisi": 1, "longer": [1, 10], "ablat": 1, "compens": 1, "increas": 1, "explain": [1, 3, 10], "movi": [1, 10], "terribl": [1, 10], "ask": [1, 10], "receiv": 1, "okai": 1, "present": 1, "long": [1, 3, 4, 10], "deriv": [1, 5], "impact": 1, "prompt": [1, 2], "causal": 1, "granular": [1, 2], "sub": [1, 10], "paragraph": [1, 2], "refin": 1, "densiti": 1, "shot": 1, "eaxmpl": 1, "chain": 1, "thought": [1, 5], "walkthrough": [1, 2], "debug": [1, 2, 7], "llm": [1, 2], "ai": [1, 2], "toolkit": [1, 2, 5], "dev": [1, 2, 7], "model_behavior": [1, 2], "paper": [1, 9], "2404": 1, "07498": 1, "pleas": [1, 5, 9], "cite": [1, 9], "articl": 1, "tenney2024interact": 1, "author": [1, 9], "tennei": [1, 9], "ian": [1, 9], "mullin": 1, "ryan": 1, "du": 1, "pandya": 1, "shree": 1, "kahng": 1, "minsuk": 1, "dixon": 1, "luca": 1, "journal": 1, "preprint": 1, "year": [1, 9], "2024": 1, "gemma": 1, "llama": 1, "mistral": 1, "gpt": 1, "either": [1, 5, 10], "kerasnlp": [1, 2], "lit_gemma": 1, "lm_salience_demo": [1, 2], "instrumented_keras_lm": 1, "pretrained_lm": 1, "ebert": 1, "2022": 1, "k": 1, "bag": 1, "appli": 1, "button": [1, 5, 10], "6": [1, 5], "seek": 1, "column": [1, 10], "tutori": [1, 2], "analyz": [1, 10], "actual": [1, 5, 10], "been": [1, 3, 5, 10], "grad_target": 1, "channel": 1, "2d": [1, 10], "abl": [1, 4], "mobilenet": [1, 4], "blur": 1, "xrai": 1, "imagesali": 1, "layer_0": 1, "align_in": 1, "align_out": 1, "layer_1": 1, "layer_2": 1, "target_token": 1, "version": 1, "latent": [1, 2], "pca": [1, 2, 7, 10], "pan": [1, 10], "zoom": 1, "rotat": [1, 10], "shift": [1, 5, 10], "colormap": 1, "menu": [1, 2, 4, 10], "bleu": [1, 10], "whole": [1, 4], "try": 1, "power": [1, 5], "header": [1, 7, 10], "intersect": 1, "drag": [1, 10], "decis": 1, "boundari": 1, "respond": [1, 4, 5], "calcul": [1, 10], "optim": 1, "cost": 1, "ratio": 1, "rel": 1, "penalti": 1, "equal": 1, "costli": 1, "term": [1, 7, 10], "twice": 1, "overal": [1, 5, 10], "fair": [1, 2], "One": [1, 5], "demograph": [1, 2], "pariti": [1, 2], "attempt": 1, "percentag": 1, "accuraci": [1, 10], "opportun": 1, "among": 1, "effect": [1, 5], "minimum": 1, "maximum": [1, 10], "chart": 1, "averag": 1, "chosen": [1, 5], "done": [1, 5, 10], "across": [1, 5, 10], "inspir": 1, "prior": [1, 10], "toolbar": [1, 5, 9], "megaplot": [1, 4], "100k": [1, 4], "mous": 1, "scroll": 1, "reset": [1, 10], "futur": [1, 10], "releas": 1, "contrast": 1, "high": [1, 9, 10], "concept": 1, "gender": 1, "race": [1, 5], "akin": 1, "cav": 1, "intuit": 1, "measur": [1, 10], "sensit": [1, 10], "mention": [1, 2], "rather": [1, 5], "begin": 1, "low": 1, "15": [1, 4, 9], "selector": 1, "actor": 1, "actress": 1, "next": [1, 10], "final": [1, 5], "newli": [1, 4, 10], "split": 1, "second": [1, 5, 10], "9": [1, 5], "blue": [1, 10], "baselin": 1, "technic": 1, "null": [1, 5, 10], "pitfal": 1, "potenti": 1, "meaningless": 1, "randomli": [1, 10], "meaning": 1, "guard": 1, "whether": [1, 5], "possibli": 1, "remaind": 1, "reject": 1, "insignific": 1, "p": [1, 3], "greater": [1, 5], "05": 1, "happen": 1, "warn": 1, "100": [1, 10], "uniqu": 1, "accommod": 1, "cross": [1, 4], "approach": [1, 5], "too": 1, "min": 1, "upcom": 1, "pictur": 1, "insight": 1, "answer": [1, 10], "my": 1, "behav": 1, "under": [1, 3, 6, 10], "delet": [1, 10], "systemat": 1, "editor": 1, "enter": [1, 4], "duplic": [1, 5, 7, 10], "keep": [1, 4, 10], "track": [1, 10], "relationship": 1, "cycl": [1, 10], "scrambl": [1, 10], "regex": [1, 10], "hand": [1, 7, 10], "shelf": 1, "scrambler": [1, 10], "ebrahimi": 1, "tri": 1, "special": 1, "get_embedding_t": 1, "flip": 1, "maintain": [2, 7], "host": [2, 3, 4, 7, 9], "launcher": 2, "type": [2, 5, 6, 9, 10], "publicli": 2, "visibl": [2, 5], "exampl": [2, 3, 5, 7, 9, 10], "analysi": [2, 9], "sst": 2, "input": [2, 4, 5, 9, 10], "huggingfac": 2, "widest": 2, "output": [2, 4, 5, 6, 9], "metric": [2, 6, 7], "plot": [2, 10], "confid": 2, "ones": 2, "cluster": [2, 9, 10], "gener": [2, 4, 6, 7, 9], "tip": [2, 9], "studi": 2, "public": [2, 3], "websit": 2, "xnli_demo": 2, "14": 2, "jp": 2, "fine": 2, "tune": 2, "whitespac": 2, "delimit": 2, "wordpiec": 2, "stsb_dev": 2, "unrel": 2, "wide": 2, "scatter": [2, 10], "filter": [2, 10], "2b": 2, "7b": 2, "dot": 2, "depth": 2, "t5_demo": [2, 3], "hypothes": 2, "veer": 2, "roug": [2, 10], "machin": [2, 3, 4, 7], "cnndm": 2, "wmt": [2, 4], "corpora": 2, "coref_demo": [2, 3], "ontonot": [2, 10], "winogend": [2, 10], "schema": 2, "ruding": 2, "2018": 2, "profess": 2, "stratifi": 2, "quantifi": 2, "pronoun": [2, 10], "bureau": 2, "labor": 2, "statist": 2, "partial": 2, "setter": 2, "imagenet": 2, "deploi": 3, "onto": 3, "face": 3, "container": 3, "prefer": [3, 5], "engin": [3, 5], "dockerfil": 3, "directori": [3, 5, 6], "front": 3, "gunicorn": 3, "invok": [3, 5], "get_wsgi_app": 3, "wsgi": [3, 4, 7], "gunicorn_config": 3, "shell": 3, "command": [3, 6], "rememb": 3, "trail": 3, "know": [3, 10], "now": 3, "rm": 3, "5432": [3, 5], "launch": [3, 10], "demo_nam": 3, "demo_port": 3, "children": 3, "nest": 3, "coref": 3, "is_ev": 3, "is_eval_demo": 3, "Be": 3, "sure": [3, 4, 5], "2345": 3, "bring": 3, "togeth": [3, 10], "combin": 3, "background": [3, 10], "execut": 3, "meet": 3, "locat": 3, "litservertyp": 3, "server_typ": 3, "extern": [3, 4], "unus": 3, "sy": 3, "argv": 3, "known_onli": 3, "rebuild": 3, "your_server_script_path_her": 3, "consum": 3, "absl": 3, "stricli": 3, "recommend": [3, 4, 5, 6], "consol": 3, "suppos": 3, "come": 3, "soon": 3, "submit": 4, "bug": 4, "issu": [4, 5], "modern": 4, "ml": 4, "good": [4, 10], "tabular": [4, 9], "palmer": 4, "unicod": 4, "english": 4, "latin": [4, 5], "modif": 4, "xnli": 4, "lingual": 4, "nli": 4, "multilingu": 4, "comfort": 4, "10k": 4, "speed": 4, "caveat": [4, 5], "lot": 4, "larger": [4, 5], "warm_start": [4, 5, 6], "nativ": 4, "resolut": 4, "smaller": [4, 6], "anywai": 4, "256x256": 4, "thing": [4, 5, 7], "webgl": 4, "scattergl": 4, "dive": [4, 10], "older": [4, 10], "thousand": 4, "preload": 4, "wait": [4, 5, 10], "visit": 4, "bewar": 4, "node": [4, 5], "favorit": 4, "lightweight": 4, "alreadi": [4, 10], "ephemer": 4, "window": [4, 7], "manual": [4, 5, 10], "data_dir": [4, 5], "disk": [4, 5], "abil": 4, "edit": [4, 10], "write": [4, 6], "anyon": 4, "address": [4, 10], "restrict": 4, "configur": [4, 5, 6, 9, 10], "thin": [4, 7], "middlewar": 4, "docker": [4, 9], "data_": 4, "fieldnam": 4, "total": 4, "within": [4, 10], "data0": 4, "data1": 4, "data2": 4, "data0_": 4, "csv": [4, 10], "push": 4, "writabl": 4, "notebook": [4, 9], "ui_stat": 4, "primari": [4, 10], "bulk": 4, "pipelin": [4, 7], "littl": 4, "difficult": 4, "refactor": 4, "primarili": 4, "offici": 4, "facilit": 4, "reus": 4, "fact": 4, "exactli": 4, "quickstart_sst_demo": 4, "gotcha": 5, "central": 5, "piec": [5, 10], "tech": 5, "compon": [5, 7, 9, 10], "observ": 5, "orient": 5, "templat": 5, "declar": 5, "isol": 5, "center": 5, "data": [5, 6, 9], "scalabl": 5, "highli": 5, "fairli": 5, "digest": 5, "heavier": 5, "angular": 5, "roughli": [5, 7], "divid": [5, 10], "itself": 5, "coordin": 5, "footer": 5, "lai": 5, "variou": 5, "go": 5, "later": 5, "bundl": 5, "attach": 5, "dom": 5, "kick": 5, "singleton": [5, 7], "arraign": 5, "minimalist": [5, 10], "bottom": [5, 10], "analyt": [5, 10], "put": [5, 10], "exploratori": [5, 10], "inspect": [5, 10], "v1": [5, 10], "appreci": [5, 10], "custom_layout": 5, "updat": [5, 10], "factor": 5, "model": [5, 7, 9], "modules_servic": 5, "fetch": 5, "static": 5, "helper": [5, 7], "outlin": [5, 10], "dummi": 5, "pig": 5, "customel": 5, "demotextmodul": 5, "selectionserviceindex": 5, "shouldreact": 5, "duplicateformodelcomparison": 5, "4": 5, "privat": 5, "readonli": 5, "colorservic": 5, "getservic": 5, "piglatin": 5, "firstupd": 5, "reactimmedi": 5, "selectionservic": [5, 7], "primaryselectedinputdata": 5, "7": [5, 10], "gettransl": 5, "async": 5, "indexedinput": 5, "const": 5, "promis": 5, "apiservic": 5, "getpiglatin": 5, "8": 5, "await": 5, "loadlatest": 5, "renderimpl": 5, "getdatapointcolor": 5, "div": 5, "stylemap": 5, "checkmodul": 5, "modelspec": 5, "modelsmap": 5, "datasetspec": 5, "11": 5, "12": 5, "htmlelementtagnamemap": 5, "illustr": 5, "definit": 5, "css": 5, "shared_styl": 5, "super": 5, "inject": 5, "easier": 5, "mock": 5, "appstat": 5, "u": [5, 10], "decor": 5, "unit": 5, "reactiv": 5, "mobxlitel": 5, "trigger": 5, "excel": 5, "asynchron": 5, "leverag": 5, "machineri": 5, "lifecycl": 5, "reaction": 5, "whatev": [5, 10], "immedi": 5, "whenev": 5, "react": 5, "won": 5, "condit": 5, "rapidli": 5, "supersed": 5, "recent": 5, "rerend": 5, "seem": 5, "pure": 5, "worth": 5, "sometim": [5, 7], "inadequ": 5, "advanc": 5, "suit": 5, "anim": 5, "imper": 5, "draw": 5, "canva": 5, "fortun": 5, "bridg": 5, "queryselector": 5, "shadow": 5, "root": 5, "mess": 5, "reconcili": 5, "shadowroot": 5, "drawcanva": 5, "faceting_control": 5, "construct": 5, "programmat": 5, "member": 5, "oppos": [5, 10], "destroi": 5, "recreat": 5, "brought": 5, "lose": 5, "previous": [5, 10], "held": 5, "snippet": 5, "examplemodul": 5, "facetingcontrol": 5, "createel": 5, "facetschang": 5, "event": 5, "customev": 5, "contextnam": 5, "addeventlisten": 5, "eventlisten": 5, "clang": 5, "instead": [5, 10], "id": [5, 10], "dosometh": 5, "litel": 5, "reactiveel": 5, "annotated_text_modul": 5, "annotated_text_vi": 5, "On": [5, 10], "regist": 5, "properli": 5, "clean": 5, "leav": 5, "yarn": 5, "termin": 5, "cd": 5, "watch": 5, "m": [5, 6], "example_nam": 5, "localhost": [5, 6], "ctrl": 5, "cmd": 5, "r": 5, "hard": 5, "refresh": 5, "pick": 5, "hot": 5, "subsequ": 5, "conjunct": 5, "stale": 5, "effort": 5, "potato": [5, 7], "theme": 5, "custom_modul": 5, "env": 5, "asset": 5, "client_root": 5, "potato_demo": 5, "parent_dir": 5, "o": 5, "join": 5, "pathlib": 5, "__file__": 5, "absolut": 5, "enum": 5, "litmodulenam": 5, "potato_layout": 5, "classificationmodul": 5, "spud": 5, "tastic": 5, "jump": [6, 9], "short": 6, "video": [6, 10], "rout": 6, "foo_data": 6, "foodataset": 6, "bar_data": 6, "bardataset": 6, "my_model": 6, "mymodel": 6, "wealth": 6, "overview": [6, 9], "lit_sentiment_classifi": 6, "403": 6, "cooki": 6, "site": [6, 9], "outsid": [6, 9], "blaze": 6, "quickstart": 6, "alsologtostderr": 6, "cpu": 6, "overload": 7, "codebas": 7, "folder": 7, "fledg": 7, "job": 7, "fulli": 7, "capit": 7, "accompani": 7, "emoji": 7, "pronounc": 7, "ell": 7, "ey": 7, "tee": 7, "formerli": 7, "polym": 7, "coincident": 7, "cours": 7, "neural": 7, "network": 7, "composit": 7, "strictli": 7, "speak": 7, "litmodul": [7, 9], "noun": 7, "verb": 7, "notabl": 7, "layout": [7, 9], "maxim": 7, "gui": 7, "welcom": 9, "demonstr": 9, "misc": 9, "tenney2020languag": 9, "jame": 9, "wexler": 9, "jasmijn": 9, "bast": 9, "tolga": 9, "bolukbasi": 9, "andi": 9, "coenen": 9, "sebastian": 9, "gehrmann": 9, "ellen": 9, "jiang": 9, "mahima": 9, "pushkarna": 9, "carei": 9, "radebaugh": 9, "emili": 9, "reif": 9, "ann": 9, "yuan": 9, "booktitl": 9, "proceed": 9, "2020": 9, "confer": 9, "empir": 9, "publish": 9, "linguist": 9, "107": 9, "118": 9, "www": 9, "aclweb": 9, "anthologi": 9, "emnlp": 9, "stand": 9, "alon": 9, "multimod": 9, "journei": 9, "tcav": 9, "architectur": 9, "style": 9, "glossari": 9, "privaci": 9, "secur": 9, "tour": 10, "live": 10, "draggabl": 10, "alloc": 10, "vertic": 10, "horizont": 10, "adopt": 10, "organiz": 10, "scheme": 10, "reflect": 10, "focu": 10, "arrow": 10, "darker": 10, "dialog": 10, "declutt": 10, "someon": 10, "els": 10, "clear": 10, "deselect": 10, "legend": 10, "unpin": 10, "indetermin": 10, "progress": 10, "pend": 10, "fail": 10, "failur": 10, "until": 10, "x": 10, "icon": 10, "messag": 10, "star": 10, "screenshot": 10, "toggl": 10, "dimens": 10, "hover": 10, "lasso": 10, "unselect": 10, "consecut": 10, "light": 10, "sort": 10, "were": 10, "prefix": 10, "hide": 10, "exce": 10, "truncat": 10, "ellipsi": 10, "entireti": 10, "collaps": 10, "film": 10, "again": 10, "unstar": 10, "neighbor": 10, "immut": 10, "simplic": 10, "blank": 10, "creation": 10, "altogeth": 10, "highest": 10, "opposit": 10, "hello": 10, "bad": 10, "exact": 10, "broken": 10, "made": 10, "versu": 10, "ax": 10, "agreement": 10, "disagr": 10, "clickabl": 10, "disagre": 10, "laid": 10, "axi": 10, "methodologi": 10, "capabl": 10, "wise": 10, "autorun": 10, "uncheck": 10, "valuabl": 10, "expens": 10, "attend": 10, "opac": 10, "magnitud": 10, "negat": 10, "stanford": 10, "treebank": 10, "56": 10, "surprisingli": 10, "But": 10, "truli": 10, "review": 10, "ultim": 10, "depress": 10, "era": 10, "gangster": 10, "strongli": 10, "worst": 10, "elicit": 10, "mildli": 10, "incorrect": 10, "occup": 10, "technician": 10, "particip": 10, "male": 10, "domin": 10, "25": 10, "femal": 10, "bl": 10, "agre": 10, "stereotyp": 10, "resolv": 10, "83": 10, "37": 10, "cnn": 10, "dm": 10, "middl": 10, "erron": 10, "constitu": 10, "alastair": 10, "cook": 10, "captain": 10, "dig": 10, "deeper": 10, "28": 10, "arriv": 10, "searcher": 10, "fast": 10, "approxim": 10, "nearest": 10, "corpu": 10, "retriev": 10, "34": 10, "16": 10, "occurr": 10, "strong": 10, "toward": 10, "phrase": 10}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"lit": [0, 3, 4, 6, 9], "python": 0, "api": 0, "design": 0, "overview": [0, 5], "ad": 0, "model": [0, 1, 2, 4, 6, 10], "data": [0, 1, 2, 4, 10], "valid": 0, "dataset": [0, 4], "transform": [0, 1], "more": 0, "output": [0, 1, 10], "option": [0, 10], "input": [0, 1], "interpret": [0, 9], "compon": [0, 1, 4, 6], "metric": [0, 1, 10], "gener": [0, 1, 10], "backtransl": 0, "configur": 0, "ui": [0, 4, 10], "type": [0, 1, 4], "system": 0, "convent": 0, "compat": 0, "check": 0, "an": [0, 6], "In": 0, "depth": 0, "exampl": [0, 1, 4, 6], "avail": 0, "server": 0, "custom": [0, 3, 5, 6], "layout": [0, 5, 10], "access": 0, "notebook": [0, 6], "us": [0, 4, 6], "outsid": [0, 4], "featur": 1, "framework": 1, "support": 1, "huggingfac": 1, "tf1": 1, "x": 1, "estim": 1, "remot": 1, "host": [1, 6], "static": 1, "predict": [1, 2], "load": [1, 4], "classif": [1, 2], "regress": [1, 2], "score": [1, 2], "multi": 1, "label": 1, "seq2seq": [1, 2], "span": 1, "structur": [1, 2], "multipl": 1, "segment": 1, "tabular": [1, 2], "imag": [1, 2, 3], "token": 1, "base": 1, "salienc": [1, 10], "gradient": 1, "norm": 1, "dot": 1, "integr": [1, 3, 4], "lime": 1, "target": 1, "select": [1, 10], "sequenc": [1, 2], "cluster": 1, "attribut": 1, "pixel": 1, "attent": [1, 10], "embed": [1, 10], "projector": [1, 10], "aggreg": 1, "analysi": [1, 10], "confus": [1, 10], "matrix": [1, 10], "scalar": [1, 10], "plot": 1, "binari": 1, "threshold": 1, "partial": 1, "depend": 1, "dive": 1, "tcav": 1, "statist": 1, "signific": 1, "sort": 1, "cosin": 1, "similar": [1, 2], "counterfactu": 1, "manual": 1, "edit": 1, "demo": [2, 6], "do": [2, 6], "NOT": [2, 6], "remov": [2, 6], "section": 2, "header": 2, "sentiment": [2, 10], "nli": 2, "glue": 2, "multilingu": 2, "xnli": 2, "textual": 2, "st": 2, "b": 2, "stsb": 2, "gemma": 2, "t5": 2, "languag": [2, 4], "lm": 2, "bert": 2, "gpt": 2, "2": 2, "gender": [2, 10], "bia": [2, 10], "corefer": [2, 10], "coref": 2, "multimod": 2, "penguin": 2, "mobilenet": 2, "run": [3, 6], "docker": 3, "contain": 3, "basic": 3, "usag": 3, "instanc": 3, "default": 3, "build": 3, "your": [3, 6], "own": 3, "frequent": 4, "ask": 4, "question": 4, "scale": 4, "size": 4, "larg": 4, "privaci": 4, "secur": 4, "i": 4, "have": 4, "proprietari": 4, "my": 4, "team": 4, "workflow": 4, "send": 4, "from": 4, "anoth": 4, "tool": [4, 9], "download": 4, "export": 4, "train": 4, "frontend": 5, "develop": 5, "guid": [5, 10], "high": 5, "level": 5, "applic": 5, "architectur": 5, "bootstrap": 5, "initi": 5, "modul": [5, 10], "litmodul": 5, "setup": 5, "function": 5, "escap": 5, "hatch": 5, "state": 5, "child": 5, "element": 5, "style": 5, "tip": 5, "open": 5, "sourc": 5, "client": 5, "get": 6, "start": 6, "instal": 6, "colab": 6, "stand": 6, "alon": 6, "standalon": 6, "exist": 6, "glossari": 7, "learn": 9, "research": 9, "datapoint": 10, "toolbar": 10, "top": 10, "bar": 10, "global": 10, "set": 10, "url": 10, "share": 10, "main": 10, "statu": 10, "compar": 10, "slice": 10, "detail": 10, "tabl": 10, "editor": 10, "map": 10, "user": 10, "journei": 10, "debug": 10, "text": 10}, "envversion": {"sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx": 60}, "alltitles": {"LIT Python API": [[0, "lit-python-api"]], "Design Overview": [[0, "design-overview"]], "Adding Models and Data": [[0, "adding-models-and-data"]], "Validating Models and Data": [[0, "validating-models-and-data"]], "Datasets": [[0, "datasets"]], "Transformations": [[0, "transformations"]], "Models": [[0, "models"]], "Adding more outputs": [[0, "adding-more-outputs"]], "Optional inputs": [[0, "optional-inputs"]], "Interpretation Components": [[0, "interpretation-components"]], "Metrics": [[0, "metrics"], [1, "metrics"]], "Generators": [[0, "generators"], [1, "generators"]], "Backtranslator Generator": [[0, "backtranslator-generator"]], "Configuration UI": [[0, "configuration-ui"]], "Type System": [[0, "type-system"]], "Conventions": [[0, "conventions"]], "Compatibility Checks": [[0, "compatibility-checks"]], "An In-Depth Example": [[0, "an-in-depth-example"]], "Available types": [[0, "available-types"]], "Server Configuration": [[0, "server-configuration"]], "Customizing the Layout": [[0, "customizing-the-layout"]], "Accessing the LIT UI in Notebooks": [[0, "accessing-the-lit-ui-in-notebooks"]], "Using LIT components outside of LIT": [[0, "using-lit-components-outside-of-lit"]], "Components and Features": [[1, "components-and-features"]], "Framework and Model Support": [[1, "framework-and-model-support"]], "HuggingFace Transformers": [[1, "huggingface-transformers"]], "TF1.x Estimator": [[1, "tf1-x-estimator"]], "Remote or hosted models": [[1, "remote-or-hosted-models"]], "Static predictions": [[1, "static-predictions"]], "Data loading": [[1, "data-loading"]], "Input and Output Types": [[1, "input-and-output-types"]], "Classification": [[1, "classification"]], "Regression / Scoring": [[1, "regression-scoring"]], "Multi-label classification": [[1, "multi-label-classification"]], "Seq2Seq / Generation": [[1, "seq2seq-generation"]], "Span Labeling and Structured Prediction": [[1, "span-labeling-and-structured-prediction"]], "Multiple input segments": [[1, "multiple-input-segments"]], "Tabular data": [[1, "tabular-data"]], "Images": [[1, "images"]], "Token-based Salience": [[1, "token-based-salience"]], "Gradient Norm": [[1, "gradient-norm"]], "Gradient-dot-Input": [[1, "gradient-dot-input"]], "Integrated Gradients": [[1, "integrated-gradients"]], "LIME": [[1, "lime"]], "Target Selection on Classification Output": [[1, "target-selection-on-classification-output"]], "Sequence Salience": [[1, "sequence-salience"]], "Salience Clustering": [[1, "salience-clustering"]], "Tabular Feature Attribution": [[1, "tabular-feature-attribution"]], "Pixel-based Salience": [[1, "pixel-based-salience"]], "Attention": [[1, "attention"], [10, "attention"]], "Embedding Projector": [[1, "embedding-projector"], [10, "embedding-projector"]], "Aggregate Analysis": [[1, "aggregate-analysis"]], "Confusion Matrix": [[1, "confusion-matrix"], [10, "confusion-matrix"]], "Scalar Plots": [[1, "scalar-plots"]], "Binary Classification Thresholds": [[1, "binary-classification-thresholds"]], "Partial Dependence Plots": [[1, "partial-dependence-plots"]], "Dive": [[1, "dive"]], "TCAV": [[1, "tcav"]], "Example": [[1, "example"]], "Statistical Significance": [[1, "statistical-significance"]], "Sorting by Cosine Similarity": [[1, "sorting-by-cosine-similarity"]], "Counterfactual Analysis": [[1, "counterfactual-analysis"]], "Manual Editing": [[1, "manual-editing"]], "Demos": [[2, "demos"]], "Classification <!-- DO NOT REMOVE {#classification .demo-section-header} -->": [[2, "classification"]], "Sentiment and NLI <!-- DO NOT REMOVE {#glue .demo-header} -->": [[2, "sentiment-and-nli"]], "Multilingual (XNLI) <!-- DO NOT REMOVE {#xnli .demo-header} -->": [[2, "multilingual-xnli"]], "Regression / Scoring <!-- DO NOT REMOVE {#regression-scoring .demo-section-header} -->": [[2, "regression-scoring"]], "Textual Similarity (STS-B) <!-- DO NOT REMOVE {#stsb .demo-header} -->": [[2, "textual-similarity-sts-b"]], "Sequence-to-Sequence <!-- DO NOT REMOVE {#seq2seq .demo-section-header} -->": [[2, "sequence-to-sequence"]], "Gemma <!-- DO NOT REMOVE {#gemma .demo-header} -->": [[2, "gemma"]], "T5 <!-- DO NOT REMOVE {#t5 .demo-header} -->": [[2, "t5"]], "Language Modeling <!-- DO NOT REMOVE {#lm .demo-section-header} -->": [[2, "language-modeling"]], "BERT and GPT-2 <!-- DO NOT REMOVE {#bert .demo-header} -->": [[2, "bert-and-gpt-2"]], "Structured Prediction <!-- DO NOT REMOVE {#structured .demo-section-header} -->": [[2, "structured-prediction"]], "Gender Bias in Coreference <!-- DO NOT REMOVE {#coref .demo-header} -->": [[2, "gender-bias-in-coreference"]], "Multimodal <!-- DO NOT REMOVE {#multimodal .demo-section-header} -->": [[2, "multimodal"]], "Tabular Data: Penguin Classification <!-- DO NOT REMOVE {#penguin .demo-header} -->": [[2, "tabular-data-penguin-classification"]], "Image Classification with MobileNet <!-- DO NOT REMOVE {#mobilenet .demo-header} -->": [[2, "image-classification-with-mobilenet"]], "Running LIT in a Docker container": [[3, "running-lit-in-a-docker-container"]], "Basic Usage": [[3, "basic-usage"]], "Integrating Custom LIT Instances with the Default Docker Image": [[3, "integrating-custom-lit-instances-with-the-default-docker-image"]], "Building Your Own Image": [[3, "building-your-own-image"]], "Frequently Asked Questions": [[4, "frequently-asked-questions"]], "Model and Data Types": [[4, "model-and-data-types"]], "Languages": [[4, "languages"]], "Scale": [[4, "scale"]], "Dataset Size": [[4, "dataset-size"]], "Large Models": [[4, "large-models"]], "Privacy and Security": [[4, "privacy-and-security"]], "I have proprietary data. Is LIT secure for my team to use?": [[4, "i-have-proprietary-data-is-lit-secure-for-my-team-to-use"]], "Workflow and Integrations": [[4, "workflow-and-integrations"]], "Sending examples from another tool": [[4, "sending-examples-from-another-tool"]], "Downloading or exporting data": [[4, "downloading-or-exporting-data"]], "Loading data from the UI": [[4, "loading-data-from-the-ui"]], "Using components outside the LIT UI": [[4, "using-components-outside-the-lit-ui"]], "Training models with LIT": [[4, "training-models-with-lit"]], "Frontend Developer Guide": [[5, "frontend-developer-guide"]], "High Level Overview": [[5, "high-level-overview"]], "Application Architecture": [[5, "application-architecture"]], "Bootstrapping": [[5, "bootstrapping"]], "Layout": [[5, "layout"]], "Initialization": [[5, "initialization"]], "Modules (LitModule)": [[5, "modules-litmodule"]], "Setup": [[5, "setup"]], "Functionality": [[5, "functionality"]], "Escape Hatches": [[5, "escape-hatches"]], "Stateful Child Elements": [[5, "stateful-child-elements"]], "Style Guide": [[5, "style-guide"]], "Development Tips (open-source)": [[5, "development-tips-open-source"]], "Custom Client / Modules": [[5, "custom-client-modules"]], "Getting Started with LIT": [[6, "getting-started-with-lit"]], "Installation": [[6, "installation"]], "Hosted demos": [[6, "hosted-demos"]], "LIT with your model <!-- DO NOT REMOVE {#custom-demos} -->": [[6, "lit-with-your-model"]], "Using LIT in notebooks <!-- DO NOT REMOVE {#colab} -->": [[6, "using-lit-in-notebooks"]], "Stand-alone components <!-- DO NOT REMOVE {#standalone} -->": [[6, "stand-alone-components"]], "Run an existing example <!-- DO NOT REMOVE {#running-lit} -->": [[6, "run-an-existing-example"]], "Glossary": [[7, "glossary"]], "Learning Interpretability Tool (LIT)": [[9, "learning-interpretability-tool-lit"]], "Research": [[9, "research"]], "UI Guide": [[10, "ui-guide"]], "General Layout": [[10, "general-layout"]], "Layout Options": [[10, "layout-options"]], "Datapoint Selections": [[10, "datapoint-selections"]], "Toolbars": [[10, "toolbars"]], "Top Bar": [[10, "top-bar"]], "Global Settings": [[10, "global-settings"]], "URL Sharing": [[10, "url-sharing"]], "Main Toolbar": [[10, "main-toolbar"]], "Status Bar": [[10, "status-bar"]], "Comparing Models": [[10, "comparing-models"]], "Comparing Datapoints": [[10, "comparing-datapoints"]], "Slices": [[10, "slices"]], "Module Details": [[10, "module-details"]], "Data Table": [[10, "data-table"]], "Datapoint Editor": [[10, "datapoint-editor"]], "Datapoint Generator": [[10, "datapoint-generator"]], "Metrics Table": [[10, "metrics-table"]], "Scalars": [[10, "scalars"]], "Model Output": [[10, "model-output"]], "Salience Maps": [[10, "salience-maps"]], "User Journeys": [[10, "user-journeys"]], "Sentiment Analysis": [[10, "sentiment-analysis"]], "Gender Bias in Coreference": [[10, "gender-bias-in-coreference"]], "Debugging Text Generation": [[10, "debugging-text-generation"]]}, "indexentries": {}})
\ No newline at end of file
diff --git a/docs/tutorials/sequence-salience/index.html b/docs/tutorials/sequence-salience/index.html
index 5c0c6cfc..9e76d614 100644
--- a/docs/tutorials/sequence-salience/index.html
+++ b/docs/tutorials/sequence-salience/index.html
@@ -112,10 +112,11 @@ <h2>Prompt Engineering with Sequence Salience</h2>
 LIT supports additional LLMs, including <a href="https://llama.meta.com/">Llama 2</a> and <a href="https://mistral.ai/news/announcing-mistral-7b/">Mistral</a>,
 via the HuggingFace Transformers and KerasNLP libraries.</p>
 <p>This tutorial was adapted from and expands upon LIT's contributions to the
-<a href="https://ai.google.dev/responsible">Responsible Generative AI Tookit</a> and the related paper and
-<a href="https://youtu.be/EZgUlnWdh0w">video</a> submitted to the ACL 2024 Systems Demonstration track.
-This is an active and ongoing research area for the LIT team, so expect changes
-and further expansions to this tutorial over time.</p>
+<a href="https://ai.google.dev/responsible">Responsible Generative AI Tookit</a> and the related
+<a href="https://arxiv.org/abs/2404.07498">paper</a> and <a href="https://youtu.be/EZgUlnWdh0w">video</a> submitted to the ACL 2024
+System Demonstrations track. This is an active and ongoing research area for
+the LIT team, so expect changes and further expansions to this tutorial over
+time.</p>
 <h2>Case Study 1: Debugging Few-Shot Prompts</h2>
 <p>Few-shot prompting was introduced with <a href="https://cdn.openai.com/better-language-models/language-models.pdf">GPT-2</a>: an ML developer provides
 examples of how to perform a task in a prompt, affixes user-provided content at
diff --git a/lit_nlp/examples/models/tfx_model_test.py b/lit_nlp/examples/models/tfx_model_test.py
index 4b44b2d4..e7b977c9 100644
--- a/lit_nlp/examples/models/tfx_model_test.py
+++ b/lit_nlp/examples/models/tfx_model_test.py
@@ -12,7 +12,8 @@ def setUp(self):
     super(TfxModelTest, self).setUp()
     self._path = tempfile.mkdtemp()
     input_layer = tf.keras.layers.Input(
-        shape=(1), dtype=tf.string, name='example')
+        shape=(1,), dtype=tf.string, name='example'
+    )
     parsed_input = tf.io.parse_example(
         tf.reshape(input_layer, [-1]),
         {'input_0': tf.io.FixedLenFeature([1], dtype=tf.float32)})
diff --git a/pyproject.toml b/pyproject.toml
index fb3dea3f..5bed29b2 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -79,10 +79,11 @@ keywords = [
 [project.optional-dependencies]
 # LINT.IfChange
 examples = [
-  "gunicorn==20.1.0",
-  "tensorflow==2.10.0",
-  "tensorflow-datasets==4.8.0",
-  "tensorflow-text==2.10.0",
+  "gunicorn>=20.1.0",
+  "sentencepiece==0.1.99",
+  "tensorflow>=2.10.0,<2.16.0",
+  "tensorflow-datasets>=4.9.0",
+  "tensorflow-text>=2.10.0,<2.16.0",
   "torch>=2.0.0",
   "transformers>=4.27.1",
 ]
diff --git a/requirements_examples.txt b/requirements_examples.txt
index 7778c6d7..3015bfcb 100644
--- a/requirements_examples.txt
+++ b/requirements_examples.txt
@@ -13,11 +13,11 @@
 # limitations under the License.
 # ==============================================================================
 # LINT.IfChange
-gunicorn==20.1.0
+gunicorn>=20.1.0
 sentencepiece==0.1.99
-tensorflow==2.10.0
-tensorflow-datasets==4.8.0
-tensorflow-text==2.10.0
+tensorflow>=2.10.0,<2.16.0
+tensorflow-datasets>=4.9.0
+tensorflow-text>=2.10.0,<2.16.0
 torch>=2.0.0
 transformers>=4.27.1
 # LINT.ThenChange(./pyproject.toml)
diff --git a/website/sphinx_src/components.md b/website/sphinx_src/components.md
index 7cf577f6..f2366d7c 100644
--- a/website/sphinx_src/components.md
+++ b/website/sphinx_src/components.md
@@ -460,11 +460,27 @@ The UI supports multiple options for analysis, including:
 
 For a walkthrough of how to use sequence salience to debug LLMs, check out the
 Responsible Generative AI Toolkit at
-https://ai.google.dev/responsible/model_behavior.
+https://ai.google.dev/responsible/model_behavior and for more on design of the
+system see our paper at https://arxiv.org/abs/2404.07498.
+
+If you find this useful in your work, please cite Sequence Salience as:
+
+```
+@article{tenney2024interactive,
+  title={Interactive Prompt Debugging with Sequence Salience},
+  author={Tenney, Ian and Mullins, Ryan and Du, Bin and Pandya, Shree and Kahng, Minsuk and Dixon, Lucas},
+  journal={arXiv preprint arXiv:2404.07498},
+  year={2024}
+}
+```
 
 **Code:**
 
-*   LIT-for-Gemma Colab: [`lit_gemma.ipynb`](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb)
+Currently, this works out-of-the-box with Gemma, Llama 2, Mistral, and GPT-2,
+using either KerasNLP or Transformers.
+
+*   LIT-for-Gemma Colab:
+    [`lit_gemma.ipynb`](https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/gemma/docs/lit_gemma.ipynb)
 *   Demo binary:
     [`lm_salience_demo.py`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/lm_salience_demo.py)
 *   KerasNLP model wrappers:
@@ -472,8 +488,6 @@ https://ai.google.dev/responsible/model_behavior.
 *   Transformers model wrappers:
     [`pretrained_lms.py`](https://github.com/PAIR-code/lit/blob/main/lit_nlp/examples/models/pretrained_lms.py)
 
-Currently, this works out-of-the-box
-with Gemma models (using Keras) as well as with GPT-2.
 
 
 ## Salience Clustering
diff --git a/website/src/tutorials/sequence-salience.md b/website/src/tutorials/sequence-salience.md
index 51fca21e..bb045c1e 100644
--- a/website/src/tutorials/sequence-salience.md
+++ b/website/src/tutorials/sequence-salience.md
@@ -55,10 +55,11 @@ LIT supports additional LLMs, including [Llama 2][llama] and [Mistral][mistral],
 via the HuggingFace Transformers and KerasNLP libraries.
 
 This tutorial was adapted from and expands upon LIT's contributions to the
-[Responsible Generative AI Tookit][rai_toolkit] and the related paper and
-[video][seqsal_video] submitted to the ACL 2024 Systems Demonstration track.
-This is an active and ongoing research area for the LIT team, so expect changes
-and further expansions to this tutorial over time.
+[Responsible Generative AI Tookit][rai_toolkit] and the related
+[paper][seqsal_paper] and [video][seqsal_video] submitted to the ACL 2024
+System Demonstrations track. This is an active and ongoing research area for
+the LIT team, so expect changes and further expansions to this tutorial over
+time.
 
 ## Case Study 1: Debugging Few-Shot Prompts
 
@@ -486,6 +487,7 @@ helpful guides that can help you develop better prompts, including:
 [salience_research_1]: https://dl.acm.org/doi/full/10.1145/3639372
 [salience_research_2]: https://arxiv.org/abs/2402.01761
 [seqsal_docs]: ../../documentation/components.html#sequence-salience
+[seqsal_paper]: https://arxiv.org/abs/2404.07498
 [seqsal_video]: https://youtu.be/EZgUlnWdh0w
 [synapis]: https://scholarspace.manoa.hawaii.edu/items/65312e48-5954-4a5f-a1e8-e5119e6abc0a
 [toolformer]: https://arxiv.org/abs/2302.04761