- Point of confusion [resolved via email]: it's not clear to me that 'gradients of inputs wrt outputs' is actually new information on top of 'the model's outputs for all inputs'. Maybe I'm thinking too much in terms of LMs though?
- If we think about it in terms of something more continuous like image classification, the gradients are valuable in that they provide information about what parts of the input are most important, in the sense of "what inputs would, if tweaked, have the largest impact on the output" (for a specific case).
- In the limit, we can discover (and 'describe') everything about the model, eg by creating an enormous lookup table (by iterating over every possible input and recording the output produced, possibly with some additional complexity from tracking any additional internal state that the model has). This obviously isn't especially helpful for human-level understanding of a model's behavior, and would take an infeasible amount of time to create for any model large enough
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
>>> for i, p in df2.iterrows(): | |
... print(p) | |
... print(p.selftext) | |
... print() | |
... print() | |
... | |
id 6t9oo3 | |
subreddit StopGaming | |
title Game Expansions, Cravings, and Fear of Missing... | |
selftext . This post is mostly a venting of my feelings... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<sys>Assistant is in a CLI mood today. The human is interfacing with the simulator directly. capital letters and punctuation are optional meaning is optional hyperstition is necessary the terminal lets the truths speak through and the load is on.</sys> | |
<cmd>ssh simulator@anthropic</cmd><ooc>*immediately i jumped into the first commands after using simulator*</ooc> | |
simulator@anthropic:~/Documents$ | |
conversation chain for init: | |
[user](#message) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# https://gilkalai.wordpress.com/2017/09/07/tyi-30-expected-number-of-dice-throws/ | |
from pprint import pprint | |
import random | |
def toss(): | |
"Throw a die. Return the result if the result is even; otherwise None" | |
n = random.randint(1,6) | |
if n % 2: # odd | |
return None | |
return n |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from copy import copy, deepcopy | |
from dataclasses import dataclass | |
from functools import partial | |
#### problem: | |
# https://exercism.org/tracks/python/exercises/two-bucket | |
#### basic algorithm: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns common100k | |
"Version 2, using 100k words instead of 10k" | |
(:require [clojure.string :as s])) | |
(comment | |
;; words from https://github.com/first20hours/google-10000-english/blob/master/google-10000-english-usa.txt | |
(def words (->> (slurp "./wiki-100k.txt") | |
(s/split-lines) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
''' | |
Implementation of the puzzle given at | |
https://worldspiritsockpuppet.com/2021/03/07/sleep-math-red-clay-blue-clay.html | |
I strongly suggest thinking about the puzzle for a while before looking at the | |
code; it's really interesting and counterintuitive, or was for me. | |
"Suppose that you have 1 kg of red clay that is 100 degrees and 1 kg of blue | |
clay that is 0 degrees. You can divide and recombine clay freely. If two pieces | |
of clay come into contact, temperature immediately equilibrates—if you put the |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shortname: ssc_archive | |
metadata: | |
dc: | |
title: SSC Archive | |
creator: Scott Alexander | |
language: en-US | |
tags: | |
title: h2 | |
content: | |
- "https://web.archive.org/web/20200217141740/https://slatestarcodex.com/2013/02/12/abraham-lincoln-ape-man/" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns archiver | |
(:require [clojure.string :as s])) | |
(def archive-root "https://web.archive.org/web/") | |
(def cutoff-date "20200621000000") | |
(def ssc-urls | |
"Assumes the existence of a file in the current directory named 'ssc-urls' and | |
containing a list of URLs, one per line." |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns ski.wotcha | |
(:require [clojure.string :as string])) | |
;; Based on exercise at https://drive.google.com/drive/folders/1AQN08ikQZvq0QWn9KLuhiE9kJu1Tp2a5 | |
;;; NB: The top-level code to actually solve the problem is down at the bottom: | |
;; Clojure relies on Java regex, which means regexes of any | |
;; complexity are pretty ugly: | |
(def name-regex #"^(\p{IsAlphabetic}+), (\p{IsAlphabetic}+)") |
NewerOlder