Skip to content

Instantly share code, notes, and snippets.

@erichgess
Last active January 4, 2016 15:39
Show Gist options
  • Save erichgess/8642680 to your computer and use it in GitHub Desktop.
Save erichgess/8642680 to your computer and use it in GitHub Desktop.
;; this will map the regular expression used to match a token type to a code for that token type.
;; For example, the regex for an integer token is "^\d+" and this will map that pattern to 0.
;; Unfortunately, I can't seem to get this to work with matching the regex pattern type with the string keys of the map.
(def TokenTypes { "^\\d+" 0
"^[a-zA-Z]+" 1
"^\\+" 2
"^\\*" 3
"^\\^" 4})
;; This will take a token and a regular expression and return a vecor with the code for the token type
;; and the token itself
;; NOTE: this is pretty weak code, because it relies on the caller of the function knowing to match the
;; token with the proper regex.
(defn ClassifyToken [re token]
[(TokenTypes re), token])
;; This is some ugly code which will get the next token from a sequence of tokens.
;; It will classify that token and return a vector with the classified token in the
;; first element and the rest of the token sequence in the second element.
( defn GetNextToken [tokenSequence]
(cond
(re-find #"^\d+" (first tokenSequence) ) [[0, (first tokenSequence)], (rest tokenSequence)]
(re-find #"^[a-zA-Z]+" (first tokenSequence) ) [[1, (first tokenSequence)], (rest tokenSequence)]
(re-find #"^\+" (first tokenSequence) ) [[2, (first tokenSequence)], (rest tokenSequence)]
(re-find #"^\*" (first tokenSequence) ) [[3, (first tokenSequence)], (rest tokenSequence)]
(re-find #"^\^" (first tokenSequence) ) [[4, (first tokenSequence)], (rest tokenSequence)]
:else ()))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment