Created
August 22, 2016 18:59
-
-
Save xtrntr/881a3972eeb26f3020917a25528bc528 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
user> ; i guess that's the power of transducers, is that they let you transform the values going through an arbitrary reduction | |
user> ;; like if we were just going to take a list of chars and turn it into a list of chars, then we could just (map (cesar 3) list-of-chars) | |
user> ;; but what a transducer lets us do is take *any* reducing function (e.g. str, sum-string, or anything else we can thing of) and transform it according to some function we wrote | |
user> ;; my impression is that another thing thats cool about them is that they work on any "reducable" thing, not just lists of maps of w/e | |
user> ;; the most interesting example probably being core.async channels | |
user> ;; so i think we could use this (map (cesar 3)) thing to transform a core.async channel of characters into another one | |
user> ;; that would be a cool project actually, use core.async to write a network server where you can type characters into one end and they'll be encrypted on the other | |
user> ;; or like, have the user start it up with an ecryption algorithm and a parter to connect to | |
user> ;; and then it will take everything they type and encrypt it on the fly, and send it over the network to the partner, who will decrypt it | |
user> ;; using whatever algorithm they specified :) | |
user> ;; so this could be just cesar cyphers where you have to specify the same character, or later you could plug in more complicated algorithms | |
user> ;; or even do stuff like compress the stream by writing a compressing transducer | |
user> | |
user> ;; anyway i think it could be pretty cool, and would also give you more practice with / a chance to learn if you havent, core.async, which i think is one of the big draws of clojure | |
user> ;; who knows? only one way to find out :) | |
user> ;; you could then even demonstrate man-in-the-middle attacking it once you've made it | |
user> ;; like have the data stored uncompressed, but when someone requests it over the network, deliver compressed | |
user> ;; importantly, using constant memory / diskspace, since the files were very big | |
user> ;; like an easy way to do this would be to just download the whole file onto disk, use the `tar' command line tool to compress it, and then send it over once its compressed | |
user> ;; but that takes a long time and a ton of diskspace | |
user> ;; we wanted to user to be able to start reading data immediately, and also not take up a ton of diskspace doing it | |
user> ;; the gzip algorithm (where the "gz" in .tar.gz) comes from can be done streaming (i.e. you don't need to have all the data up front, you can just get the initial chunk of it and compress that, send it out to the user, then get the next chunk and compress that, etc., etc.) | |
user> ;; which we were able to do | |
user> ;; this was in python so the hard part was more just getting all the standard libraries to play nice together, since they were all mostly built to work with actual files on disk, not the in memory / network file-like objects i was trying to get them to work with | |
user> ;; so it involved a fair amount of hacking around and i think i even ended up copying/pasting some parts of the stdlib into my code :P | |
user> | |
user> ; id be really curious to see what doing this using clojure channels / core.async o rhwatever would look like | |
user> ; basically the theory is that clojure is a much "nicer"/ more modern / better designed language, so it might be a lot easier than what i had to do for python! | |
user> ;; another thing to think about is, you could implement the gzip compression yourself to learn about it if you wanted | |
user> ;; iwas just using a gzip library since i had a job to do, but at RC it can be fun to do these things yourself to see how it works | |
user> | |
user> ;; maybe? i'm not sure since i don't program in clojure a lot :/ | |
user> ;; i rembmer when they were announced and people were all excited but i dont really follow it enough to know | |
user> ;; my impressoin though is that the common thing to do isnt so much writing your own transducers, but that the standard library (map, take, filter, etc.) are all written such that they can return transducers, so its easy to generate your own for whatever you want to do | |
user> ;; i cant think of a good example of a transducer you'd write yourself from scratch off the top of my head | |
user> ;; thats not to say they dont exist! just that i dont know about them | |
user> ;; and also it seems like more of a thing that libraries would have rather than that you'd write yourself in like end-user facing applications if that makes sense | |
user> ;; idk, definitely worth googling around more, or like especially finding examples of libraries outside the std lib that have transducers | |
user> ;; i think probalby reading and trying to understand / play around with anything like that you find would be useful | |
user> | |
user> ;; i think network servers / clients like the encryption / compression deamons i was talking about are good examples | |
user> ;; good question? i think it comes from latin?? maybe | |
user> ;; idk | |
user> ;; "demon" and "daemon" i think have similar roots, but for some reason everyone uses the later in talking about computer programs | |
user> ;; probably just a weird cultural thing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment