This document is meant to be a soup-to-nuts explanation of the Panoptes->Caesar->MAST pipeline for the TESS project.
When a classification is made in the new PFE Light Curve Viewer, the classification is streamed to Caesar, where processing begins.
There are two primary paths existing in parallel depending on the type of subject being classified:
| # ssh onto the redis node in question and run these cmds | |
| # congestion:* keys are for https://github.com/parrish/Sidekiq-Congestion | |
| # uniquejobs:* https://github.com/mhenrixon/sidekiq-unique-jobs | |
| # get a count of the locks | |
| redis-cli -n 0 --scan --pattern 'congestion:*' | wc -l | |
| redis-cli -n 0 --scan --pattern 'uniquejobs:*' | wc -l | |
| # clear them for both key namespaces |
| # Manual csv classifications dump | |
| # ensure you have a manual secondary database connection specified in config/database.yml | |
| # | |
| # run via rails runner from the panoptes cmd line via | |
| # rails r project_classifications_csv_dump_export.rb | |
| require 'csv' | |
| PROJECT_ID = 1 |
| [ | |
| { | |
| "task": "T1", | |
| "value": [ | |
| { | |
| "task": "T5", | |
| "value": "John Adams" | |
| }, | |
| { | |
| "task": "T7", |
| { | |
| "id": 5919, | |
| "display_name": "Descriptive book", | |
| "tasks": { | |
| "T1": { | |
| "next": "T2", | |
| "type": "combo", | |
| "tasks": [ | |
| "T5", | |
| "T7", |
| SELECT bl.pid AS blocked_pid, | |
| a.usename AS blocked_user, | |
| ka.query AS current_statement_in_blocking_process, | |
| now() - ka.query_start AS blocking_duration, | |
| kl.pid AS blocking_pid, | |
| ka.usename AS blocking_user, | |
| a.query AS blocked_statement, | |
| now() - a.query_start AS blocked_duration | |
| FROM pg_catalog.pg_locks bl | |
| JOIN pg_catalog.pg_stat_activity a ON a.pid = bl.pid |
| [02:23:06 (master) ~/mozilla/phageParser]$ pep8 --statistics -qq ./ | |
| 4 E101 indentation contains mixed spaces and tabs | |
| 2 E111 indentation is not a multiple of four | |
| 1 E124 closing bracket does not match visual indentation | |
| 6 E201 whitespace after '[' | |
| 1 E202 whitespace before ']' | |
| 12 E211 whitespace before '(' | |
| 5 E221 multiple spaces before operator | |
| 8 E222 multiple spaces after operator | |
| 32 E225 missing whitespace around operator |
| SELECT | |
| relid::regclass AS table, | |
| indexrelid::regclass AS index, | |
| pg_size_pretty(pg_relation_size(indexrelid::regclass)) AS index_size, | |
| idx_tup_read, | |
| idx_tup_fetch, | |
| idx_scan | |
| FROM | |
| pg_stat_user_indexes | |
| JOIN pg_index USING (indexrelid) |
| #!/usr/bin/env ruby | |
| if `git diff HEAD spec` =~ /^\+.*,\s?(:focus|focus:\s?true|:focus\s?=>\s?true)/ | |
| puts "\e[31mPlease focus and remove your :focus tags before committing :)" | |
| exit 1 | |
| end |
| Adam: https://open.spotify.com/user/spotifydiscover/playlist/0f4h4WmPq27eYrXCz93Ncj | |
| Zach: https://open.spotify.com/user/spotifydiscover/playlist/4oovRX5zvmhuzC3uX7Es8v | |
| Chris: https://open.spotify.com/user/spotifydiscover/playlist/0FsK5ub7IXpa7J78t03uth | |
| Sascha: https://open.spotify.com/user/spotifydiscover/playlist/4NfabV5DBk1RdMQOUiRIiE | |
| Cam: https://open.spotify.com/user/spotifydiscover/playlist/7KnDkFHwq8BY6WAMbdB8Wf | |
| Sarah: https://open.spotify.com/user/spotifydiscover/playlist/7EFdKTPZKnzbrTj8wf4YMG | |
| Becky: https://open.spotify.com/user/spotify/playlist/37i9dQZEVXcS4Wh3BmdREX |