Skip to content

Instantly share code, notes, and snippets.

View johnmcdowall's full-sized avatar
💭
😺

John McDowall johnmcdowall

💭
😺
View GitHub Profile
from transformers import AutoTokenizer, AutoModel
import torch
# MODEL CKPT is downloaded from: "jinaai/jina-embeddings-v2-base-en" # has context len of 8192
MODEL_CKPT = "/Users/rohan/3_Resources/ai_models/jina-embeddings-v2-base-en"
def recursive_splitter(text: str, separators: list[str], chunk_size: int) -> list[str]:
if len(separators) == 0:
words = text.strip().split(' ')
@Duartemartins
Duartemartins / batchtrequest.rb
Last active May 2, 2024 19:57
Google Search Console Batch Index Requesting
require 'googleauth'
require 'json'
require 'net/http'
require 'uri'
require 'nokogiri' # to parse the sitemap
require 'cgi'
SCOPES = ['https://www.googleapis.com/auth/indexing']
ENDPOINT = 'https://indexing.googleapis.com/v3/urlNotifications:publish'
JSON_KEY_FILE = 'page-indexing-000000-26ca419af0a4.json' #replace with your json key file
@johnmcdowall
johnmcdowall / aws.tf
Created November 16, 2023 03:34
The Terraform config to setup a managed LB, Firewall, PSQL DB, Redis and droplets on DO, with bonus ECR on AWS.
resource "aws_ecr_repository" "repo" {
name = "${var.product_name}-app"
}
resource "aws_ecr_lifecycle_policy" "ecr-lifecycle-policy" {
repository = aws_ecr_repository.repo.name
policy = jsonencode({
rules = [
{
@fractaledmind
fractaledmind / has_many_attached.rb
Created October 25, 2023 20:21
Vanilla Rails isomorphic attachment validations
# /app/models/concerns/has_many_attached.rb
module HasManyAttached
extend ActiveSupport::Concern
class_methods do
def has_many_attached(name, dependent: :purge_later, service: nil, strict_loading: false, **options)
super(name, dependent: :purge_later, service: nil, strict_loading: false)
if options[:file_types].any?
validate "validate_#{name}_file_types".to_sym
@huksley
huksley / mrsk-accessory-docker-registry.yml
Last active August 13, 2023 10:56
Run docker registry as mrsk accessory
registry-github: &docker
username: username
password:
- DOCKER_TOKEN
registry-internal: &internal
server: registry.example.com
username: testuser
password: testpassword123
# This uses YAML magic and Ruby template language to select registry on the fly
registry: *<%= ENV["MRSK_REGISTRY"] || 'internal' %>
@huksley
huksley / mrsk.md
Last active January 12, 2024 17:24
mrsk - the missing manual

MRSK

This documentation adds important additions to the docs for mrsk deploy tool (see github.com/mrsked/mrsk)

Destination flag

You can use mrsk deploy --destination staging

This will read config/deploy.yml and config/deploy.staging.yml files, and also will read .env.staging file if it exists.

@bogsen
bogsen / fix-plex-hdr-subtitles.md
Last active September 6, 2024 10:48
Fix Plex transcoding HDR videos very slowly when burning subtitles in at the same time

This workaround should no longer be necessary with newer Plex versions: https://forums.plex.tv/t/subtitle-burn-in-improvement-test-build/884481


Old content left here for reference:

What?

For some reason, although Plex supports hardware-accelerated HDR tone mapping, Plex decides not to use it when subtitles need to be burned in. This results in extremely slow transcoding on hardware that should

@talentdeficit
talentdeficit / json_config.ex
Created January 5, 2017 03:50
using a json blob as application config
def reload_config(path_to_config) do
config = File.read!(path_to_config) |> Poison.decode!
Application.put_env(AppA, KeyA, config["key_a"])
Application.put_env(AppB, KeyB, config["key_b"])
Application.stop(AppA)
Application.stop(AppB)
Application.start(AppA)
Application.start(AppB)
@timhwang21
timhwang21 / renderDoctor.js
Last active February 8, 2019 23:33
Diagnose inefficient renders by identifying "changed" props that are actually equal
import React from "react";
import { isEqual } from "underscore";
/**
* HOC for diagnosing unnecessary renders and identifying faulty selectors
*
* Adds a logger function to a component that lists all changed props
* Also checks if changed props are deeply equal
*
* Usage: Decorate the component you are investigating with renderDoctor:
@tj
tj / Connector.js
Last active November 26, 2019 15:43
import React from 'react'
const bgStyles = {
strokeWidth: 3,
strokeLinejoin: 'round',
strokeLinecap: 'round',
fill: 'none',
stroke: '#c3fdff'
}