Yusuke Oda odashi

🏠

Working from home

Project Associate Professor at NII, Affiliate Assistant Professor @naist-nlp, Technical Director @cierpa

odashi / parallel.py

Created September 10, 2024 19:24

Parallel command invocation with specified workers

	import argparse as ap
	import dataclasses as dc
	import multiprocessing as mp
	import subprocess as sp
	import sys


	def parse_args() -> ap.Namespace:
	p = ap.ArgumentParser("Runs parallel jobs")

odashi / install_local_python.sh

Last active September 1, 2024 18:02

Local Python installer

	#!/bin/bash
	# This script installs a Python environment built from source.
	# Usage:
	# URL='https://gist.githubusercontent.com/odashi/...copy-paste raw path.../install_local_python.sh'
	# bash <(curl -sL ${URL}) ./env 3.10.14 64

	set -eu -o pipefail

	if [ $# -ne 3 ]; then
	>&2 echo "Usage: install_local_python.sh INSTALL_DIR MAJOR.MINOR.PATCH BUILD_THREADS"

odashi / download_c4_en.bash

Last active January 14, 2024 00:22

Downloads AllenAI mC4 dataset

	#!/bin/bash

	url_prefix=https://huggingface.co/datasets/allenai/c4/resolve/1ddc917116b730e1859edef32896ec5c16be51d0/en

	function download() {
	dataset_name=$1; shift
	splits=$1; shift
	last_split=$(perl -e "print ${splits} - 1")
	mkdir -p ${dataset_name}
	for i in $(seq 0 ${last_split}); do

odashi / gemini_adc.py

Last active December 20, 2023 05:59

Example to call Gemini API with ADC

	# You may need to install the following libraries:
	# * google-auth
	# * requests

	import json
	from typing import Any

	import google.auth
	import google.auth.credentials
	import google.auth.transport.requests

odashi / auth.tsx

Created December 9, 2023 14:48

Firebase Authentication provider with next.js

	"use client";

	import { getApp, getApps, initializeApp } from "firebase/app";
	import {
	getAuth,
	GoogleAuthProvider,
	signInWithPopup,
	User,
	} from "firebase/auth";
	import {

odashi / BUILD

Last active November 17, 2022 06:27

Bazel rule to generate Python structs from OpenAPI documents

	load("@pip_deps//:requirements.bzl", "requirement")

	py_binary(
	name = "datamodel_codegen",
	srcs = ["datamodel_codegen_main.py"],
	main = "datamodel_codegen_main.py",
	visibility = ["//visibility:public"],
	deps = [
	requirement("datamodel-code-generator"),
	],

odashi / class_fastapi_app.py

Last active April 13, 2023 15:14

FastAPI server defined on a class

	# Sometimes you want to define your FastAPI app within a class for some reason, e.g.,
	# you wanted to initialize the servers at the point you intended, not the point of import.
	#
	# In this case, @app.method decorator doesn't work as-is with instance methods due to the
	# difference of its signature.
	#
	# You need to manually call the decorator as a usual function after obtaining self (e.g.,
	# in the __init__ method like below) rather than using the decorator syntax.

	class MyApp:

odashi / nai_prompt_generator.py

Last active October 26, 2022 14:57

Colab UI to generate NovelAI prompts.

	!pip install ipywidgets &> /dev/null
	import ipywidgets as wgt

	controls = wgt.VBox([])
	add_button = wgt.Button(description="Add", icon="plus")
	pos_prompt_area = wgt.Textarea(placeholder="Positive prompts appear here.")
	neg_prompt_area = wgt.Textarea(placeholder="Negative prompts appear here.")
	ui = wgt.VBox([pos_prompt_area, neg_prompt_area, add_button, controls])

	def generate_prompt(change):

odashi / should_mute_on_twitter.txt

Last active December 26, 2022 13:02

should_mute.txt

odashi / delete_old_tweets.py

Last active July 26, 2023 20:35

Cloud Functions script to delete old tweets.

	import datetime
	import logging
	import os

	from google.cloud import logging as cloud_logging
	import tweepy

	THRESHOLD_DAYS = 3
	BATCH_SIZE = 25