Dan Zheng dan-zheng

Using Linear Algebra to Convert a Large Code Model

Background

The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
mono, which is fine-tuned from the multi model on Python code only.

Compile Time Constant Expressions for Swift

Proposal: SE-NNNN
Authors: Marc Rasi, Chris Lattner
Review Manager: TBD
Status: Awaiting implementation

Swift-evolution thread: https://forums.swift.org/t/pitch-compile-time-constant-expressions-for-swift/12879

Introduction

Generic protocols

One of the most commonly requested features is the ability to parameterize protocols themselves. For example, a protocol that indicates that the Self type can be constructed from some specified type T or converted into T:

// T -> Self:
protocol ConstructibleFrom<T> {
  init(from value: T)
}

	Here's a block of code in some imaginary assembly language:

	jmp A
	block1
	jmp A
	block2
	jmp A
	block3
	.A ...

	import jax
	from jax import core
	from graphviz import Digraph
	import itertools


	styles = {
	'const': dict(style='filled', color='goldenrod1'),
	'invar': dict(color='mediumspringgreen', style='filled'),
	'outvar': dict(style='filled,dashed', fillcolor='indianred1', color='black'),

	trait Futamura1 {
	// Roughly follow the idea from http://blog.sigfpe.com/2009/05/three-projections-of-doctor-futamura.html
	type P[_] // program representation type
	type M[_] // machine type

	/* To run a machine: */
	def run[A](ma: M[A]): A

	/* The specializer: a machine that
	takes a program representation of A => B,

	-- The Datafun runtime.
	module Runtime where

	import qualified Data.Set as Set
	import Data.Set (Set)

	class Eq a => Preord a where
	(<:) :: a -> a -> Bool

	class Preord a => Semilat a where

	import Darwin

	@dynamicMemberLookup
	struct Environment {
	subscript(dynamicMember name: String) -> String? {
	get {
	guard let value = getenv(name) else { return nil }

	return String(validatingUTF8: value)
	}

	public protocol RandomSource {
	static var shared: RandomSource { get }
	func next<T : FixedWidthInteger>(_ type: T.Type) -> T
	func nextUniform<T : BinaryFloatingPoint>(_ type: T.Type) -> T
	}

	// Utilizes /dev/urandom
	public class DevRandom: RandomSource {
	public static let shared = DevRandom()