Karan Acharya thekaranacharya

🎯

Focusing

Machine Learning Engineer | Computer Vision, NLP, Large Language Models, Explainable AI

11 followers · 17 following

View GitHub Profile

Recently created

Least recently created

Recently updated

Least recently updated

thekaranacharya / lora_fine_tuning.py

Last active July 4, 2024 06:54

Implementation: Using Low-rank Adaptation(LoRA) fine-tuning for LLMs as described in https://arxiv.org/pdf/2106.09685

	# Imports
	import torch
	from functools import partial
	from transformers import AutoModelForSequenceClassification

	# Classes
	# Define the LoRA layer
	class LoRA(torch.nn.Module):
	def __init__(self, in_dim, out_dim, rank, alpha) -> None:
	"""

thekaranacharya / adapter_fine_tuning.py

Last active July 4, 2024 06:48

Implementation: Using Adapter layers to fine-tune LLMs as described in [Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/pdf/1902.00751)

	# Imports
	import torch
	from functools import partial
	from transformers import AutoModelForSequenceClassification

	# Classes
	# Define the Adapter layer
	class Adapter(torch.nn.Module):
	"""
	Implements Adapter layer as described in the paper.

thekaranacharya / simple_fine_tuning.py

Last active July 3, 2024 22:54

Implementation: Using simple fine-tuning (freezing all layers except the last few Linear layers) for LLMs

	# Imports
	from transformers import AutoModelForSequenceClassification

	###################
	model_uri = "distilbert/distilbert-base-uncased"
	num_classes = 2

	# Initialise the model
	model = AutoModelForSequenceClassification.from_pretrained(
	model_uri, num_labels=num_classes