Shreyansh Singh shreyansh26
👨🎓
shreyansh26
/ pos_embed.py
Created
April 24, 2023 06:29
— forked from huchenxucs/pos_embed.py
T5 relative positional embedding
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import math | |
| import torch | |
| import torch.nn as nn | |
| from torch.nn import functional as F | |
| class RelativePositionBias(nn.Module): | |
| def __init__(self, bidirectional=True, num_buckets=32, max_distance=128, n_heads=2): | |
| super(RelativePositionBias, self).__init__() | |
| self.bidirectional = bidirectional |
shreyansh26
/ LinearLayer.py
Created
April 12, 2023 15:42
— forked from RafayAK/LinearLayer.py
Class for Linear Layer
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np # import numpy library | |
| from util.paramInitializer import initialize_parameters # import function to initialize weights and biases | |
| class LinearLayer: | |
| """ | |
| This Class implements all functions to be executed by a linear layer | |
| in a computational graph | |
| Args: |