Created
February 14, 2020 19:14
-
-
Save MattPitlyk/45541145ad48b93da395f0a72ec2e7dc to your computer and use it in GitHub Desktop.
Fine-Tuning GPT-2 on a Custom Dataset
Just a text file with each example sentence on its own line. No json needed.
If I want to create a text summary ml model by fine tuning GPT2, then how the text file should be formatted?
the text will get tokenized in its own?
we just have to pass the text file, what kind of formatting should be done,
Question: 'the ques'
Answer: 'the answer'
will this format work?
And secondly my colab session is crashing when we train the model, what can be the solution to this?
It requires >20gb memory so you can subscribe to colab plus or use a free trial virtual machine.
Is that gpt2 fine tuning approach effective btw?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What would the format be for the dataset? Could it just be sentences? JSON Format or Text format? This is the stuff that one needs to know, at least for beginners in machine learning. If you're still active on GitHub, would love a reply. Thanks!