Poetic AI
Generating Poetry through Fine-Tuning on GPT-2
Poetic AI
Poetic AI is a short project I did with Dawit Gebregziabher1 as part of Comp 119 (Big Data) during my Masters program at Tufts. The project is a natural language model that can write poetry given a short prompt.
The model is created by fine tuning the GPT-22 small and medium models on the Gutenberg Poetry Corpus3. It requires a lot of parameter tuning and failed poems to get results that look reasonable (see below). In parameter tuning, we encountered a lot of the same problems that Deepmind encountered, namely that for temperature values that are too low, the model tends to repeat itself over and over agin. Also, the model doesn’t know to stop generating words, so the output has to be cutoff at an arbitrary predefined length.
Check out the colab notebook we ran our code in here. Scroll down to check out some of the poetry we generated!
Poetry samples
Given the seed word ‘Love’
Love
Is the first thing in the world,
We all know.
But I’ll tell you all, love,
Love gives us more
Than all words ever told;
When it has taken hold of us,
I will tell you all,
Love is the first thing.
We all know.
Given the seed phrase ‘Beautiful Sunsets’
Beautiful Sunsets in a far away place
As for you from afar, the day is gone when the suns begin
To glow, and your eyes of peace will not be sad or sad,
You have never been alone to this fair and pleasant town,
But I have known you at the close and long ago,
And I know you remember well your fair and lovely face,
And the way in which the morning starlight blanches the sun.
I remember all of you, happy lovers dear,
And the great things that were told to you before your death,
And the things that you spoke of in your heart so true.
Of all the people in that town, none are in all the world,
The richest are all, and the least are poor, and many a great
The most humble are the poor, and these are the best."
A failed poem
The parameter tuning wasn’t set quite right here (or maybe it was depending on how much you like this poem).
Love
I fear, and fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
And fear grow,
Acknowledgements
Cover image by Álvaro Serrano on Unsplash
Featured image by Art Lasovsky on Unsplash
-
link to Dawit’s github or linkedin ↩