Driven by the idea of creating generative text bots I had to experiment with Markov chains.

I looked for some free text samples to train the algorithm. The writing style of Lovecraft did seem to fit very well for my purpose. Fictional stories, a lot of repeating words. Long sentences with many linking words help as well.

Another option to get results which are close to human expression could be music lyrics. There are some interesting experiments out there, for example the Bowie Lyrics Generator by Stefan Schmid.

I created a Jupyter notebook, and with just a few lines of code I was able to generate texts, which aren’t really distinctive from the source. Or at least funny enough to entertain me. I used a library called markovify. Nothing really difficult, just load the sample text into a pandas.DataFrame, create a text model and let it create strings in a loop.

text_model = markovify.NewlineText(df.sentence, state_size=3)
for i in range(10):
    print(text_model.make_sentence())

Here is an example of the result and the three original sentences.

So Iranon went out of Hatheg into the stony desert near Ulthar, beyond the River Skai.

So Iranon went out of the stable and walked over the narrow stone streets between the gloomy square house of granite, seeking something green, for all was of stone.

Banzai and Atal went out of Hatheg into the stony desert despite the prayers of peasants, and talked of earth’s gods by their campfires at night.

Years of the Titans were recalled, but the host grew timid when he spoke of the dim first age of chaos before the gods or even the Elder Ones were born, and when the other gods came to dance on the peak of Hatheg-Kia in the stony desert near Ulthar, beyond the River Skai

Finally I created a Twitter account, to post some of the text creations. You can find it here.

https://twitter.com/wilhelm_junzt

The next steps would be automating the whole thing to act as autonomous twitter bot, and of course a scoring mechanism to improve quality of generated texts.

Further reading:

Wikipedia on Markov chains: https://en.wikipedia.org/wiki/Markov_chain
Python library used for this bot: https://github.com/jsvine/markovify
Texts by H. P. Lovecraft: https://www.hplovecraft.com/writings/texts/fiction/cc.aspx
Scoring algorithm: https://github.com/google-research/bleurt