Uber’s Plug and Play Language Model (PPLM) Allows Steering Topic and Attributes of GPT-2 Models

It’s impressive that Generative models like Open AI’s GPT-2 automatically create texts using limited input. But controlling the attributes (topics, context, sentiment) of these texts, and paragraphs need an extra layer of work that includes architectural modifications/specific data understanding, etc. This work is done by a team of professionals from Uber, Caltech, and the Hong Kong University of Science and Technology. They worked on the model and created the Plug and Play Language Model (PPLM), which takes one or two attributes classifier and combines it with a pre-trained language model.

Paper with Initial Results: https://arxiv.org/pdf/1912.02164.pdf

Github: https://github.com/uber-research/PPLM

Colab Notebook: https://colab.research.google.com/drive/1Ux0Z4-ruiVtJ6jUk98uk6FqfvGHCOYL3

Demo: https://transformer.huggingface.co/model/pplm

Installation

Setup

Asif Razzaq is an AI Journalist and Cofounder of Marktechpost, LLC. He is a visionary, entrepreneur and engineer who aspires to use the power of Artificial Intelligence for good.

Asif's latest venture is the development of an Artificial Intelligence Media Platform (Marktechpost) that will revolutionize how people can find relevant news related to Artificial Intelligence, Data Science and Machine Learning.

Asif was featured by Onalytica in it’s ‘Who’s Who in AI? (Influential Voices & Brands)’ as one of the 'Influential Journalists in AI' (https://onalytica.com/wp-content/uploads/2021/09/Whos-Who-In-AI.pdf). His interview was also featured by Onalytica (https://onalytica.com/blog/posts/interview-with-asif-razzaq/).