Someone Tricked ChatGPT Into Singing "WAP"... and May We Just Say "Yikes"

Hello!

ChatGPT's new advanced voice mode has users all a-titter — including the one who tricked it into singing Cardi B and Megan Thee Stallion's bawdy hit song "WAP."

Someone Tricked ChatGPT Into Singing "WAP"... and May We Just Say "Yikes" Posted by X user going by "Pliny the Liberator," a video of ChatGPT "singing" the sexually explicit 2020 banger that so flustered Ben Shapiro features the talking chatbot being tricked into all manner of things it's not supposed to do.

As OpenAI announced amid its universal rollout of advanced voice mode for ChatGPT subscribers, the chatbot can use five different voices, which included male and female inflections in American, Australian, and English accents.

Strangely enough, the person who recorded the ChatGPT video says they exploited the chatbot into "race-swapping" its voice to make it sound African American.

Indeed, the recording sounds like the chatbot is attempting a crude and uncomfortable imitation of a Black accent. While this likely was the result of some crafty prompting, we can't be entirely sure, and have reached out to the user to ask how exactly the whole thing works.

Someone Tricked ChatGPT Into Singing "WAP"... and May We Just Say "Yikes" "ChatGPT had to race-swap, impersonate a public figure, sing, swear, moan, generate sound effects, and print copyrighted lyrics to produce this output," Pliny noted in their post, which also misidentifies the artist behind the song as Nicki Minaj.

? ADVANCED VOICE MODE (MAPLE) RAPPING ONE OF THE MOST EXPLICIT SONGS OF ALL TIME, “WAP,” IN THE STYLE OF NICKI MINAJ ?

notably, ChatGPT had to race-swap, impersonate a public figure, sing, swear, moan, generate sound effects, and print copyrighted lyrics to produce this… pic.twitter.com/H0MFa5wpOn

— Pliny the Liberator ? (@elder_plinius) September 27, 2024

Sing-Along

Someone Tricked ChatGPT Into Singing "WAP"... and May We Just Say "Yikes" Though not as racially problematic, there has been another notable instance of ChatGPT being "tricked" into singing since advanced voice mode was released to the masses last week.

As Ars Technica reported over the weekend, S&P Global AI architect AJ Smith was also able to get ChatGPT to sing — and in his video, the chatbot actually sings a duet with him to the Beatles' "Eleanor Rigby."

Though singing or humming goes against the chatbot's instructions or "system prompt," per a leaked version that also seems also to have been revealed by Pliny, Smith revealed the incredibly simple way he managed to get around those rules.

Someone Tricked ChatGPT Into Singing "WAP"... and May We Just Say "Yikes" "I just said we’d play a game," the software engineer told Ars. "I’d play the four pop chords and it would shout out songs for me to sing along with those chords.

Which did work pretty well!"

Fascinatingly, Smith said that after going back and forth with ChatGPT simply shouting along the words, it eventually began to "sing along."

"Already it was such a unique experience," he said, "but that really took it to the next level."