LLM Experiments

Better Conversations through ML?

Comments on the internet. Like everyone else, I can't help to look at them, only to quickly want to look away.

Somehow no matter the audience or dryness of the topic, comment forums seem to almost inevitably spiral into a chaotic mess of personal attacks. And yet, there's so much value to be extracted (...much of the time) from the knowledge and perspective of others.

I was curious if we could use LLMs (and how much it would cost) to let us communicate with each other better. Instead of filtering, what if we let LLMs help us express ourselves to each other in a healthier manner?

Unsurprisingly, the LLMs are good at deflecting obviously personal attacks. Perhaps more surprising was that they aren't that easy to jailbreak out of their system prompt, and that LLama3-405B was the best of all of models I tried at following prompts. All of the models tend to rewrite comments in a relatively stilted manner (sigh, other than of course when you turn on the "evil" prompt), but it's not bad.

To see an example of how LLMs handle a charged conversation, see the conversation demo page.

Cost

So how much does it cost to do this? It's surprisingly affordable, and I imagine someone will figure out how to package it into an API soon enough. Processing the average comment and parent tree costs about $0.002 (2/10 of a cent) when using a hosted LLama3 provider.

With some simple filters and cheaper models in front of it, it would certainly be viable to deploy for most use cases. (I mean, if you've got enough users that this matters, you probably can afford it...). A risk would be someone figuring out how to jailbreak and use you as a free LLM service, but rate limits and filters would likely catch the vast majority of freeloaders.

Example Site

Below I mocked up a simple replica of Hacker News which uses an LLM to gently guide comments to ensure they are respectful and contribute to the conversation. Feel free to try adding comments and interacting with them (topics and comments are reset daily).

You can also change the system prompt to make the LLM deliberately evil or incoherent if you like in the settings.