Conception

Decision making and consensus among humans (and machines) have been topics I've had on my mind for some time, especially after many years working with decentralized technologies where, part of the core of these systems, is for computers to agree on something and even extend that to human decisions.

The basic idea I wanted to explore here is: given a high number of participants in a process of decision making, where analyzing each argument becomes impractical or even impossible, can an AI be used to automate this step? We also discussed this with Stephen Wolfram at the Wolfram Summer School 2023 in Boston during a session called "AI Ethics/Constitution Q&A" on July 3, 2023.

*UPDATE*: A new version of this experiment (AKA Promptocracy livecoding experiment) was performed during the Wolfram Summer School 2024. Here's the gallery of Proof of Prompting cards. I'll write more about this soon.

For example, when deciding how to solve a problem inside a large community, it becomes impractical or too expensive to analyze the arguments of each person to eventually achieve a consensus. That's one of the reasons people designate a smaller group of persons (e.g. elections) that, ideally, represent different arguments and needs of the community and can make decisions in a more practical way. In reality, representation is not always achieved and decisions can be influenced by many other factors. Can we try to fix this with machines?

I've been wanting to do an experiment about this with real people and real data. Some days ago I had the chance to run a first prototype of an AI democracy experiment during a talk I was invited to and, fortunately, the code worked! It was based on asking the question "If humans could focus on developing one single technology, which one would you choose and why?". Not only was I able to gather the opinions of around 30 participants and run them through GPT4 to get a "general sentiment" from the audience, but also, each person got a digital souvenir with a graphical representation of their answer and a short quote based on it. I plan to improve my code and repeat the experiment as many times as possible.

While I have my own views regarding politics and social structures, I don't know enough about them in a way that makes me think I can contribute to the general discussion around those topics in a meaningful way. So my approach here is more intuitive and mainly focused on the computational discussion.

Development

The basic idea is pretty simple. I needed a way to collect the answers to my question, process them with GP4 and produce text and images with the results.

To collect the answers I deployed a FormPage connected to a databin. The FormPage had the question:

If humans could focus on developing one single technology, which one would you choose and why?

and allowed the users to enter a custom text:

To do this with a live audience, a regular survey would probably use multiple selection answers because not only you would need to parse each answer but also, for a live presentation, you can't read each answer. In this experiment we allow the users to express themselves without predefined answers but we are still able to process all the answers in real time.

This is a sample of the answers:

You can download the complete dataset of answers as a JSON file here.

So I provide the list of answers to GPT-4 and set the system prompt with the context of the experiment:

You are a useful assistant that will help me analyze answers provided by many people to the question: "If humans could only focus on developing one single technology or science, what do you think this technology or science should be?". I will provide the answers as a JSON file where each value is an answer from a person. The answers are mainly in Spanish.

And then I ask the model to return a single answer based on the participants' arguments.

please provide a single line explaining what single technology was chosen based on the best arguments provided by people. You need to provide just one technology.

The criteria to say what's a "best argument" is very basic in this example. I'm just asking GPT to produce an answer based on the "best arguments", so the model is choosing its own criteria. This step could of course be improved by parameterizing each answer to customize the criteria and judgements the model will use to process the answers. This is actually something we are working on at Curador.

I used that same set of answers to feed another program I wrote that uses GPT-4 and DALLE to generate a graphical representation of each answer and a "quote" based on the text. With them, I create a custom digital souvenir for each participant. These are some of my favorite ones:

Results

I generated 167 cards during the last event, which was organized by an innovation office of the Peruvian government. You can see here the full gallery of cards I produced during my talk.

In general, I am very satisfied with the overall quality of cards. I have run this livecoding experiment in two different events and got very positive feedback and enthusiasm from the audiences. I think it's cool that if you attend an online event, you get a personalized Thank You digital souvenir.

However, something I was not expecting was to receive messages from attendees asking how to use this tool for their own events. I wrote this program in about 45min so I haven't designed it yet to be shared with other users. I'm working on a next version of these tools that I can share with more people. I'll update this post once that's done.

In the meantime, please let know if you are interested in this type of tools so I can consider your feedback for my next version.