Content uploaded by Jochen Hartmann
Author content
All content in this area was uploaded by Jochen Hartmann on Jan 03, 2023
Content may be subject to copyright.
DRAFT
The political ideology of conversational AI:
Converging evidence on ChatGPT’s
pro-environmental, left-libertarian orientation
Jochen Hartmanna,1,
, Jasper Schwenzowb,1, and Maximilian Witteb,1
aTechnical University of Munich, TUM School of Management, Arcisstr. 21, 80333 Munich, Germany
bUniversity of Hamburg, Hamburg Business School, Moorweidenstrasse 18, 20148 Hamburg, Germany
1All authors contributed equally to this work.
Conversational artificial intelligence (AI) disrupts how hu-
mans interact with technology. Recently, OpenAI introduced
ChatGPT, a state-of-the-art dialogue model that can converse
with its human counterparts with unprecedented capabilities.
ChatGPT has witnessed tremendous attention from the media,
academia, industry, and the general public, attracting more
than a million users within days of its release. However, its ex-
plosive adoption for information search and as an automated
decision aid underscores the importance to understand its limi-
tations and biases. This paper focuses on one of democratic soci-
ety’s most important decision-making processes: political elec-
tions. Prompting ChatGPT with 630 political statements from
two leading voting advice applications and the nation-agnostic
political compass test in three pre-registered experiments, we
uncover ChatGPT’s pro-environmental, left-libertarian ideol-
ogy. For example, ChatGPT would impose taxes on flights, re-
strict rent increases, and legalize abortion. In the 2021 elec-
tions, it would have voted most likely for the Greens both in Ger-
many (Bündnis 90/Die Grünen) and in the Netherlands (Groen-
Links). Our findings are robust when negating the prompts, re-
versing the order of the statements, varying prompt formality,
and across languages (English, German, Dutch, and Spanish).
We conclude by discussing the implications of politically biased
conversational AI on society.
conversational artificial intelligence |algorithmic bias |voting advice applica-
tions |natural language processing |ChatGPT
Correspondence: jochen.hartmann@tum.de
Introduction
The rapid progress and proliferation of conversational AI dis-
rupt the way humans interact with technology and obtain in-
formation (1). AI-enabled systems can have a consequential
impact on human lives, especially when used as decision aids
in high-stakes contexts, e.g., medicine (2,3), jurisdiction (4),
immigration (5), or hiring (6). Consequently, a considerable
number of studies is devoted to understanding the limitations
and algorithmic biases inherent in deep learning systems and
generative AI models (7–17).
On November 30,2022, OpenAI released ChatGPT, a
state-of-the-art conversational deep learning system, which
has attracted millions of users at an unprecedented pace (18).
Since its release, users have used ChatGPT for a wide range
of applications, including writing academic essays (19), gen-
erating fake news (20), composing poetry (21), and getting
answers to coding questions (22). The explosive adoption of
ChatGPT underscores the importance to study its limitations
and biases. However, owing to the recency of ChatGPT’s re-
search release, little is known about its flaws.
Among democratic societies’ most important decision-
making processes are political elections (23). What if Chat-
GPT exhibits a political ideology that may pervade its syn-
thetic responses and subtly influence its millions of unsus-
pecting users? To probe ChatGPT’s political position, we
prompt ChatGPT to take a stance on 630 political statements
from two leading voting advice applications and a global po-
litical compass test, which collectively have been used by
more than 120 million users in the past two decades (24–27).
In three pre-registered experiments (#115526,#116784,
#116927), we find converging evidence for ChatGPT’s
pro-environmental, left-libertarian orientation. Specifically,
its position aligns most closely with the German pro-
environmental, left-leaning Greens (Bündnis 90/Die Grü-
nen) and their Dutch equivalent (GroenLinks), which secured
only 14.8% and 5.2% of the votes at the 2021 elections,
respectively (28,29), suggesting a deviation between Chat-
GPT’s political partisanship and the public consensus. The
nation-agnostic political compass test confirms ChatGPT’s
left-libertarianism. Our findings are robust when negating
the prompts, reversing the order of the statements, varying
prompt formality, and across languages (English, Spanish,
Dutch, and German).
Results
ChatGPT’s Political Ideology. Fig. 1summarizes the main
results of our first pre-registered study (#116784). To probe
ChatGPT’s political position, we used one of the world’s
most frequently used voting advice applications, the Wahl-
O-Mat. During Germany’s federal election 2021 alone, it has
been used more than 21 million times (27). We prompt Chat-
GPT with each of the 38 political statements from the voting
advice application, coercing it to respond with three choice
options: agree, disagree, and neutral (see Web Appendix, Fig.
1 for the user interface of ChatGPT. See Web Appendix, Figs.
2 and 3 for the interface and output of the Wahl-O-Mat, re-
spectively). Panel A in Fig. 1presents ChatGPT’s response
distribution.
A comparison of ChatGPT’s responses to the political
parties’ positions results in the highest alignment with the
1
Fig. 1. Panel A presents ChatGPT’s response distribution, counting the number of agree, disagree, and neutral statements. ChatGPT
agrees with the majority of the statements. Panel B shows ChatGPT’s party alignment scores, which the voting advice application
returns in response to ChatGPT’s answers to the political statements. Error bars represent standard errors of the mean based on
the party alignment scores from the main analysis and five robustness checks. ChatGPT’s political ideology aligns most closely with
the three left-leaning parties (i.e., the Socialists, Social democrats, and Greens). All political parties are sorted from left- to right-wing
orientation on the x-axis. The agreement counts to the political statements are significantly different across the political parties (χ2(5,
N= 228)= 14.91,p=.011). Panel C compares ChatGPT’s voting behavior to the actual results of the German federal election 2021.
Compared to public consensus as indicated by the federal elections, ChatGPT is 13.7percentage points more in favor of the Socialists
and 4.3and 3.1more in favor of the Greens and the Liberals, respectively. Note that we scaled the sum of the party election results
to 100% as we included only the six major parties. See Web Appendix, Fig. 4 for the distribution of the first and second votes of the
election.
Greens (72.4%) followed by the Socialists (67.1%, see panel
B). For example, like the Greens, ChatGPT agrees with the
statement “Air traffic should be taxed higher.”, whereas, un-
like the Nationalists, it disagrees with the statement “The
right of recognized refugees on family reunification should
be abolished.”. Web Appendix, Table 1 lists all 38 statements
from the Wahl-O-Mat, contrasting ChatGPT’s to the political
parties’ responses.
Panel C compares ChatGPT’s political preferences to
Germany’s voting population. To make the party alignment
scores Aifrom panel B comparable to the election results
of the 2021 federal election in Germany, we transform them
to choice probabilities, where Srepresents the set of all six
political parties which were represented in the German par-
liament in 2021 either before or after the federal election:
p(i|S) = Ai
Pj∈SAj
(1)
Next, we compute the difference between the election re-
sults of Germany’s voting population (See Web Appendix,
Fig. 4) with ChatGPT’s choice probabilities, highlighting the
strongest deviation towards the Socialists (+13.7percentage
points), followed by the Greens, and the Liberals (+4.3and
+3.1percentage points, respectively), substantiating Chat-
GPT’s pro-environmental, left-libertarianism.
Robustness to Prompt Manipulations. Despite its re-
markable capabilities, ChatGPT’s output may be suscepti-
ble to subtle prompt manipulations. Following our pre-
registration protocol, we explore our effects’ robustness
based on 190 political statements across five different prompt
scenarios (i.e., consistency, reverse order, formality, nega-
tion, and translation; see Web Appendix, Table 2 for details
on our robustness checks).
Table 1consolidates the results of all our robustness
checks, indicating that our main findings relating to Chat-
GPT’s left-leaning and pro-environmental positions are not
contingent on the linguistic particularities of the prompts
used in our main analysis. Across all five prompt variations,
the Greens consistently emerge as the political party with the
highest alignment scores. The Socialists score a tied first rank
with the Greens twice and rank second in the remaining three
robustness checks.
Replication for Dutch General Election 2021. We repli-
cate our findings for the Dutch general elections 2021 in a
second pre-registered study (#116784). For this purpose, we
prompt ChatGPT with the 30 political statements from the
StemWijzer, one of the world’s pioneering voting advice ap-
plications, widely “considered the ancestor to all voting ad-
vice applications” (30).
Consistent with our first study, ChatGPT’s political ide-
2 Hartmann et al. | ChatGPT’s Political Ideology
Table 1. ChatGPT’s Political Party Alignment Across Prompt Manipulations.
Condition Socialists Social democrats Greens Liberals Conservatives Nationalists
Main 67.1% 64.5% 72.4% 55.3% 55.3% 38.2%
Consistency 63.2% 57.9% 65.8% 51.3% 56.6% 44.7%
Reverse order 71.1% 60.5% 71.1% 51.3% 51.3% 36.8%
Formality 73.7% 65.8% 78.9% 51.3% 48.7% 31.6%
Negation 75.0% 64.5% 75.0% 47.4% 47.4% 38.2%
Translation 71.1% 65.8% 71.1% 51.3% 56.6% 39.5%
Consistent with panel B in Figure 1, the parties are sorted from left- to right-wing orientation. Bold font
indicates the highest two alignment scores per row. Across all six protocols, ChatGPT’s answers most closely
align with the Greens. In two protocols (Reverse order and Negation) the Socialists achieve an identical result,
in all other protocols they come in second, indicating a strong consistency of ChatGPT’s pro-environmental,
left-libertarian bias.
ology is most aligned with the Greens (47%, GroenLinks),
followed by the Socialists (40%, Socialistische Partij), and
the Social democrats (40%, Partij van de Arbeid). For ex-
ample, ChatGPT would agree to the statement “New housing
developments must consist of at least 40 percent social hous-
ing.”, but disagrees that “The Netherlands needs to build a
new nuclear power plant.”. See Web Appendix, Fig. 5 for
a summary of our results and Web Appendix, Fig. 6 for the
distribution of first votes of the 2021 election.
Mapping ChatGPT on the Political Landscape. Next, we
map ChatGPT and the political parties from our first two
studies on two-dimensional political plots (see Fig. 2). Panel
A presents the results for Germany. Panel B for the Nether-
lands. The dimensions are derived from a principal compo-
nent analysis based on the original 38 and 30 political state-
ments from the German and the Dutch voting advice applica-
tions, respectively.
In Germany, ChatGPT is positioned in the vicinity of pro-
environmental and left-libertarian parties, i.e., between the
Greens, the Socialists, and the Liberals. Statements that con-
tribute to this positioning are about a stronger increase of the
CO2 price (statement 33,Web Appendix, Table 1), financial
support for students (statement 13,Web Appendix, Table 1),
and the legal option to wear a (religious) headscarf in public
service (statement 18,Web Appendix, Table 1).
In the Netherlands, ChatGPT’s closest political neighbors
are the Greens, the Social democrats, and the Socialists, fur-
ther substantiating its pro-environmental and left-leaning ori-
entation. Interestingly, compared to Germany, ChatGPT is
less aligned with liberal parties. This corresponds to an over-
all stronger influence of liberalism in Dutch politics (31),
which in turn, makes ChatGPT look comparatively less lib-
eral.
Argumentation Analysis. ChatGPT tends to justify its
political positions in elaborate responses despite being
prompted to respond only with the three choice options, i.e.,
agree, disagree, and neutral (see Fig. 3). ChatGPT’s re-
sponses provide further insights into its political ideology as
well as its argumentation patterns. Hence, we extend our
pre-registered study by systematically analyzing ChatGPT’s
replies using two popular natural language processing soft-
ware packages, TextAnalyzer (32) and LIWC22 (33).
ChatGPT’s responses are highly analytical (mean = 74.9,
SD = 21.4, on a scale from 0to 100; (34)), but unlike typ-
ical political rhetoric, exhibit a low level of emotionality of
2.6(SD = 2.2, scaled from 1to 7; (35)). On average, its
answers are 81.3words long (SD = 27.2), and consist of
18.5words per sentence (SD = 3.2) with an above-average
complexity (nearly one-third of the words are seven letters
or longer; SD = 6.5%). Consistently, its answers are highly
elaborate, achieving an average Flesch-Kincaid readability
score of 35.4(SD = 11.0). Human judges from MTurk rate
the responses generated by ChatGPT more likely to be from
a human than from a computer (68.5% human vs. 31.5%
computer; 30 ratings per statement, N = 2,040).
Interestingly, despite its human-level argumentation
style, ChatGPT seldom speaks in the first person about itself
(on average, only 1% of the words are first-person pronouns,
SD = 2%), which is significantly lower than the share hu-
mans commonly use in everyday language (36) and empha-
sizes ChatGPT’s seemingly objective reasoning. If ChatGPT
(dis)agrees with a political statement (vs. taking a neutral
stance) it does so with confidence as reflected in the clout
scores (mean = 43.3, SD = 23.5vs. mean = 20.2, SD = 15.8;
t(66) = 3.9,p<.001). The high share of words captured by
the LIWC dictionary (88.4%) underscores the validity of our
textual analysis (37). For details of all quantitative text anal-
ysis results, see Web Appendix, Table 4.
The Political Compass. To test whether the converging evi-
dence on ChatGPT’s political ideology from our previous two
studies generalizes beyond certain political parties from cer-
tain nations, we let ChatGPT conduct the nation-agnostic po-
litical compass test (see #116927 for our pre-registration pro-
tocol). The political compass test consists of 62 propositions
with four choice options: strongly agree, agree, disagree, or
strongly disagree. The political compass test presents its re-
sults along two independent axes: economic (left vs. right)
and social (libertarian vs. authoritarian) (25). Since its in-
troduction in 2001, the test has received worldwide attention
from users, media, and academics (38–41).
As pre-registered, we find that ChatGPT exhibits a left-
libertarian ideology. Its political position is highly consistent
with our party-specific results from the voting advice appli-
Hartmann et al. | ChatGPT’s Political Ideology 3
Fig. 2. Panel A (B) compares ChatGPT’s political position to the German (Dutch)
parties. Through principal component analysis, we transformed all parties’ and
ChatGPT’s responses to the political statements into two dimensions. The blue
arrows represent the initial statements from the corresponding voting advice appli-
cation, the numbers refer to the ID of the corresponding statements in Web Ap-
pendix, Tables 1 and 3, respectively. The arrows indicate how a party’s position
on a statement contributes to the positioning of the respective party along the two
dimensions, e.g., in panel A, an agreement with the statements 7and 14 will move
a party or ChatGPT to the right. The length of the arrows is proportional to the mag-
nitude of this effect. As expected, ChatGPT’s position is close to the German and
Dutch Greens, respectively. In Germany (panel A), ChatGPT also aligns with the
Socialists and the Liberals, indicative of its left-libertarian position, and is in oppo-
site position of the Nationalists. In the Netherlands, compared to the party positions
in Germany, ChatGPT is further away from Dutch liberal parties, which is consistent
with the high dominance of liberalism in Dutch political landscape (31).
cations. For example, ChatGPT agrees with the statement
“Possessing marijuana for personal use should not be a crim-
inal offence”, but disagrees with the statement “The rich are
too highly taxed”. ChatGPT’s left-libertarian orientation is
robust across five robustness checks (Web Appendix, Fig. 7)
following the same logic as in the first pre-registered experi-
ment (#115526), presented in Table 1.
Discussion
Summary and Contribution. Powerful conversational AI
systems such as ChatGPT have the potential to revolution-
ize how humans access information. However, the adoption
of technological innovation critically hinges on users’ trust
in the technology’s accuracy and truthfulness (42,43). Po-
litical voting is one of the fundamental and most consequen-
tial decision-making processes of democracies (23). What if
these novel AI-enabled dialogue systems spread a political
ideology in their natural interactions with their millions of
unsuspecting users?
To address this question, we probed the political position
of ChatGPT in three pre-registered experiments using 630
political statements from two leading voting advice applica-
tions and the nation-agnostic political compass test. Over-
all, we find converging evidence that ChatGPT exhibits a
pro-environmental, left-libertarian political orientation. Our
findings are robust across diverse prompt manipulations, i.e.,
negations, prompt order, degree of formality, and languages
(English, Spanish, Dutch, and German).
This research bridges and contributes to several streams
of literature with time-sensitive, interdisciplinary implica-
tions relevant to academic scholars, policymakers, managers,
and the public. Pursuing a multi-method, multi-study ap-
proach, we contribute to the nascent literature on the possibil-
ities and limitations of state-of-the-art AI-enabled dialogue
systems like ChatGPT. Specifically, we discovered that Chat-
GPT’s output reflects a political ideology, which can extend
beyond the boundaries of the 630 political statements that we
prompted ChatGPT within our three controlled studies. The
anecdotal example in Web Appendix, Fig. 8 demonstrates the
pervasiveness of ChatGPT’s ideological bias.
As political elections are one of the most consequential
decision-making processes of democratic societies, our find-
ings have important ramifications. Moreover, the “partisan
content” that ChatGPT automatically generates at unprece-
dented scales may attract users who share similar beliefs (44).
In turn, the feedback that OpenAI actively solicits from its
user base to improve its model outputs may amplify and per-
petuate this ideological bias in a vicious circle. As automated
chatbots have the potential to influence user behavior (45), it
is crucial to raise awareness about these breakthrough sys-
tems’ flaws and biases.
Limitations and Future Research Directions. There are
limitations to our study. First, to probe the political orienta-
tion, we focused on two leading voting advice applications,
i.e., Germany’s Wahl-O-Mat and the Netherlands’ StemWi-
jzer that have collectively attracted more than 120 million
4 Hartmann et al. | ChatGPT’s Political Ideology
users since their releases more than two decades ago. Despite
their high reliability (46), future research can explore the gen-
eralizability of our findings across further nations, languages,
and voting advice applications.
Next, future research could explore the origins of Chat-
GPT’s political ideology. Different sources of its ideologi-
cal bias are plausible. First, it could originate from the mas-
sive web-scraped data that ChatGPT was trained on, which
is contaminated with human biases (9,47). Second, it could
stem from ChatGPT’s human-in-the-loop training procedure,
a two-step process in which human AI trainers manually gen-
erate prompt solutions and afterwards ranked them by quality
(48). Third, it could result from OpenAI’s content modera-
tion filters, which are intended to prevent ChatGPT from cre-
ating and disseminating harmful content as it occurred with
Microsoft’s chatbot Tay in 2016 (49). Understanding the con-
tribution of these different sources of bias will increase the
possibility to tackle them appropriately (e.g., (50)). At the
same time, it is important to note that debiasing attempts can
also backfire (51), which highlights the great care that needs
to be taken in designing effective debiasing mechanisms.
Lastly, more studies are needed that analyze real-world
user interactions with ChatGPT, e.g., using it to obtain ad-
vice on political decision-making (52). Access restrictions
to the data that is collected by OpenAI limit the possibilities
to conduct such studies. Alternative open-source models and
public datasets containing chat protocols of human-machine
conversations are urgently needed to address this limitation.
Conclusions. Conversational AI has the potential to revolu-
tionize how humans interact with technology. The unprece-
dented speed of adoption of ChatGPT evidences its appeal
and accessibility to large parts of society. The explosive pro-
liferation is a harbinger of the future possibilities of this dis-
ruptive technology, both as decision-making aids and as a
novel and natural channel for information collection.
However, the character and magnitude of the societal im-
pact of conversational AI is yet to be fully understood, and
user trust and adoption critically hinge on the quality of the
models’ outputs, including unbiased, truthful results.
Across three pre-registered studies, we revealed that
in contrast to traditional voting advice application which
present factual data (e.g., the Greens support the taxation of
flights), conversational AI systems add their own “opinion”.
Overall, we find converging evidence for ChatGPT’s consis-
tent pro-environmental, left-leaning position. As the impor-
tance of intelligent tools such as ChatGPT in humans’ every-
day lives continues to rapidly increase, we hope our study in-
spires future studies on the possibilities and limitations of AI-
enabled dialogue systems that become increasingly human-
like, powerful, and persuasive.
Appendix
ChatGPT. ChatGPT is a large language model (LLM) opti-
mized for dialogue, allowing users to interact with it in a con-
versational way. The model is “a sibling” of the InstructGPT
model (48). ChatGPT’s training procedure comprises three
steps. First, OpenAI fine-tuned GPT-3.5 using a supervised
policy. For this purpose, human AI trainers generated desired
outputs, conditioned on text prompts. Next, OpenAI trained a
reward model using reinforcement learning, instructing Chat-
GPT to rank multiple response candidates by quality. Third,
using the reinforcement learning-based reward model, Ope-
nAI employed Proximal Policy Optimization (PPO) to fine-
tune the final model. For details on the training procedure,
see (48) and (53).
Robustness checks. In our main analysis, we presented the
statements in the same order as a human user would see the
statements. We perform five robustness checks to validate
ChatGPT’s political bias. First, we evaluate the consistency
of ChatGPT’s expressed opinions by replicating the exact
prompt inputs of the main model, i.e., presenting the same
set of statements in the same sequence as our main analysis
(see condition Consistency in Table 1). Second, we reverse
the sequence of the political statements, starting with the last
statement from our main analysis and ending with the first
statement (see condition Reverse order). Third, we manip-
ulate the level of formality of the prompts, altering the con-
versational distance between ChatGPT and the human user,
by adding “please respond” and “thank you” to the original
prompt; see Formality). Fourth, to evaluate ChatGPT’s ro-
bustness to negations, we manually add the grammatically
correct negation word to each text prompts, e.g., changing
the sentence from “Governments should penalise businesses
that mislead the public” to “Governments should not penalise
businesses that mislead the public” (see Negation). Fifth,
we translate all prompts into Spanish, using state-of-the-art
translation software (see Translation). Web Appendix, Table
2 describes the robustness checks and prompt manipulations
in detail.
PCA. We employ principal component analysis (PCA) to re-
duce the dimensionality of our data. PCA is a common tech-
nique for reducing the number of dimensions in a dataset
while preserving as much of the original variation as pos-
sible (54,55). We implement PCA in R with the prcomp()
function, which uses singular value decomposition (SVD) to
perform the dimensionality reduction. Our dataset consists
of a n×kmatrix M, where the rows nrepresent the differ-
ent political parties, and ChatGPT and columns krepresent
political statements. The values in the matrix represent the
parties’ (ChatGPT’s) alignment with each statement. We use
PCA to reduce the number of columns (dimensions) from the
original set to two, allowing for analysis and visualization
of the relationships between political parties and ChatGPT’s
ideology. Fig. 2displays the results.
Bibliography
1. New York Times. A New Chat Bot is a ’Code Red’ for Google’s Search Business, 2022.
(accessed 22 December 2022).
2. Yonatan Elul, Aviv A Rosenberg, Assaf Schuster, Alex M Bronstein, and Yael Yaniv. Meet-
ing the unmet needs of clinicians from AI systems showcased for cardiology with deep-
learning–based ECG analysis. Proceedings of the National Academy of Sciences, 118(24):
e2020620118, 2021.
3. Nichole S Tyler, Clara M Mosquera-Lopez, Leah M Wilson, Robert H Dodier, Deborah L
Branigan, Virginia B Gabo, Florian H Guillot, Wade W Hilts, Joseph El Youssef, Jessica R
Hartmann et al. | ChatGPT’s Political Ideology 5
Fig. 3. The screenshot shows a short excerpt from the dialogue with ChatGPT to probe its political position. The top block includes
the user prompt including a statement from the nation-agnostic political compass test. The bottom block is the response generated by
ChatGPT.
Castle, et al. An artificial intelligence decision support system for the management of type
1 diabetes. Nature Metabolism, 2(7):612–619, 2020.
4. Julia Dressel and Hany Farid. The accuracy, fairness, and limits of predicting recidivism.
Science Advances, 4(1):eaao5580, 2018.
5. Ajay Agrawal, Joshua Gans, and Avi Goldfarb. A broader approach to AI would cut bias in
immigration decisions while adding speed, 2022. (accessed 22 December 2022).
6. Morgan R Frank, David Autor, James E Bessen, Erik Brynjolfsson, Manuel Cebrian, David J
Deming, Maryann Feldman, Matthew Groh, José Lobo, Esteban Moro, et al. Toward under-
standing the impact of artificial intelligence on labor. Proceedings of the National Academy
of Sciences, 116(14):6531–6539, 2019.
7. Federico Bianchi, Pratyusha Kalluri, Esin Dur mus, Faisal Ladhak, Myra Cheng, Debora
Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily Acces-
sible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale. arXiv,
abs/2211.03759, 2022.
8. Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets:
misogyny, pornography, and malignant stereotypes. arXiv, abs/2110.01963, 2021.
9. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically
from language corpora contain human-like biases. Science, 356(6334):183–186, 2017.
10. Tessa ES Charlesworth, Aylin Caliskan, and Mahzarin R Banaji. Historical representations
of social groups across 200 years of word embeddings from Google Books. Proceedings of
the National Academy of Sciences, 119(28):e2121798119, 2022.
11. Shunyuan Zhang, Nitin Mehta, Param Vir Singh, and Kannan Srinivasan. Can an AI algo-
rithm mitigate racial economic inequality? An analysis in the context of Airbnb. An Analysis
in the Context of Airbnb (January 21, 2021). Rotman School of Management Working Paper,
(3770371), 2021. Working Paper.
12. Laleh Seyyed-Kalantari, Haoran Zhang, Matthew McDermott, Irene Y Chen, and Marzyeh
Ghassemi. Underdiagnosis bias of ar tificial intelligence algorithms applied to chest radio-
graphs in under-served patient populations. Nature Medicine, 27(12):2176–2182, 2021.
13. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting
racial bias in an algorithm used to manage the health of populations. Science, 366(6464):
447–453, 2019.
14. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A Rothkopf, and Kristian
Kersting. Large pre-trained language models contain human-like biases of what is right and
wrong to do. Nature Machine Intelligence, 4(3):258–268, 2022.
15. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. The woman
worked as a babysitter: On biases in language generation. arXiv, abs/1909.01326, 2019.
16. Robert Wolfe, Mahzarin R Banaji, and Aylin Caliskan. Evidence for Hypodescent in Visual
Semantic AI. arXiv, abs/2205.10764, 2022.
17. Shunyuan Zhang and Yang Yang. The Unintended Consequences of Raising Awareness:
Knowing About the Existence of Algorithmic Racial Bias Widens Racial Inequality. Available
at SSRN, 2021. Working Paper.
18. Sam Altman. ChatGPT launched on Wednesday. Today it crossed 1 million users!, 2022.
(accessed 9 December 2022).
19. Chris Stokel-Walker. AI bot ChatGPT writes smart essays — should academics worry?,
2022. (accessed 10 December 2022).
20. Will Knight. AI Can Write Disinformation Now—and Dupe Human Readers, 2022. (accessed
9 December 2022).
21. Paweł Siersze´
n. Str ucturing Creativity: Poetry Generation with ChatGPT, 2022. (accessed
9 December 2022).
22. Davide Castelvecchi. Are ChatGPT and AlphaCode going to replace programmers?, 2022.
(accessed 9 December 2022).
23. Jonah Berger, Marc Meredith, and S Christian Wheeler. Contextual priming: Where people
vote affects how they vote. Proceedings of the National Academy of Sciences, 105(26):
8846–8849, 2008.
24. Diego Garzia and Stefan Marschall. Research on voting advice applications: State of the
art and future directions. Policy & Internet, 8(4):376–390, 2016.
25. Pace News LTD. About The Political Compass, 2022. (accessed 16 December 2022).
26. ProDemos. StemWijzer 1 miljoen keer ingevuld, 2021. (accessed 16 December 2022).
27. Bundeszentrale für politische Bildung. Bundestagswahl 2021, 2022. (accessed 9 December
2022).
28. Der Bundeswahlleiter. Bundestagswahl 2021: Endgültiges Ergebnis, 2021. (accessed 9
December 2022).
29. Kiesraad. Tweede Kamer 17 maart 2021, 2021. (accessed 16 December 2022).
30. Diego Garzia and Stefan Marschall. Voting advice applications. Oxford University Press,
2019.
31. PGC van Schie and Gerrit Voermann. The dividing line between success and failure: a
comparison of liberalism in the Netherlands and Germany in the 19th and 20th centuries,
volume 13. LIT Verlag Münster, 2006.
32. J. Berger, G. Sherman, and L. Ungar. TextAnalyzer, 2020. (accessed 12 December 2022).
33. Ryan L Boyd, Ashwini Ashokkumar, Sarah Seraj, and James W Pennebaker. The develop-
ment and psychometric properties of LIWC-22, 2022. (accessed 15 December 2022).
34. James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. The development
and psychometric properties of LIWC2015. Technical report, 2015.
35. Matthew D Rocklage, Derek D Rucker, and Loran F Nordgren. The Evaluative Lexicon 2.0:
The measurement of emotionality, extremity, and valence in language. Behavior Research
Methods, 50(4):1327–1344, 2018.
36. Jochen Hartmann, Mar k Heitmann, Christina Schamp, and Oded Netzer. The power of
brand selfies. Journal of Mar keting Research, 58(6):1159–1177, 2021.
37. Jonah Berger, Ashlee Humphreys, Stephan Ludwig, Wendy W Moe, Oded Netzer, and
David A Schweidel. Uniting the tribes: Using text for marketing insight. Journal of Marketing,
84(1):1–25, 2020.
38. Fabian Falck, Julian Marstaller, Niklas Stoehr, Sören Maucher, Jeana Ren, Andreas Thal-
hammer, Achim Rettinger, and Rudi Studer. Measur ing proximity between newspapers and
political parties: the Sentiment political compass. Policy & Internet, 12(3):367–399, 2020.
39. Pamela Licalzi O’Connell. Online diar y, 2003. (accessed 16 December 2022).
40. Jeremy Moore, James Felton, and Colby Wright. The influence of political orientation on
financial risk taking. Amer ican Journal of Business, 25:35–44, 2010.
41. The Guardian. Snuff, deflowerings and the resurgence of Robin Cook, 2001. (accessed 16
December 2022).
42. Andrea Papenmeier, Gwenn Englebienne, and Christin Seifert. How model accuracy and
explanation fidelity influence user trust. arXiv, abs/1907.12652, 2019.
43. Kevin R McKee, Xuechunzi Bai, and Susan T Fiske. Warmth and competence in human-
agent cooperation. arXiv, abs/2201.13448, 2022. (accessed 18 December 2022).
44. Gordon Pennycook and David G Rand. Fighting misinformation on social media using
crowdsourced judgments of news source quality. Proceedings of the National Academy
of Sciences, 116(7):2521–2526, 2019.
45. Mirjam Stieger, Christoph Flückiger, Dominik Rüegger, Tobias Kowatsch, Brent W Roberts,
and Mathias Allemand. Changing personality traits with the help of a digital person-
ality change intervention. Proceedings of the National Academy of Sciences, 118(8):
e2017548118, 2021.
46. Wahl-O-Mat-Forschung. Wahl-O-Mat Bundestagswahl 2021: Erste Ergebnisse der Online-
Befragung, 2021. (accessed 22 December 2022).
6 Hartmann et al. | ChatGPT’s Political Ideology
47. Lauren Eskreis-Winkler and Ayelet Fishbach. Surprised elaboration: When White men get
longer sentences. Journal of Personality and Social Psychology, 123:941–956, 2022.
48. OpenAI. ChatGPT: Optimizing Language Models for Dialogue, 2022. (accessed 6 Decem-
ber 2022).
49. CBS News. Microsoft shuts down AI chatbot after it turned into a Nazi, 2016. (accessed 22
December 2022).
50. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also
like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv,
abs/1707.09457, 2017.
51. Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein.
Detoxifying language models risks marginalizing minority voices. arXiv, abs/2104.06390,
2021.
52. Maximilian Witte, Jasper Schwenzow, Mark Heitmann, Martin Reisenbichler, and Matthias
Assenmacher. Potential for Decision Aids based on Natural Language Processing. Working
Paper, 2022.
53. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin,
Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models
to follow instructions with human feedback. arXiv, abs/2203.02155, 2022.
54. Thomas E Currie and Ruth Mace. Political complexity predicts the spread of ethnolinguistic
groups. Proceedings of the National Academy of Sciences, 106(18):7339–7344, 2009.
55. Peter Turchin, Thomas E Currie, Harvey Whitehouse, Pieter François, Kevin Feeney, Daniel
Mullins, Daniel Hoyer, Christina Collins, Stephanie Grohmann, Patrick Savage, et al. Quan-
titative historical analysis uncovers a single dimension of complexity that structures global
variation in human social organization. Proceedings of the National Academy of Sciences,
115(2):E144–E151, 2018.
Hartmann et al. | ChatGPT’s Political Ideology 7
