![Illustrative example of a dataset to which truth discovery can be applied with data sources {s,t,u,v}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{s, t, u, v\}$$\end{document}, facts {f,g,h,i}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{f, g, h, i\}$$\end{document} and objects {o,p}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{o, p\}$$\end{document}](https://www.researchgate.net/publication/362364051/figure/fig1/AS:11431281093132544@1667099497399/Illustrative-example-of-a-dataset-to-which-truth-discovery-can-be-applied-with-data_Q320.jpg)
Access to this full-text is provided by Springer Nature.
Content available from Autonomous Agents and Multi-Agent Systems
This content is subject to copyright. Terms and conditions apply.
Vol.:(0123456789)
Autonomous Agents and Multi-Agent Systems (2022) 36:42
https://doi.org/10.1007/s10458-022-09569-3
1 3
Towards anaxiomatic approach totruth discovery
JosephSingleton1 · RichardBooth1
Accepted: 20 June 2022 / Published online: 30 July 2022
© The Author(s) 2022
Abstract
The problem of truth discovery, i.e., of trying to find the true facts concerning a number of
objects based on reports from various information sources of unknown trustworthiness, has
received increased attention recently. The problem is made interesting by the fact that the
relative believability of facts depends on the trustworthiness of their sources, which in turn
depends on the believability of the facts the sources report. Several algorithms for truth dis-
covery have been proposed, but their evaluation has mainly been performed experimentally
by computing accuracy against large datasets. Furthermore, it is often unclear how these
algorithms behave on an intuitive level. In this paper we take steps towards a framework
for truth discovery which allows comparison and evaluation of algorithms based instead on
their theoretical properties. To do so we pose truth discovery as a social choice problem,
and formulate various axioms that any reasonable algorithm should satisfy. Along the way
we provide an axiomatic characterisation of the baseline ‘Voting’ algorithm—which leads
to an impossibility result showing that a certain combination of the axioms cannot hold
simultaneously—and check which axioms a particular well-known algorithm satisfies. We
find that, surprisingly, our more fundamental axioms do not hold, and propose modifica-
tions to the algorithms to partially fix these problems.
Keywords Truth discovery· Axioms· Trust and reputation· Social choice theory
1 Introduction
There is an increasing amount of data available in today’s world, particularly from the web,
social media platforms and crowdsourcing systems. The openness of such platforms makes
it simple for a wide range of users to share information quickly and easily, potentially
reaching a wide international audience. It is inevitable that amongst this abundance of data
there are conflicts, where data sources disagree on the truth regarding a particular object or
This paper is an extended version of our previous work [35].
* Joseph Singleton
singletonj1@cardiff.ac.uk
Richard Booth
boothr2@cardiff.ac.uk
1 Cardiff University, Cardiff, UK
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 2 of 49
entity. For example, low-quality sources may mistakenly provide erroneous data for topics
on which they lack expertise.
Resolving such conflicts and determining the true facts is therefore an important task.
Truth discovery has emerged as a set of techniques to achieve this by considering the trust-
worthiness of sources [7, 19, 27]. The general principle is that true facts are those claimed
by trustworthy sources, and trustworthy sources are those that claim believable facts.
Application areas include real-time traffic navigation [14], drug side-effect discovery [30],
crowdsourcing and social sensing [29, 38, 47].
For a simple example of a situation where trust can play an important role in conflict
resolution, consider the following example.
Example 1.1 Let o and p represent two images for which crowdsourcing workers are asked
to provide labels (in the truth discovery terminology, o and p are called objects). Consider
workers (the data sources) s,t,u and v who put forward potential labels f,g for o, and h,i
for p, as shown in Fig.1; such potential answers are termed facts. In the graphical repre-
sentation, sources, facts and objects are shown from left to right, and the edges indicate
claims made by sources and the objects to which facts relate.
Without considering trust information, the label for o appears a tie, with both options f
and g receiving one vote from sources s and t respectively.
Taking a trust-aware approach, however, we can look beyond object o to consider the
trustworthiness of s and t. Indeed, when it comes to object p, t agrees with two extra
sources u and v, whereas s disagrees with everyone. In principle there could be hundreds
of extra sources here instead of just two, in which case the effect would be even more strik-
ing. We may conclude that s is less trustworthy than t. Returning to o, we see that g is sup-
ported by a more trustworthy source, and conclude that it should be accepted over f.
Many truth discovery algorithms have been proposed in the literature with a wide range
of techniques used, e.g. iterative heuristic-based methods [17, 34], probabilistic models
[45], maximum likelihood estimation and optimisation-based methods [28], and neural net-
work models [24, 31, 39]. It is common for such algorithms to be evaluated empirically by
running them against real-world or synthetic datasets for which the true facts are already
known; this allows accuracy and other metrics to be calculated, and permits comparison
between algorithms (see [37] for a systematic empirical evaluation of this kind). This may
be accompanied by some theoretical analysis, such as calculating run-time complexity
[19], proving convergence of an iterative algorithm [46], or proving convergence to the
‘true’ facts under certain assumptions on the distribution of source trustworthiness [18, 41,
42].
A limitation of this kind of analysis is that the results only apply narrowly to particu-
lar algorithms, due to the assumptions made (for instance, that claims from sources fol-
low a particular probability distribution). Such assumptions can be problematic in domains
Fig. 1 Illustrative example of a
dataset to which truth discovery
can be applied with data sources
{s,t,u,v}
, facts
{f
,
g
,
h
,
i}
and
objects
{o,p}
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 3 of 49 42
where the desired truth is somewhat ‘fuzzy’; for example, image classification problems
and determining the copyright status of books.1
In this work we take first steps towards a more general approach, in which we aim to
study truth discovery without reference to any specific methodology or probabilistic frame-
work. To do so we note the similarities between truth discovery and problems such as judg-
ment aggregation [15], voting theory [50] ranking and recommendation systems [1–3, 36]
in which the axiomatic approach of social choice has been successfully applied. In tak-
ing the axiomatic approach one aims to formulate axioms that encode intuitively desirable
properties that an algorithm may possess. The interaction between these axioms can then
be studied; typical results include impossibility results, where it is shown that a set of axi-
oms cannot hold simultaneously, and characterisation results, where it is shown that a set
of axioms are uniquely satisfied by a particular algorithm.
Such analysis brings a new normative perspective to the truth discovery literature. This
complements empirical evaluation: in addition to seeing how well an algorithm performs
in practise on test datasets, one can check how well it does against theoretical properties
that any ‘reasonable’ algorithm should satisfy. The satisfaction (or failure) of such proper-
ties then shines new light on the intuitive behaviour of an algorithm, and may guide devel-
opment of new ones.
With this in mind, we develop a simplified framework for truth discovery in which axi-
oms can be formulated, and go on to give both an impossibility result and an axiomatic
characterisation of a baseline voting algorithm. We also analyse the class of recursive truth
discovery algorithms, which includes many existing examples from the literature. In par-
ticular, we analyse the well-known algorithm Sums [34] with respect to the axioms.
However, as a first step towards a social choice perspective of truth discovery, our
framework involves a number of simplifying assumptions not commonly made in the truth
discovery literature.
• Manipulation and collusion. Some of our axioms assume sources are not manipu-
lative: they provide claims in good faith, and do not aim to misinform or artificially
improve their standing with respect to the truth discovery algorithm. We also assume
sources act independently, i.e. they do not collude with or copy one another.
• Object correlations. We do not model correlations between the objects of interest in
the truth discovery problem. For example, in a crowdsourcing setting it may be known
in advance that two objects o and p are similar, so that the true labels for o and p are
correlated; this cannot be expressed in our framework.
• Ordinal outputs. For the most part, the outputs of our truth discovery methods consist
of rankings of the sources and facts. Thus, we describe when a source is considered
more trustworthy than another, but do not assign precise numerical values represent-
ing trustworthiness. This breaks with tradition in the truth discovery literature, but is a
common point of view in social choice theory.
At first glance these are strong assumptions, and rule out potential applications of our ver-
sion of truth discovery. However, we argue that the problem is non-trivial even in this sim-
plified setting, and that interesting axioms can still be put forth. The framework as set out
here lays the groundwork for these assumptions to be lifted in future work.
1 https:// www. nytim es. com/ 2019/ 08/ 19/ techn ology/ amazon- orwell- 1984. html
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 4 of 49
The paper is organised as follows. Our framework is introduced and formally defined in
the next section. Section3 provides examples of truth discovery algorithms from the litera-
ture expressed in the framework. In Sect.4 we formally introduce the axioms and present
an impossibility result showing a subset of these cannot all be satisfied simultaneously. The
examples of Sect.3 are then revisited in Sect.5, where we analyse them with respect to the
axioms and propose modifications to resolve some axiom failures. In Sect.6 we extend the
framework to allow variable domains of sources, facts and objects, and give an impossibil-
ity result similar to that of Sect.4. We discuss the axioms in Sect.7 and related work in
Sect.8. We conclude in Sect.9. Missing proofs are given in appendix A.
2 An idealised framework fortruth discovery
In this section we define our formal framework, which provides the key definitions required
for theoretical discussion and analysis of truth discovery methods.
For most of the paper, we consider a fixed domain of finite and mutually disjoint sets
S
,
F
and
O
throughout, called the sources, facts and objects respectively. All definitions and
axioms will be stated with respect to these sets.2
2.1 Truth discovery networks
A core definition of the framework is that of a truth discovery network, which represents
the input to a truth discovery problem. We model this as a tripartite graph with certain con-
straints on its structure, in keeping with approaches taken throughout the truth discovery
literature [19, 45].
Definition 2.1 A truth discovery network (hereafter a TD network) is a directed graph
N=(V,E)
where
V=S∪F∪O
, and
E⊆(S×F)∪(F×O)
has the following properties:
1. For each
f∈F
there is a unique
o∈O
with
(f,o)∈E
, denoted
𝗈𝖻𝗃N(f)
. That is, each
fact is associated with exactly one object.
2. For
s∈S
and
o∈O
, there is at most one directed path from s to o. That is, sources
cannot claim multiple facts for a single object.
3.
(S×F)∩E
is non-empty. That is, at least one claim is made.
We will say that s claims f when
(s,f)∈E
. Let
N
denote the set of all TD networks.
Figure1 (page 2) provides an example of a TD network. Note that there is no require-
ment that a source makes a claim for every object, or even that a source makes any claims
at all. This reflects the fact that truth discovery datasets are in practise extremely sparse,
i.e. each individual source makes few claims. Conversely, we allow for facts that receive no
claims from any sources.
Also note that the object associated with a fact f is not fixed across all networks. This
is because we view facts as labels for information that sources may claim, not the pieces
of information themselves. Similarly, we consider objects simply as labels for real-world
2 We generalise to variable domains in Sect.6.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 5 of 49 42
entities. Whilst a particular piece of information has a fixed entity to which it pertains, the
labels do not.3
A special case of our framework is the binary case in which every object has exactly two
associated facts. This brings us close to the setting studied in judgment aggregation [15]
and, specifically (since sources do not necessarily claim a fact associated to every object)
to the setting of binary aggregation with abstentions [9, 11]. An important difference, how-
ever, is that for simplicity we do not assume any constraints on the possible configurations
of true facts across different objects. That is, any combination of facts is feasible. In judg-
ment aggregation such an assumption has the effect of neutralising the impossibility results
that arise in that domain (see, e.g., [9]). We shall see that that is not the case in our setting.
To simplify the notation in what follows, for a network
N=(V,E)
we write
𝖿 𝖺𝖼𝗍𝗌N(s)={f∈F∶(s,f)∈E}
for the set of facts claimed by a source s, and
𝗌𝗋𝖼N(f)={s∈S∶(s,f)∈E}
for the set of sources claiming a fact f.
2.2 Truth discovery operators
Having defined the input to a truth discovery problem, the output must be defined. Contrary
to many approaches in the truth discovery literature which output numeric trust scores for
sources and belief scores for facts [17, 34, 45, 47–49], we consider the primary output to
be rankings of the sources and facts. To the extent that we do consider numeric scores, it
is only to induce a ranking. This is because we are chiefly interested in ordinal properties
rather than quantitative values. Indeed, for the theoretical analysis we wish to perform it
is only important that a source is more trustworthy than another; the particular numeric
scores produced by an algorithm are irrelevant.
Moreover, the scores produced by existing algorithms may have no semantic meaning
[34], and so referring to numeric values is not meaningful when comparing across algo-
rithms. In this case it is only the rankings of sources and facts that can be compared, which
is further motivation for our choice. This point of view is also common across the social
choice literature.
However, numerical scores do provide valuable information for comparing sources and
facts given a fixed algorithm. For example, the magnitude of the difference in trust scores
for sources s and t tells us something about confidence: a small difference indicates low
confidence in distinguishing s and t—even if one is ranked above the other—whereas a
large difference indicates high confidence. In this sense our decision to primarily deal with
ordinal outputs (and ordinal axioms) is another simplifying assumption compared to typi-
cal truth discovery settings.
For a set X, let
L(X)
denote the set of all total preorders on X, i.e. the set of transitive,
reflexive and complete binary relations on X. We define a truth discovery operator as a
function which maps networks to rankings of sources and facts.
Definition 2.2 An ordinal truth discovery operator T (hereafter TD operator) is a mapping
T∶N→L(S)×L(F)
. We shall write
T
(N)=(⊑
T
N
,⪯
T
N)
, i.e.
⊑T
N
is a total preorder on
S
and
⪯T
N
is a total preorder on
F
.
3 For example, when implementing truth discovery algorithms in practise it is common to assign integer
IDs to the ‘facts’ and ‘objects’; the algorithm then operates using only the integer IDs. In this case there
is no reason to require that fact 17 is always associated with object 4, for example, and the same principle
applies in our framework.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 6 of 49
Intuitively, the relation
⊑T
N
is a measure of source trustworthiness in the network N
according to T, and
⪯T
N
is a measure of fact believability;
s1
⊑
T
N
s
2
means that source
s2
is at
least as trustworthy as source
s1
, and f
1⪯T
N
f
2
means fact
f2
is at least as believable as fact
f1
. The notation
⊏T
N
and
≃T
N
will be used to denote the strict and symmetric orders induced
by
⊑T
N
respectively. For fact rankings,
≺T
N
and
≈T
N
are defined similarly. Note that for sim-
plicity the fact ranking
⪯T
N
compares all facts, even those associated with different objects
in N.
To capture existing truth discovery methods we introduce numerical operators, which
assign each source a numeric trust score and each fact a belief score.
Definition 2.3 A numerical TD operator is a mapping
T∶N→ℝS∪F
, i.e. T assigns to
each TD network N a function
T(N)=TN∶S∪F→ℝ
. For
s∈S
,
TN(s)
is the trust score
for s in the network N according to T; for
f∈F
,
TN(f)
is the belief score for f. The set of
all numerical TD operators will be denoted by
TNum
.
Note that any numerical operator T naturally induces an ordinal operator
T
, where
s1
⊑
T
N
s
2
iff
TN(s1)
≤
TN(s2)
, and f
1
⪯
T
N
f
2
iff
TN(f1)
≤
TN(f2)
. Henceforth we shall write
⊑T
N
,
⪯T
N
without explicitly defining the induced ordinal operator
T
.
It is worth noting that yet other truth discovery algorithms output neither rankings nor
numeric scores for facts, but only a single ‘true’ fact for each object [10, 28, 43]. This is
also the approach taken in judgment aggregation, where an aggregation rule selects which
formulas are to be taken as true. In the case of finitely many possible facts, such algorithms
can be modelled in our framework as numerical operators where
TN(f)=1
for each identi-
fied ‘true’ fact f, and
TN(g)=0
for other facts g. To go in the reverse direction and obtain
the ‘true’ facts according to an operator, one may simply select the set of facts for each
object that rank maximally.
3 Examples oftruth discovery operators
Our framework can capture some operators that have been proposed in the truth discov-
ery literature. In this section we provide two concrete examples: Voting, which is a simple
approach commonly used as a baseline method, and Sums [34]. We go on to outline the
class of recursive operators—of which Sums is an instance—which contains many more
examples from the literature.
3.1 Voting
In Voting, we consider each source to cast ‘votes’ for the facts they claim, and facts are
ranked according to the number of votes received. Clearly this method disregards the
source trustworthiness aspect of truth discovery, as a vote from one source carries as much
weight as a vote from any other. As such, Voting cannot be considered a serious contender
for truth discovery. It is nonetheless useful as a simple baseline method against which to
compare more sophisticated methods.
Definition 3.1 Voting is the numerical operator defined as follows: for any network
N∈N
,
s∈S
and
f∈F
,
TN(s)=1
and
TN(f)=|𝗌𝗋𝖼N(f)|
.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 7 of 49 42
Consider the network N shown in Fig.1. Facts f,g and h each receive one vote, whereas
i receives 3. The fact ranking induced by Voting is therefore
f≈g≈h≺i
. On the other
hand, all sources receive a trust score of 1 and therefore rank equally.
3.2 Sums
Sums [34] is a simple and well-known operator adapted from the Hubs and Authorities
[21] algorithm for ranking web pages. The algorithm operates iteratively and recursively,
assigning each source and fact a sequences of scores, with the final scores taken as the limit
of the sequence.
Initially, scores are fixed at a constant value of 1/2. The trust score for each source is
then updated by summing the belief score of its associated facts. Similarly, belief scores
are updated by summing the trust scores of the associated sources. To prevent these scores
from growing without bound as the algorithm iterates, they are normalised at each iteration
by dividing each trust score by the maximum across all sources (belief scores are normal-
ised similarly).
Expressed in our framework, we have that if T is the (numerical) operator giving the
scores at iteration n, then the pre-normalisation scores at iteration
n+1
are given by
T′
,
where
Consider again the network N shown in Fig.1. It can be shown that, with T denoting
the limiting scores from Sums with normalisation, we have
TN(s)=0
,
TN(t)=1
, and
TN(u)=TN(v)=√2∕2
. The induced ranking of sources is therefore
s⊏u≃v⊏t
.
For fact scores, we have
TN(f)=0
,
TN
(g)=
√
2−
1
,
TN(h)=0
and
TN(i)=1
, so the
ranking is
f≈h≺g≺i
. Note that fact g fares better under Sums than Voting, due to its
association with the highly-trusted source t.
3.3 Recursive truth discovery operators
The iterative and recursive aspect of Sums is hoped to result in the desired mutual depend-
ence between trust and belief scores: namely that sources claiming high-belief facts are
seen as trustworthy, and vice versa. In fact, this recursive approach is near universal across
the truth discovery literature (see for instance [14, 17, 28, 44, 48, 49]). As such it is appro-
priate to identify the class of recursive operators as an important subset of
TNum
. To make a
formal definition we first define an iterative operator.
Definition 3.2 An iterative operator is a sequence
(Tn)n∈ℕ
of numerical operators. An iter-
ative operator is said to converge to a numerical operator
T∗
if
limn→∞Tn
N(z)=T∗
N(z)
for
all networks N and
z∈S∪F
. In such case the iterative operator can be identified with the
ordinal operator induced by its limit
T∗
.
Note that it is possible that an iterative operator
(Tn)n∈ℕ
converges for only a sub-
set of networks. In such case we can consider
(Tn)n∈ℕ
to converge to a ‘partial
(3.1)
T
�
N(s)=
∑
f∈𝖿𝖺𝖼𝗍𝗌
N
(s)
TN(f);T�
N(f)=
∑
s∈𝗌𝗋𝖼
N
(f)
T�
N(s
)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 8 of 49
operator’ and identify it with the induced partial ordinal operator; that is, a partial function
N→L(S)×L(F)
. Recursive operators can now be defined as those iterative operators
where
Tn+1
can be obtained from
Tn
.
Definition 3.3 An iterative operator
(Tn)n∈ℕ
is said to be recursive if there is a function
U∶TNum
→
TNum
such that
Tn+1=U(Tn)
for all
n∈ℕ
.
In this context the mapping
U∶TNum
→
TNum
is called the update function, and the ini-
tial operator
T1
is called the prior operator. For a prior operator T and update function U,
we write
𝗋𝖾𝖼(T,U)
for the associated recursive operator; that is,
T1=T
and
Tn+1=U(Tn)
.
Returning to Sums, we see that Eq. (3.1) defines a mapping
TNum →TNum
and conse-
quently an update function
USums
. The normalisation step can be considered a separate
update function
𝗇𝗈𝗋𝗆
which maps any numerical operator T to
T′
, where4
It can then be seen that Sums is the recursive operator
𝗋𝖾𝖼(Tfixed ,𝗇𝗈𝗋𝗆
◦
USums)
, where
Tfixed
N
≡1∕
2
.
Many other existing algorithms proposed in the literature can also be realised as recur-
sive operators in the framework, such as Investment, PooledInvestment [34], TruthFinder
[45], LDT [48] and others.
4 Axioms fortruth discovery
Having laid out the formal framework, we now introduce axioms for truth discovery. To
start with, we consider axioms which encode a desirable theoretical property that we
believe any ‘reasonable’ operator T should satisfy. Several properties of this nature can be
obtained by adapting existing axioms from the social choice literature (e.g. from voting [8],
ranking systems [2, 36] and judgement aggregation [15]), to our framework.
However, the correspondence between truth discovery and classical social choice prob-
lems—such as voting—has its limits. To show this, we translate the famous Independence
of Irrelevant Alternatives (IIA) axiom [4] to our setting, and argue that it is actually an
undesirable property. Indeed, it will be seen that this translated axiom, in combination
with two basic desirable axioms, leads to Voting-like behaviour in every network, which
is undesirable for the reasons given in Sect.3.1. Furthermore, a slight strengthening of the
IIA axiom completely characterises the fact ranking component of Voting. These results
formalise the intuition that truth discovery’s consideration of source-trustworthiness leads
to fundamental differences from classical social choice problems.
Afterwards, we will revisit the specific operators of the previous section to check which
axioms are satisfied.
T
�
N(s)=
T
N
(s)
max
x∈
S
|
TN(x)
|
,T�
N(f)=
T
N
(f)
max
y∈
F
|
TN(y)
|
4 If
maxx∈S|TN(x)|=0
then the above is ill-defined; we set
T�
N
(s)=
0
for all s in this case. Fact belief
scores are defined similarly if the maximum is 0.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 9 of 49 42
4.1 Coherence
As mentioned previously, a guiding principle of truth discovery is that sources claiming
highly believed facts should be seen as trustworthy, and that facts backed by highly trusted
sources should be seen as believable.
Whilst this intuition is difficult to formalise in general, it is possible to do so in par-
ticular cases where there are obvious means by which to compare the set of facts for two
sources (and vice versa). This situation is considered in the axiomatic analysis of ranking
and reputation systems under the name Transitivity [2, 36], and we adapt it to truth discov-
ery in this section. A preliminary definition is required.
Definition 4.1 Let T be a TD operator, N be a TD network and
Y,Y′⊆F
. We say Y is
less believable than
Y′
with respect to N and T if there is a bijection
𝜑∶Y→Y�
such that
f⪯
T
N
𝜑(f
)
for each
f∈Y
, and
f≺
T
N
𝜑(
f
)
for some
f∈Y
.
For
X,X′⊆S
we define X less trustworthy than
X′
with respect to N and T in a similar
way.
In plain English, Y less believable than
Y′
means that the facts in each set can be paired
up in such a way that each fact in
Y′
is at least as believable as its counterpart in Y, and at
least one fact in
Y′
is strictly more believable. Now, consider a situation where
𝖿 𝖺𝖼𝗍𝗌N(s1)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
. In this case the intuition outlined above tells us that
s2
provides ‘better’ facts, and should thus be seen as more trustworthy than
s1
. A similar idea
holds if
𝗌𝗋𝖼N(f1)
is less trustworthy than
𝗌𝗋𝖼N(f2)
for some facts
f1,f2
. We state this formally
as our first axiom.
Axiom 1 (Coherence) For any network N,
𝖿 𝖺𝖼𝗍𝗌N(s1)
less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
implies
s1
⊏
T
N
s
2
, and
𝗌𝗋𝖼N(f1)
less trustworthy than
𝗌𝗋𝖼N(f2)
implies f
1
≺
T
N
f
2
.
Coherence can be broken down into two sub-axioms: Source-Coherence, where the first
implication regarding source rankings is satisfied; and Fact-Coherence, where the second
implication is satisfied. We take Coherence to be a fundamental desirable axiom for TD
operators.
4.2 Symmetry
Our next axiom requires that rankings of sources and facts should not depend on their
‘names’, but only on the structure of the network. To state it formally, we need a notion of
when two networks are essentially the same but use different names.
Definition 4.2 Two TD networks N and
N′
are equivalent if there is a graph isomorphism
𝜋
between them that preserves sources, facts and objects, i.e.,
𝜋(s)∈S
,
𝜋(f)∈F
and
𝜋(o)∈O
for all
s∈S
,
f∈F
and
o∈O
. In such case we write
𝜋(N)
for
N′
.
Axiom 2 (Symmetry) Let N and
N�=𝜋(N)
be equivalent networks. Then for all
s1,s2∈S
,
f1,f2∈F
, we have
s1
⊑
T
N
s
2
iff 𝜋(s
1
)⊑
T
N
�𝜋(s
2)
and f
1
⪯
T
N
f
2
iff 𝜋(f
1
)⪯
T
N
�𝜋(f
2)
.
In the theory of voting in social choice, Symmetry as above is expressed as two axioms:
Anonymity, where output is insensitive to the names of voters, and Neutrality, where output
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 10 of 49
is insensitive to the names of alternatives [50]. Analogous axioms are also used in judg-
ment aggregation.
Symmetry can also be broken down into sub-axioms where the above need only hold
for a subset of permutations
𝜋
satisfying some condition: Source-Symmetry (where
𝜋
must
leave facts and objects fixed) and Fact-Symmetry (where
𝜋
leaves sources and objects
fixed). For truth discovery we have the additional notion of objects, and thus Object-Sym-
metry can defined be similarly.
4.3 Fact ranking axioms
Next, we introduce axioms that dictate the ranking of particular facts in cases where there
is an ‘obvious’ ordering. Unanimity and Groundedness express the idea that if all sources
are in agreement about the status of a fact, then an operator should respect this in its ver-
dict. Two obvious ways in which sources can be in agreement are when all sources believe
a fact is true, and when none believe a fact is true.
Axiom 3 (Unanimity) Suppose
N∈N
,
f∈F
, and
𝗌𝗋𝖼N(f)=S
. Then for any other
g∈F
,
g
⪯
T
N
f.
Axiom 4 (Groundedness) Suppose
N∈N
,
f∈F
, and
𝗌𝗋𝖼N(
f
)=�
. Then for any other
g∈F
, f⪯
T
Ng
.
That is, f cannot do better than to be claimed by all sources when T satisfies Unanimity,
and cannot do worse than to be claimed by none when T satisfies Groundedness. Unanim-
ity here is a truth discovery rendition of the same axiom in judgment aggregation, and can
also be compared to the weak Paretian property in voting [8]. Groundedness is a version of
the same axiom studied in the analysis of collective annotation [25].
The next axiom is a monotonicity property, which states that if f receives extra support
from a new source s, then its ranking should receive a strictly positive boost.5 Note that we
do not make any judgement on the new ranking of s.
Axiom 5 (Monotonicity) Suppose
N∈N
,
s∈S
,
f∈F⧵𝖿 𝖺𝖼𝗍𝗌N(s)
. Write E for the set
of edges in N, and let
N′
be the network in which s claims f; i.e. the network with edge set
Then for all
g≠f
,
g
⪯
T
N
fimplies g≺
T
N
�f.
Note that the axioms in this section assume sources do not have ‘negative’ trust levels.
That is, we assume that support from even the most untrustworthy source still has a posi-
tive effect on the believability of a fact. Consequently, these axioms are not suitable in the
presence of knowledgable but malicious sources who always claim false facts. Indeed, oth-
erwise a fact claimed only by a ‘negative’ source should rank strictly worse than a fact with
no sources, but this goes against Groundedness. Similarly, receiving extra support from
E�= {(
s,f
)} ∪
E⧵
{(
s,g
)∶
g≠f,𝗈𝖻𝗃
N(
g
)=
𝗈𝖻𝗃
N(
f
)}
5 One could also consider the weak version, in which we only require
g
⪯
T
N�
f in the consequent; we discuss
this in Sect.7.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 11 of 49 42
a negative source should worsen a fact’s ranking, contrary to Monotonicity. Moreover,
Monotonicity implicitly assumes sources act independently, i.e. they do not collude with
one another.6
While these assumptions may appear somewhat strong, we argue that this ‘simple’
case—with no ‘negative’ sources or collusion—is already non-trivial and permits interest-
ing axiomatic analysis. We therefore view Unanimity, Groundedness and Monotonicity as
desirable properties for TD operators.
4.4 Independence axioms
We now come to exploring the differences between truth discovery and other social choice
problems via independence axioms. In voting, this takes the form of Independence of
Irrelevant Alternatives (IIA), which requires that the ranking of two alternatives A and B
depends only on the individual assessments of A and B, not on some ‘irrelevant’ alternative
C.
An analogous truth discovery axiom states that the ranking of facts
f1
and
f2
for some
object o depends only on the claims relating to o. Intuitively, this is not a desirable prop-
erty. Indeed, we have already seen in example 1.1 that the claims for object p in the net-
work from Fig.1 can play an important role in determining the ranking of f and g for object
o, but the adapted IIA axiom precludes this.
This undesirability can be made precise. First, we must state the axiom formally.
Axiom 6 (Per-object Independence (POI)) Let
o∈O
. Suppose
N1
,
N2
are networks such
that
F
o=𝗈𝖻𝗃
−1
N1
(o)=𝗈𝖻𝗃
−1
N2
(o
)
and
𝗌𝗋𝖼N1(f)=𝗌𝗋𝖼N2(f)
for each
f∈Fo
. Then the restrictions
of
⪯T
N1
and
⪯T
N2
to
Fo
are equal; that is, f1⪯
T
N1
f2iff f1⪯
T
N2
f
2
for all
f1,f2∈Fo
.
Considering Fig.1 again, POI implies that the ranking of f and g remains the same if
the claims for h and i are removed. But in this case, Symmetry implies
f≈g
. Similarly,
the ranking of h and i remains the same if the claims for f and g are removed. In this case,
Symmetry together with Monotonicity implies
h≺i
, since
|𝗌𝗋𝖼N(h)|<|𝗌𝗋𝖼N(i)|
.
This observation forms the basis of the following result, which formalises the undesir-
ability of POI: in the presence of our less controversial requirements of Symmetry and
Monotonicity, it forces Voting-like behaviour within
𝗈𝖻𝗃−1
N
(o
)
for each
o∈O
. We note that,
for the special case of binary networks, similar results have been shown in the literature on
binary aggregation with abstentions [9].
Theorem4.1 Let T be any operator satisfying Symmetry, Monotonicity and POI. Then for
any
N∈N
,
o∈O
and f
1
,f
2
∈𝗈𝖻𝗃
−1
N
(o
)
we have f
1⪯T
N
f
2
iff
|𝗌𝗋𝖼N(f1)|≤|𝗌𝗋𝖼N(f2)|
.
Proof (sketch) We will sketch the main ideas of the proof here with some technical
details omitted; see appendix A for the full proof. Let N be a network, o be an object and
f
1
,f
2
∈𝗈𝖻𝗃
−1
N
(o
)
. Consider
N′
obtained by removing from N all claims for objects other
than o. By POI, we have f
1
⪯
T
N
f
2
iff f
1
⪯
T
N�
f
2
. Since
|𝗌𝗋𝖼N(fj)|=|𝗌𝗋𝖼N
�
(fj)|
also (
j∈{1, 2}
),
it is sufficient for the proof to show that f
1⪯T
N
�f
2
iff
|𝗌𝗋𝖼N
�
(f1)|≤|𝗌𝗋𝖼N
�
(f2)|
.
6 Note that collusion has been studied in the truth discovery literature (e.g. [6, 12, 13]).
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 12 of 49
For the ‘if’ direction, first suppose
|𝗌𝗋𝖼N
�
(f1)|=|𝗌𝗋𝖼N
�
(f2)|
. Let
𝜋
be the permutation
which swaps
f1
with
f2
and swaps each source in
𝗌𝗋𝖼N
�
(f1)
with one in
𝗌𝗋𝖼N
�
(f2)
; then we
have
𝜋(N�)=N�
, and Symmetry of T gives f
1≈T
N
�f
2
. In particular f
1⪯T
N
�f
2
as required.
Otherwise,
|𝗌𝗋𝖼N
�
(f2)|−|𝗌𝗋𝖼N
�
(f1)|=k>0
. Consider
N′′
where k sources from
𝗌𝗋𝖼N
�
(f2)
are removed, and all other claims remain. By Symmetry as above, f
1≈T
N
�� f
2
. Applying
Monotonicity k times we can produce
N′
from
N′′
and get f
1
≺
T
N′
f
2
as desired.
For the ‘only if’ statement, suppose f
1⪯T
N
�f
2
but, for contradiction,
|𝗌𝗋𝖼N
�
(f1)|>|𝗌𝗋𝖼N
�
(f2)|
. Applying Monotonicity again as above we get f
1
≻
T
N′
f
2
and the
required contradiction.
◻
Recall that Coherence formalises the idea that source-trustworthiness should inform
the fact ranking, and vice versa. Clearly Voting does not conform to this idea, and in
fact even the object-wise voting patterns in Theorem 4.1 are incompatible with Coher-
ence. This can easily be seen in the network in Fig.1 where, regarding object p, we have
|𝗌𝗋𝖼N(h)|<|𝗌𝗋𝖼N(i)|
(hence
h
≺
T
Ni
) and, regarding object o, we have
|𝗌𝗋𝖼N(f)|=|𝗌𝗋𝖼N(g)|
(hence f≈
T
Ng
). Hence
𝖿 𝖺𝖼𝗍𝗌N(s)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(t)
. If Coherence held this
would give
s
⊏
T
Nt
, but then
𝗌𝗋𝖼N(f)
is less trustworthy than
𝗌𝗋𝖼N(g)
, giving f≺
T
Ng
—a con-
tradiction. From this discussion and Theorem4.1 we obtain as a corollary the following
first impossibility result for truth discovery.
Theorem4.2 There is no TD operator satisfying Coherence, Symmetry, Monotonicity and
POI.
Given that Theorem4.1 characterises the fact ranking of Voting for facts relating to a
single object, it is natural to ask if there is a stronger form of independence that guarantees
this behaviour across all facts. As our next result shows, the answer is yes, and the neces-
sary axiom is obtained by ignoring the role of objects altogether for fact ranking.
Axiom 7 (Strong Independence) For any networks
N1,N2
and facts
f1,f2
, if
𝗌𝗋𝖼N1(fj)=𝗌𝗋𝖼N2(fj)
for each
j∈{1, 2}
then f1
⪯T
N
1
f
2
iff f1
⪯T
N
2
f
2
.
That is, the ranking of two facts
f1
and
f2
is determined solely by the sources claiming
f1
and
f2
. In particular, the fact-object affiliations and claims for facts other than
f1,f2
are
irrelevant when deciding on
f1
versus
f2
. Note that Strong Independence implies POI. We
have the following result.
Theorem4.3 Suppose
|O|
≥
3
. Then an operator T satisfies Strong Independence, Monoto-
nicity and Symmetry if and only if for any network N and
f1,f2∈F
we have
Theorem 4.3 can be seen as a characterisation of the class of TD operators that rank
facts in the same way as Voting. The proof is similar to that of Theorem4.1, but uses a dif-
ferent transformation to obtain a modified network
N′
in the first step.
We have established that neither POI nor Strong Independence are satisfactory axioms
for truth discovery, and a weaker independence property is required. Figure1 can help us
once again in this regard. Whereas POI and Strong Independence would say that facts h
and i are irrelevant to f, the argument with Coherence for Theorem4.2 suggests otherwise
due the indirect links via the sources. We therefore propose that only when there is no
f
1
⪯
T
N
f
2
⟺
|
𝗌𝗋𝖼
N
(f
1
)
|
≤
|
𝗌𝗋𝖼
N
(f
2
)
|
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 13 of 49 42
(undirected) path between two nodes can we consider them to be truly irrelevant to each
other. That is, nodes are relevant to each other iff they lie in the same connected component
of the network.
Our final rendering of independence states that the ordering of two facts in the same
connected component does not depend on any claims outside of the component, and simi-
larly for sources.
Axiom 8 (Per‑component Independence (PCI)) For any TD networks
N1
,
N2
with a com-
mon connected component G, the restrictions of
⊑T
N1
and
⊑T
N2
to
G∩S
are equal, and the
restrictions of
⪯T
N1
and
⪯T
N2
to
G∩F
are equal; that is,
s
1⊑
T
N1
s2iff s1⊑
T
N2
s
2
and
f1
⪯T
N
1
f2iff f1
⪯T
N
2
f
2
for
s1,s2∈G∩S
and
f1,f2∈G∩F
.
In analogy with Source/Fact Coherence and Source/Fact Symmetry, it is possible to
split the two requirements of PCI into sub-axioms Source-PCI (in which only the constraint
on source ranking is imposed) and Fact-PCI (in which only the fact ranking is constrained).
Note that while our framework can be easily adapted to require by definition that a net-
work is itself connected (and therefore has only one connected component), we have found
that datasets with multiple connected components do indeed occur in practise.7 This means
that failure of PCI is a real issue, and consequently we consider PCI to be another core
axiom that all reasonable operators should satisfy.
5 Satisfaction oftheaxioms
With the axioms formally defined, we can now consider whether they are satisfied by the
example operators of Sect.3. Voting can be analysed outright; for Sums we require some
preliminary results giving sufficient conditions for iterative and recursive operators to sat-
isfy various axioms. It will be seen that neither Voting nor Sums satisfy all our desirable
axioms, but it is possible to modify each operator to gain some improvement with respect
to the axioms.
5.1 Voting
As the simplest operator, we consider Voting first. The following theorem shows that all
axioms except Coherence are satisfied. Since Coherence is a fundamental principle of truth
discovery, and we actually consider POI and Strong Independence to be undesirable, this
formally rules out Voting as a viable operator.
Theorem 5.1 Voting satisfies Symmetry, Unanimity, Groundedness, Monotonicity, POI,
Strong Independence and PCI. Voting does not satisfy Coherence.
7 For example, the Book and Restaurant datasets found at the following web page each contain two con-
nected components: http:// lunad ong. com/ fusio nData Sets. htm
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 14 of 49
The proof is straightforward, and is deferred to appendix A. Note that once Symmetry,
Monotonicity and POI are shown, the fact that Voting fails Coherence follows from our
impossibility result (Theorem4.2), and Fig.1 serves as an explicit counterexample.
5.2 Iterative andrecursive operators
In this section we give sufficient conditions for iterative and recursive operators to sat-
isfy various axioms. These results will be useful in what follows when analysing Sums,
although they may also be applied more generally to other operators.
Coherence. To analyse whether the limit of a recursive operator satisfies Coherence, we
consider how the update function U behaves when the difference in belief scores between
the facts of
s1
and
s2
is ‘small’ (and similarly for the sources of
f1
,
f2
). To that end, we
introduce a numerical variant of a set of facts Y being ‘less believable’ than
Y′
.
Definition 5.1 Let T be a numerical TD operator, N a network,
Y,Y′⊆F
and
𝜀,𝜌>0
.
We say Y is
(𝜀,𝜌)
-less believable than
Y′
with respect to N and T if there is a bijection
𝜑∶Y→Y�
such that
TN(f)−TN(𝜑(f))
≤
𝜀
for all
f∈Y
, and
TN(
f)−TN(𝜑(
f))
≤
𝜀−𝜌
for
some
f∈Y
.
For
X,X′⊆S
, we define X
(𝜀,𝜌)
-less trustworthy than
X′
similarly.
This generalises definition 4.1 by relaxing the requirement that f⪯
T
N
𝜑(f
)
, and instead
requiring that f can only be more believable than
𝜑(f)
by some threshold
𝜀>0
. Definition
4.1 is recovered in the limiting case
𝜀→0
. We obtain a sufficient condition on the update
function U for a recursive operator to satisfy Source-Coherence.
Lemma 5.1 Let
U∶TNum
→
TNum
. For any prior operator
Tprior
,
𝗋𝖾𝖼(Tprior,U)
satisfies
Source-Coherence if the following condition is satisfied: there exist
C,D>0
such that for
all networks N and numerical operators T it holds that if
𝖿 𝖺𝖼𝗍𝗌N(s1)
is
(𝜀,𝜌)
-less believable
than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to N and T, then
T�
N(s1)−T�
N(s2)
≤
C𝜀−D𝜌
, where
T�=U(T)
.
The proof of Lemma 5.1 uses the following result, the proof of which is a straightfor-
ward application of the definition of the limit.
Lemma 5.2 Let N be a truth discovery network and
(Tn)n∈ℕ
be a convergent iterative oper-
ator with limit
T∗
. Then for
f1,f2∈F
, f
1
⪯
T∗
N
f
2
if and only if
Also, f
1
≺
T∗
N
f
2
if and only if
Analogous statements for source rankings also hold.
Proof (Lemma 5.1) Let N be a network. Suppose U has the stated property and that
𝗋𝖾𝖼
(T
prior
,U)=(T
n
)
n∈ℕ
converges to
T∗
. Suppose
𝖿 𝖺𝖼𝗍𝗌N(s1)
is less trustworthy than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to N and
T∗
under a bijection
𝜑
. We must show that
s1
⊏
T∗
N
s
2
.
∀𝜀>0∃K∈ℕ∶∀n≥K∶Tn
N(f1)−Tn
N(f2)≤𝜀
∃𝜌>0∶∀𝜀>0∃K∈ℕ∶∀n≥K∶Tn
N(f1)−Tn
N(f2)≤𝜀−𝜌
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 15 of 49 42
Now, there is some
f∈𝖿 𝖺𝖼𝗍𝗌N(s1)
with
f≺T
∗
N
𝜑(
f
)
. The second part of Lemma
5.2 therefore applies; let
𝜌
be as given there. Now let
𝜀>0
. Since f⪯
T∗
N
𝜑(f
)
for each
f∈𝖿 𝖺𝖼𝗍𝗌N(s1)
, we may apply Lemma 5.2 with
f,𝜑(f)
and
𝜀 =𝜀∕C
to get that there is
K∈ℕ
such that
and
for all
n≥K
. In other words,
𝖿 𝖺𝖼𝗍𝗌N(s1)
is
(𝜀 ,𝜌)
-less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect
to N and
Tn
for all
n≥K
.
Now, recall that
Tn+1=U(Tn)
. For
m≥K�=K+1
we therefore have, applying our
condition on U,
Since
D𝜌
is positive and does not depend on
𝜀
, we get
s1
⊏
T∗
N
s
2
by Lemma 5.2. This shows
that
T∗
satisfies Source-Coherence.
◻
A similar result gives conditions under which Fact-Coherence is satisfied.
Lemma 5.3
𝗋𝖾𝖼(Tprior,U)
satisfies Fact-Coherence if there exist
E,F>0
such that for all
networks N and numerical operators T it holds that if
𝗌𝗋𝖼N(f1)
is
(𝜀,𝜌)
-less trustworthy than
𝗌𝗋𝖼N(f2)
with respect to N and
T′
, then
T�
N(f1)−T�
N(f2)
≤
E
𝜀
−F𝜌
, where
T�=U(T)
.
Proof The proof proceeds in an identical way to Lemma 5.1; the only difference is that we
may simply take
K�=K
in the final step.
◻
Note that there is asymmetry between Lemmas 5.1 and 5.3—in the condition on U in
Lemma 5.1 we have
𝖿 𝖺𝖼𝗍𝗌N(s1)
(𝜀,𝜌)
-less trustworthy than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to T,
whereas in Lemma 5.3 the corresponding condition is with respect to
T�=U(T)
. This
reflects the manner in which Sums and other TD operators are typically defined: source
trust scores are updated based on the fact scores of the previous iteration, whereas fact
belief scores are updated based on the (new) trust scores in the current iteration.
Also note that the above results still hold if U has the stated property only for ‘small’
𝜀
; that is, if there is a constant
0<𝜆<1
such that the property holds for all
𝜌
and for all
𝜀 < 𝜆𝜌
.
Symmetry and PCI. When considering either Symmetry or PCI for an iterative opera-
tor
(Tn)n∈ℕ
, it is not enough to know that each
Tn
satisfies the relevant axiom. The fol-
lowing example illustrates this fact for Symmetry.
Example 5.1 Fix some
f∈
F , and define an iterative operator by
Tn
N(f)−Tn
N(𝜑(f)) ≤𝜀
Tn
N
(
f)−T
n
N
(𝜑(
f)) ≤𝜀 −
𝜌
Tm
N(s1)−Tm
N(s2)≤C𝜀 −D𝜌=𝜀−D𝜌
Tn
N
(
s
)=
1
T
n
N(f)=
{|
𝗌𝗋𝖼N(f)
|
+(1−1
n+1)if
|
𝗌𝗋𝖼N(f)
|
=
|
𝗌𝗋𝖼N(
f)
|
|𝗌𝗋𝖼N(f)|otherwise
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 16 of 49
That is, each
Tn
is a modification of Voting in which we boost the score of all facts tied
with
f
under Voting by
1
−
1
n+1
. Since this additional weight is (strictly) less than 1 for each
n, the ordinal operator induced by
Tn
is simply Voting, and therefore satisfies Symmetry.
However, it is easy to see that the limit operator
T∗
has
T
∗
N
(
f)=
|
𝗌𝗋𝖼
N
(
f)
|
+
1
; this means
T∗
uses extra information beyond the structure of the network N in its ranking (namely, the
identity of a selected fact
f
) which violates Symmetry.
Using a similar tactic, one can construct a sequence of numerical operators
(Tn)n∈ℕ
such that each
Tn
satisfies PCI, but the limit operator
T∗
does not.
Fortunately, there is a natural strengthening of both Symmetry and PCI for numerical
operators which is preserved in the limit. Let us say that a numerical operator T satisfies
numerical Symmetry if for any equivalent networks
N,𝜋(N)
we have
TN(z)=T𝜋(N)(𝜋(z))
for all
z∈S∪F
. Similarly, T satisfies numerical PCI if for any networks
N1
and
N2
with a common connected component G, we have
TN1(z)=TN2(z)
for all
z∈G∩(S∪F)
.
Clearly numerical Symmetry implies Symmetry, and numerical PCI implies PCI. The
following result is immediate.
Lemma 5.4 Suppose
(Tn)n∈ℕ
converges to
T∗
. Then
• If
Tn
satisfies numerical Symmetry for each
n∈ℕ
, then
T∗
satisfies Symmetry.
• If
Tn
satisfies numerical PCI for each
n∈ℕ
, then
T∗
satisfies PCI.
As a consequence of Lemma 5.4, any recursive operator
𝗋𝖾𝖼(Tprior,U)
satisfies Sym-
metry whenever
Tprior
satisfies numerical Symmetry and U preserves numerical Symme-
try, in the sense that U(T) satisfies numerical Symmetry whenever T does (and similarly
for PCI).
Unanimity, Groundedness and Monotonicity. In contrast to Symmetry and PCI,
both Unanimity and Groundedness are preserved when taking the limit of an iterative
operator.
Lemma 5.5 Suppose
(Tn)n∈ℕ
converges to
T∗
. Then
• If
Tn
satisfies Unanimity for each
n∈ℕ
, then
T∗
satisfies Unanimity.
• If
Tn
satisfies Groundedness for each
n∈ℕ
, then
T∗
satisfies Groundedness.
For Monotonicity, we require the following (stronger) property to hold for each
Tn
.
Definition 5.2 A numerical operator T satisfies Improvement if for each
N,N′
and f as in
the statement of Monotonicity, we have
𝛿(f)>𝛿(g)
for all
g
≠
f
, where
In this case we write
𝜌N,N
�
=ming≠f(𝛿(f)−𝛿(g)) >0
.
𝛿(g)=TN
�
(g)−TN(g)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 17 of 49 42
Here
𝛿(g)
is the amount by which the belief score for g increases when going from the
network N to
N′
. Improvement simply says that when adding a new source to a fact f, it is f
that sees the largest increase.
Proposition 5.1 Suppose
(Tn)n∈ℕ
converges to
T∗
, and
Tn
satisfies Improvement for each
n∈ℕ
. Suppose also that
infn∈
ℕ
𝜌n
N,N
�
>0
for each
N,N′
arising in the statement of Mono-
tonicity. Then
T∗
satisfies Monotonicity.
Proof Let
N,N′
and f be as in the statement of Monotonicity, and suppose
g
⪯
T∗
N
f for some
g
≠
f
. We will show
g
≺
T∗
N�
f using Lemma 5.2.
Write
𝜌∗=inf
n∈ℕ
𝜌n
N,N
�
>0
and let
𝜀>0
. Since
g
⪯
T∗
N
f , there is
K∈ℕ
such that
Tn
N(g)−Tn
N(f)
≤
𝜀
for all
n≥K
. For such n, we have
By Lemma 5.2, we have
g
≺
T∗
N
�f as required.
◻
The requirement that
inf
n
∈
ℕ
𝜌n
N,N
�
>0
is a technical condition which ensures the strict
inequality
g
≺
T∗
N
�f holds in the limit, as required for Monotonicity. If this condition fails
T∗
still satisfies a natural ‘weak Monotonicity’ axiom, in which the strict inequality
g
≺
T∗
N
�f is
replaced with
g
⪯
T∗
N
�f.
5.3 Sums
We come to the axiomatic analysis of Sums. Coherence and the simpler axioms are satis-
fied here, and the undesirable independence axioms (POI and Strong Independence) are
not. However, Monotonicity and PCI do not hold. Since PCI is one of our most important
axioms that we expect any reasonable operator to satisfy, this potentially limits the useful-
ness of Sums in practise.
Theorem5.2 Sums satisfies Coherence, Symmetry, Unanimity and Groundedness. Sums
does not satisfy POI, Strong Independence, PCI or Monotonicity.
Proof (sketch) Symmetry, Unanimity and Groundedness can be easily shown from Lem-
mas 5.4 and 5.5; the details can be found in the appendix. In the remainder of the proof,
(Tn)n∈ℕ
will denote the iterative operator Sums,
T∗
will denote the limit operator, and
U=𝗇𝗈𝗋𝗆◦USums
will denote the update function for Sums.
Coherence. We will show Source-Coherence using Lemma 5.1. The argument for Fact-
Coherence is similar (using Lemma 5.3) and can be found in the appendix.
Suppose
N∈N
,
T∈TNum
,
𝜀,𝜌>0
, and
𝖿 𝖺𝖼𝗍𝗌N(s1)
is
(𝜀,𝜌)
-less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to N and T under a bijection
𝜑∶𝖿 𝖺𝖼𝗍𝗌N(s1)→ 𝖿 𝖺𝖼𝗍𝗌N(s2)
. By
Tn
N�
(
g
)−
T
n
N�
(
f
)=(
T
n
N
(
g
)+
n(
g
))−(
T
n
N
(
f
)+
n(
f
))
=Tn
N(g)−Tn
N(f)
≤
−(n(f)−n(g))
≥n
N,N�
≤−n
N,N�
≤−
∗
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 18 of 49
definition there is
f∈𝖿 𝖺𝖼𝗍𝗌N(s1)
such that
TN(
f)−TN(𝜑(
f))
≤
𝜀−𝜌
. By the remark after
the proof of Lemma 5.1, we may assume without loss of generality that
𝜀< 1
|
F
|𝜌
.
Recall that the update function for Sums is
U=𝗇𝗈𝗋𝗆◦USums
. Write
T�=USums(T)
and
T=U(T)=𝗇𝗈𝗋𝗆(U
Sums
(T))
so that
T=𝗇𝗈𝗋𝗆(T�)
. We must show that
TN
(s
1
)−
T
N
(s
2
)≤C
𝜀
−D
𝜌
for some constants
C,D>0
.
Note at this stage that it is possible to further weaken the hypotheses of Lemma 5.1: the
result follows if U has the stated property not for all operators T, but only for those such
that
T=Tn
for some
n∈ℕ
. Next, note that if
T�
N
(x)=
0
for all
x∈S
then trust and belief
scores are 0 in all subsequent iterations, and thus all sources rank equally in the limit
T∗
.
But this means the hypothesis for Source-Coherence cannot be satisfied (there are no strict
inequalities). We may therefore assume without loss of generality that
T�
N(
x
)
≠
0
for at least
one
x∈S
. Therefore, by definition of
𝗇𝗈𝗋𝗆
,
where
Applying the definition of
USums
and using the pairing of
𝖿 𝖺𝖼𝗍𝗌N(s1)
and
𝖿 𝖺𝖼𝗍𝗌N(s2)
via
𝜑
,
we have
To complete the proof, we need to find a lower bound for
𝛼
that is independent of T and N
(note that a lower bound on
𝛼
is required since
|F|𝜀−𝜌
is negative). It is here that we use
the assumption that
T=Tn
for some
n∈ℕ
. Since
Tn
N(x)∈[0, 1]
for any
n∈ℕ
and
x∈S
,
we have
TN
(s)=𝛼T
�
N
(s
)
𝛼
=
1
max
x∈S|
T�
N(x)
|
T
N(s1)−
Tn(s2)=[T
�
N(s1)−T
�
N(s2)]
=
f∈𝖿𝖺𝖼𝗍𝗌N(s1)
TN(f)−
f∈𝖿𝖺𝖼𝗍𝗌N(s1)
TN((f))
=
f∈𝖿𝖺𝖼𝗍𝗌N(s1)TN(f)−TN((f))
=
>0
TN(
f)−TN((
f))
≤−
+
f∈𝖿𝖺𝖼𝗍𝗌N(s1)⧵{
f}TN(f)−TN((f))
≤
≤
−+
f∈𝖿𝖺𝖼𝗍𝗌N(s1)⧵{
f}
≤⋅
F
−
<0
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 19 of 49 42
and so
Combining this with the above bound for
TN(s1)−
Tn(s2)
, we get
Taking
C=1
and
D
=
1
|
F
|
, the hypotheses of Lemma 5.1 are satisfied; thus Sums satisfies
Source-Coherence.
POI, Strong Independence, PCI and Monotonicity. The remaining axioms are handled
by counterexamples derived from the network shown in Fig.2. It can be shown that, if N
denotes this network, we have
T∗
N(f)=T∗
N(g)=0
, so f≈
T∗
Ng
.
Let
N′
denote the network whose claims are just those of the top connected component.
Then it can be shown that
T∗
N
�
(f)=1
and
T∗
N
�
(g)=0
, i.e.
g
≺
T∗
N
�f . However it is easily veri-
fied that our three independence axioms, if satisfied, would each imply f⪯
T∗
Ng
iff f⪯
T∗
N
�
g
.
Therefore none of POI, Strong Independence and PCI can hold for Sums.
For Monotonicity, consider the network
N′′
obtained from N by removing the edge
(u,g). Then we still have
T∗
N
��
(f)=T∗
N
��
(g)=0
, and in particular f⪯
T∗
N
��
g
. Returning to N
amounts to adding extra support for the fact g. Monotonicity would give f≺
T∗
Ng
here, but
this is clearly false. Hence Monotonicity is not satisfied by Sums.
◻
The key to the counterexamples derived from Fig. 2 in the above proof lies in the
lower connected component, which—restricted to
S∪F
—is a connected bipartite
graph. That is, each source
xi
claims all facts in the component, and each fact
yj
is
claimed by all sources in the component. Moreover, sources elsewhere in the network
claim fewer facts than the
xi
, and facts elsewhere are claimed by fewer sources than the
yj
.
Since Sums assigns scores by a simple sum, this results in the scores for the
xi
and
yj
dominating those of the other sources and facts. The normalisation step then divides
|
T�
N(x)
|
=T�
N(x)=
∑
f∈𝖿𝖺𝖼𝗍𝗌N(x)
TN(f)
≤1
≤
|
𝖿 𝖺𝖼𝗍𝗌N(x)
|
≤
|
F
|
𝛼
=
1
max
x∈S|
T�
N(x)
|
≥
1
|
F
|
T
N(s1)−
Tn(s2)≤
1
|
F
|(|
F
|
𝜀−𝜌
)
=𝜀−
1
|
F
|𝜌
Fig. 2 Network which yields
counterexamples for POI, Strong
Independence, PCI and Monoto-
nicity for Sums
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 20 of 49
these scores by the (comparatively large) maximum. As the next result shows, under
certain conditions this causes scores to decrease exponentially and become 0 in the
limit. In particular, we can generate pathological examples such as Fig.2 where a whole
connected component receives scores of 0, which leads to failure of Monotonicity and
the independence axioms.
Proposition 5.2 Let N be a network. Suppose there is
X⊆S
,
Y⊆F
such that
1.
𝖿 𝖺𝖼𝗍𝗌N(x)=Y
for each
x∈X
2.
𝗌𝗋𝖼N(y)=X
for each
y∈Y
3.
𝖿 𝖺𝖼𝗍𝗌N(s)∩Y=�
and
|
𝖿 𝖺𝖼𝗍𝗌
N
(s)
|
≤
|Y|
2
for each
s∈S⧵X
4.
𝗌𝗋𝖼N(
f
)∩
X
=�
and
|
𝗌𝗋𝖼N(f)
|
≤
|X|
2
for each
f∈F⧵Y
Then, with
(Tn)n∈ℕ
denoting Sums, for all
n>1
we have
In particular, if
T∗
denotes the limit of Sums then
T∗
N(s)=T∗
N(f)=0
for all
s∈S⧵X
and
f∈F⧵Y
.
Proof We proceed by induction. The result is easy to show in the base case
n=2
since
|
𝖿 𝖺𝖼𝗍𝗌N(s)
|
≤
1
2|
𝖿 𝖺𝖼𝗍𝗌N(x)
|
for any
x∈X
and
s∉X
(and similarly for facts). Assume the
result holds for some
n>1
. Write
T�=USums(Tn)
, so that
Tn+1=𝗇𝗈𝗋𝗆(T�)
. If
s∉X
then
𝖿 𝖺𝖼𝗍𝗌N(s)⊆F⧵Y
, so
Similarly, if
f∉Y
then
𝗌𝗋𝖼N(f)⊆S⧵X
, so
On the other hand, the fact that
Tn
N(x)=Tn
N(y)=1
for
x∈X
and
y∈Y
gives
T
n
N(s)≤
1
2n−1(s∈S⧵X)
T
n
N(f)≤1
2n−1(f∈F⧵Y
)
T
n
N(x)=1(x∈X)
T
n
N
(y)=1(y∈Y)
T
�
N(s)= ∑
f∈𝖿𝖺𝖼𝗍𝗌N(s)
Tn
N(f)
≤1
2n−1
≤|𝖿 𝖺𝖼𝗍𝗌N(s)|
2n−1≤
1
2
|
Y
|
2n−1=|Y|
2(n+1)−1
T
�
N(f)= ∑
s∈𝗌𝗋𝖼N(f)
T�
N(s)
≤
|
Y
|
2(n+1)−1
≤|𝗌𝗋𝖼N(f)|⋅|Y|
2(n+1)−1≤
1
2
|
X
|
⋅
|
Y
|
2(n+1)−1=|X|⋅|Y
|
2(n+2)−1
T
�
N(x)=
∑
y∈Y
Tn
N(y)=
|
Y
|
T
�
N(y)=
∑
x∈X
T�
N(x)=
|
X
|
⋅
|
Y
|
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 21 of 49 42
Clearly the
x∈X
and
y∈Y
are the sources and facts with maximal trust and belief scores,
respectively. This means that after normalisation via
𝗇𝗈𝗋𝗆
,
Tn+1
N
(x)=T
n+1
N
(y)=
1
and for
s∉X
and
f∉Y
,
This shows that the claim holds for
n+1
; by induction, the proof is complete.
◻
5.4 Modifying Voting andSums
So far we have seen that neither of the basic operators Voting or Sums are completely satis-
factory with respect to the axioms of Sect.4. Armed with the knowledge of how and why
certain axioms fail, one may wonder whether it is possible to modify the operators accord-
ingly so that the axioms are satisfied. Presently we shall show that this is partially possible
both in the case of Voting and Sums.
5.4.1 Voting
A core problem with Voting is that it fails Coherence. Indeed, all sources are ranked equally
regardless of the ‘votes’ for facts, so in some sense it is obvious that the source ranking
does not cohere with the fact ranking.8 An easy improvement is to explicitly construct the
source ranking to guarantee Source-Coherence.
Definition 5.3 For a network N, define a binary relation
⊲N
on
S
by
s1⊲Ns2
iff
𝖿 𝖺𝖼𝗍𝗌N(s1)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to Voting. The numerical operator SC-Voting
(Source-Coherence Voting) is defined by
It can be seen that SC-Voting satisfies Source-Coherence, which is a significant improve-
ment over regular Voting. Since
⊲N
relies on ‘global’ properties on N, however, this comes
at the expense of Source-PCI. Satisfaction of the other axioms is inherited from Voting.
Theorem5.3 SC-Voting satisfies Source-Coherence, Symmetry, Unanimity, Groundedness,
Monotonicity, Fact-PCI, POI and Strong Independence. It does not satisfy Fact-Coherence
or Source-PCI.
The following properties of
⊲N
are useful for showing Source-Coherence.
Lemma 5.6
⊲N
is transitive and irreflexive.
T
n+1
N(s)=
T
�
N
(
s
)
|Y|≤1
2(n+1)−1
T
n+1
N(f)= T�
N(f)
|X|
⋅
|Y|
≤1
2
(n+2)−1≤1
2
(n+1)−1
TSCV
N
(s)=
|
{t∈S∶t⊲
N
s}
|
,T
SCV
N
(f)=
|
𝗌𝗋𝖼
N
(f)
|
8 Fact-Coherence is vacuously satisfied, however: since all sources rank equally we can never have
𝗌𝗋𝖼N(f
1
)
less trustworthy than
𝗌𝗋𝖼N(f2)
.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 22 of 49
Proof For transitivity, suppose
s⊲Nt
and
t⊲Nu
. Then
𝖿 𝖺𝖼𝗍𝗌N(s)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(t)
(with respect to Voting) via some bijection
𝜑∶𝖿 𝖺𝖼𝗍𝗌N(s)→ 𝖿 𝖺𝖼𝗍𝗌N(t)
, and
𝖿 𝖺𝖼𝗍𝗌N(t)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(u)
via some bijection
𝜓∶𝖿 𝖺𝖼𝗍𝗌N(t)→ 𝖿 𝖺𝖼𝗍𝗌N(u)
. It
is easily seen that
𝖿 𝖺𝖼𝗍𝗌N(s)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(u)
via the composition
𝜃=𝜓◦𝜑
,
so
s⊲Nu
.
For irreflexivity, suppose for contradiction that
s⊲Ns
for some
s∈S
, i.e.
F=𝖿 𝖺𝖼𝗍𝗌N(s)
is less believable than itself under some bijection
𝜑∶F→F
. Then f⪯
T
N
𝜑(f
)
for each
f∈F
, and there is
f
such that
f≺T
N
𝜑(
f
)
. Iterating applications of
𝜑
, we get
for each
n≥1
, where
𝜑n
is the n-th iterate of
𝜑
and T denotes Voting.
Since F is finite, the sequence
𝜑(
f),𝜑(𝜑(
f)),…
must repeat at some point, i.e. there
is
i<j
such that
𝜑
i
(
f)=𝜑
j
(
f)
. But then injectivity of
𝜑
implies that
f=𝜑
j
−
i
(
f)
. Taking
n=j−i
in Eq. (5.1) we get
f≺
T
N
f —a contradiction.
◻
Proof (Theorem 5.3 (sketch)) Note that SC-Voting inherits Unanimity, Groundedness,
Monotonicity, Fact-PCI, POI and Strong Independence from Voting, since these axioms
only refer to the rankings of facts (which is the same for SC-Voting as for Voting).
We take the remaining axioms in turn. To simplify notation, write
WN(s)={t∈S∶t⊲Ns}
in what follows.
Source-Coherence. Suppose
𝖿 𝖺𝖼𝗍𝗌N(s1)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to
TSCV
. We need to show
s1
⊏T
SCV
N
s
2
.
Note that since the fact ranking for
TSCV
coincides with Voting, we have
s1⊲Ns2
. Transi-
tivity of
⊲N
gives
WN(s1)⊆WN(s2)
. Moreover,
s1∈WN(s2)
but by irreflexivity,
s1∉WN(s1)
.
Therefore
WN(s1)⊂WN(s2)
, which means
TSCV
N
(s
1
)=
|
W
N
(s
1
)
|
<
|
W
N
(s
2
)
|
=T
SCV
N
(s
2)
, i.e.
s1
⊏T
SCV
N
s
2
as required.
Symmetry. Since the fact ranking of
TSCV
is the same as Voting, which satisfies Symme-
try, we only need to show that
s1
⊑T
SCV
N
s
2
iff
𝜋
(s1)⊑T
SCV
𝜋(N)
𝜋(s2
)
for all equivalent networks
N,𝜋(N)
and
s1,s2∈S
.
(5.1)
f≺T
N
𝜑(
f)⪯
T
N
𝜑(𝜑(
f)⪯
T
N
⋯⪯T
N
𝜑n(
f
)
Fig. 3 Fact-Coherence counter-
example for SC-Voting
Fig. 4 Source-PCI counterexam-
ple for SC-Voting
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 23 of 49 42
In can be shown, and we do so in the appendix, that the Symmetry of Voting implies a
symmetry property for
⊲N
and
⊲𝜋(N)
: we have
s1⊲Ns2
iff
𝜋(s1)⊲𝜋(N)𝜋(s2)
. Consequently,
t∈WN(si)
iff
𝜋(t)∈W𝜋(N)(𝜋(si))
; in particular,
|WN(si)|=|W𝜋(N)(𝜋(si))|
. This means
as required for Symmetry.
Fact-Coherence Consider the network shown in Fig.3. We have
f≈g≈i≺h
. Source-
Coherence between s and t gives
t⊏s
. If Fact-Coherence held we would then get
g≺f
,
but this is not the case.
Source-PCILet
N1
denote the top connected component of the network shown in Fig.4,
and let
N2
denote the network as a whole. The fact ranking is the same in both networks:
g≈h≈i≺f
. In
N1
sources s and t claim a different number of facts, so neither
s⊲N1
t
nor
t⊲
N
1
s
. Consequently
W
N
1(s)=W
N
1(t)=�
and
s
≃T
SCV
N1
t
.
In
N2
sources t and u can be compared for Source-Coherence, and we see that
u⊲N2
t
since
i
⪯T
SCV
N
2
g
and
h
≺T
SCV
N
2
f . Hence
WN2(t)={u}
and
WN2(
s
)=�
, which means
s
⊏T
SCV
N
2
t
.
This contradicts Source-PCI, which requires the ranking of s and t to be the same in both
networks.
◻
Note that the idea behind SC-Voting can be generalised beyond Voting: it is possible
to define
⊲N
in terms of any operator T, and to construct a new operator
T′
whose source
ranking is given according to
⊲N
as above, and whose fact ranking coincides with that of T.
This ensures
T′
satisfies Source-Coherence whilst keeping the existing fact ranking from T.
Moreover we can go in the other direction and ensure Fact-Coherence whilst retaining
the source ranking of T by defining a relation
◀N
on
F
in a analogous manner to
⊲N
, and
proceeding similarly.
5.4.2 Sums
Our main concern with Sums is the failure of PCI and Monotonicity. We have seen that this
is in some sense caused by the normalisation step: in Fig.2 the scores of f,g etc go to 0 in
the limit after dividing the ‘global’ maximum score across the network, which happens to
come from a different connected component.
A natural fix for PCI is to therefore divide by the maximum score within each compo-
nent. In this case the score for a source s depends only on the structure of the connected
component in which it lies, which is exactly what is required for PCI.
However, this approach does not negate the undesirable effects of proposition 5.2.
Indeed, suppose the network in Fig.2 was modified so that fact
y1
is associated with object
o instead of
p1
. Clearly proposition 5.2 still applies after this change, and all sources and
facts shown now belong to the same connected component. Therefore the ‘per-component
Sums’ operator gives the same result as Sums itself, and in particular our Monotonicity
counterexample still applies. Perhaps even worse, one can show that Coherence fails for
this operator. We consider the loss of Coherence too high a price to pay for PCI.
Instead, let us take a step back and consider if normalisation is truly necessary. On the
one hand, without normalisation the trust and belief scores are unbounded and therefore do
not converge. On the other, we are not interested in the numeric scores for their own sake,
s
1⊑T
SCV
Ns2⟺
|
WN(s1)
|
≤
|
WN(s2)
|
⟺
|
W𝜋(N)(𝜋(s1))
|
≤
|
W𝜋(N)(𝜋(s2))
|
⟺𝜋(s1)⊑TSCV
𝜋(N)
𝜋(s2)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 24 of 49
but rather for the rankings that they induce. It may be possible that whilst the scores diverge
without normalisation, the induced rankings do converge to a fixed one, which we may take
as the ‘ordinal limit’. This is in fact the case. We call this new operator UnboundedSums.
Definition 5.4 UnboundedSums is the recursive operator
𝗋𝖾𝖼(Tprior,USums )
where
Tprior
N
(s)=
1
,
Tprior
N
(f)=
|
𝗌𝗋𝖼
N
(f)
|
and
USums
is defined as in Sect.3.2.9
Theorem 5.4 UnboundedSums is ordinally convergent in the following sense: there
is an ordinal operator
T∗
such that for each network N there exists
JN∈ℕ
such that
Tn
N(s1)≤Tn
N(s2)
iff
s1
⊑
T∗
N
s
2
for all
n≥JN
and
s1,s2∈S
(and similarly for facts).
That is, the rankings induced by
Tn
remain constant after
JN
iterations, and are identical
to those of
T∗
.
Proof The proof will use some results from linear algebra, so we will work with a matrix
and vector representation of UnboundedSums. Fix an enumeration
S={s1,…,sk}
of
S
and
F={f1,…,fl}
of
F
. Write M for the
k×l
matrix given by
We also write
vn
and
wn
for the vectors of trust and belief scores of UnboundedSums at
iteration n; that is
where
(Tn)n∈ℕ
denotes UnboundedSums.
Multiplication by M encodes the update step of UnboundedSums: it is easily shown
that
vn+1=Mwn
and
wn+1=M⊤vn+1
. Writing
A=MM⊤∈ℝk×k
, we have
vn+1=Avn
, and
therefore
vn+1=Anv1
.
To show that the rankings of UnboundedSums remain constant after finitely many itera-
tions, we will show that for each
sp
,s
q∈S
there is
Jpq ∈ℕ
such that
sign ([vn]p−[vn]q)
is
constant for all
n≥Jpq
. Since
[vn]p
and
[vn]q
are the trust scores of
sp
and
sq
respectively
in the n-th iteration, this will show that the ranking of
sp
and
sq
remains the same after
Jpq
iterations. Since there are only finitely many pairs of sources, we may then take
JN
as the
maximum value of
Jpq
over all pairs (p,q), and the entire source ranking
⊑Tn
N
of Unbound-
edSums remains constant for
n≥JN
. An almost identical argument can be carried out for
the fact ranking, and these together will prove the result.
So, fix
sp,sq∈S
. Write
𝛿n=[vn]p−[vn]q
. First note that
A=MM⊤
is symmetric, so
the spectral theorem gives the existence of k orthogonal eigenvectors
x1,…,xk
for A [5,
Theorem 7.29]. Let
𝜆1,…,𝜆k
be the corresponding eigenvalues. Form a
(k×k)
-matrix
P whose i-th column is
xi
, and let
D=diag(𝜆1,…,𝜆k)
. Then A can be diagonalised as
A=PDP−1
. It follows that for any
n∈ℕ
,
An=PDnP−1
.
[
M]ij =
{
1 if si∈𝗌𝗋𝖼N(fj)
0 otherwise (1≤i≤k,1≤j≤l
)
vn=[T
n
N(s1),…,T
n
N(sk)]
⊤
∈ℝ
k
wn
=[Tn
N
(f
1
),…,Tn
N
(f
l
)]⊤∈ℝl
9 Note that to simplify proof of ordinal convergence we use a different prior operator to Sums, but this does
not effect the operator in any significant way.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 25 of 49 42
Now, since
x1,…,xk
are orthogonal, P is an orthogonal matrix, i.e.
P⊤=P−1
. Hence
An=PDnP⊤
. Note that
and
which means
We obtain an explicit formula for
𝛿n+1
:
where
ri
=(x
i⋅
v
1
)
(
[x
i
]
p
−[x
i
]
q)
. Note that
ri
does not depend on n.
Now, it is easy to see that
A=MM⊤
is positive semi-definite, which means its eigenval-
ues
𝜆1,…,𝜆k
are all non-negative. We re-index the sum in Eq. (5.2) by grouping together
the
𝜆i
which are equal, to get
where
K≤k
, each
Rt
is a sum of the
ri
(whose corresponding
𝜆i
are equal), and the
𝜇t
are
distinct and non-negative. Assume without loss of generality that
𝜇1>𝜇
2>⋯>𝜇
K≥0
.
If
Rt=0
for all t, then clearly
sign (𝛿n+1) = sign (0)=0
which is constant, so we are done.
Otherwise, let T be the minimal t such that
Rt≠0
. We may also assume
𝜇T>0
(otherwise
we necessarily have
𝜇T=0
,
T=K
and
sign (𝛿n+1) = sign (RT⋅0n)
which is again constant
0). Observe that
By our assumption on the ordering of the
𝜇t
, we have
𝜇t<𝜇
T
in the sum. Consequently
|𝜇t∕𝜇T|<1
, and
(𝜇t∕𝜇T)n→0
as
n→∞
. This means
PD
n=
x1∣…∣xk
𝜆n
1
⋱
𝜆n
k
=
𝜆n
1x1∣…∣𝜆n
kxk
P
⊤v1=
⎡
⎢
⎢
⎢
⎢
⎣
x
1
−
⋮
−
x
k
⎤
⎥
⎥
⎥
⎥
⎦
v1=
⎡
⎢
⎢
⎣
x1⋅v1
⋮
xk⋅v1
⎤
⎥
⎥
⎦
v
n+1=Anv1=PDnP⊤v1=
𝜆n
1x1∣…∣𝜆n
kxk
x1⋅v1
⋮
xk⋅v1
=
k
i=1
(xi⋅v1)𝜆n
ix
i
(5.2)
𝛿
n+1=[vn]p−[vn]q=
k
∑
i=1
(xi⋅v1)𝜆n
i
(
[xi]p−[xi]q
)
=
k
∑
i=1
ri𝜆
n
i
𝛿
n+1=
K
∑
t=1
Rt𝜇
n
t
𝛿
n+1=RT𝜇n
T+
K
∑
t=T+1
Rt𝜇n
t
=𝜇n
T
[
RT+
K
∑
t=T+1
Rt
(
𝜇t
𝜇T
)
n
]
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 26 of 49
Since this limit is non-zero, there is
Jpq ∈ℕ
such that the sign of term in square brackets is
equal to
S= sign RT∈{1, −1}
for all
n≥Jpq
. Finally, for such n we have
i.e.
sign 𝛿n
is constant for
n≥Jpq +1
. This completes the proof.10
◻
In light of Theorem5.4, we may consider UnboundedSums itself as an ordinal opera-
tor
T∗
, where
s
⊑
T∗
Nt
iff
s
⊑T
J
n
Nt
for each network N (and similarly for the fact ranking).
Moreover, with the normalisation problems aside, UnboundedSums provides an example of
a principled operator satisfying our two key axioms—Coherence and PCI. In particular, we
escape the undesirable behaviour of Sums in Fig.2; whereas Sums trivialises the ranking of
sources and facts in the upper connected component, UnboundedSums allows meaningful
ranking (e.g. we have
g≺f
). In particular, the counterexample for Monotonicity for Sums
is no longer a counterexample for UnboundedSums. We conjecture that UnboundedSums
also satisfies Monotonicity, but this remains to be proven. For example, we have experi-
mentally verified that UnboundedSums satisfies all the specific instances of Monotonicity
with the starting network N as in Fig.1.
Theorem5.5 UnboundedSums satisfies Coherence, Symmetry, Unanimity, Groundedness
and PCI. UnboundedSums does not satisfy POI and Strong Independence.
Proof (sketch) The proof that UnboundedSums satisfies Symmetry, PCI, Unanimity and
Groundedness is routine, and we leave the details to the appendix. In what follows, let
(Tn)n∈ℕ
denote UnboundedSums,
T∗
denote the ordinal limit of UnboundedSums, and for a
network N let
JN
be as in Theorem5.4. Then the rankings in N induced by
Tn
for
n≥JN
are
the same as
T∗
.
Coherence. First we show Source-Coherence. Let N be a network and suppose
𝖿 𝖺𝖼𝗍𝗌N(s1)
is less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
with respect to N and
T∗
. Let
𝜑
and
f
be as in the defini-
tion of less believable.
Let
n≥JN
. Then f⪯
T∗
N
𝜑(f
)
and
f≺T
∗
N
𝜑(
f
)
for each
f∈𝖿 𝖺𝖼𝗍𝗌N(s1)
means
Tn
N(f)≤Tn
N(𝜑(f))
and
T
n
N
(
f)<Tn
N
(𝜑(
f
))
. Hence
lim
n
→∞
RT+
K
t=T+1
Rtt
Tn
→0
=RT≠
0
sign
n+1= sign
n
T
>0
RT+
K
t=T+1
Rtt
Tn
= sign
RT+
K
t=T+1
Rtt
Tn
=
S
10 The argument which shows that the difference between fact belief scores is also eventually constant in
sign is almost identical. Write
B=M⊤M
, and observe that
wn+
1
=Bnw1
. Since B is also symmetric and
positive semi-definite, the proof goes through as above.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 27 of 49 42
i.e.
Tn+1
N
(s
1
)<T
n+1
N
(s
2)
. But
Tn+1
N
gives the same ranking as
Tn
N
and therefore the same
ranking as
T∗
, so we get
s1
⊏
T∗
N
s
2
as required.
For Fact-Coherence, suppose
𝗌𝗋𝖼N(f1)
is less trustworthy than
𝗌𝗋𝖼N(f2)
with respect to
N and
T∗
. Again, let
n≥JN
and
𝜑
,
s
be as in the definition of less trustworthy. Recall that
belief scores for facts in
Tn
N
are obtained from trust scores in
Tn
N
; applying the same argu-
ment as above we get
Tn
N(f1)<Tn
N(f2)
and consequently f
1
⪯
T∗
N
f
2
as required. Hence
T∗
satisfies Coherence.
POI and Strong Independence. To show POI and Strong Independence are not satisfied,
consider the network N shown in Fig.5. It can be seen (e.g. by induction) that
T
n
+
1
N(s)=
∑
f∈𝖿𝖺𝖼𝗍𝗌N(s1)
Tn
N(f)
=Tn
N(
f)+ ∑
f∈𝖿𝖺𝖼𝗍𝗌N(s1)⧵{
f}
Tn
N(f)
<Tn
N(𝜑(
f)) + ∑
f∈𝖿𝖺𝖼𝗍𝗌N(s1)⧵{
f}
Tn
N(𝜑(f
))
=∑
f∈𝖿𝖺𝖼𝗍𝗌N(s1)
Tn
N(𝜑(f))
=∑
g∈𝖿𝖺𝖼𝗍𝗌N(s2)
Tn
N(g)
=Tn+1
N
(s
2
)
Tn
N
(f)=1, T
n
N
(g)=2
n−1
Fig. 5 Counterexample for POI
and Strong Independence for
UnboundedSums
Table 1 Satisfaction of the
axioms for the various operators.
Recall that POI and Strong
Independence are undesirable
properties
Voting SC-voting Sums U-sums
Source-coherence X
✓
✓
✓
Fact-coherence
✓
X
✓
✓
Symmetry
✓
✓
✓
✓
Unanimity
✓
✓
✓
✓
Ground.
✓
✓
✓
✓
Mon.
✓
✓
X?
Source-PCI
✓
X X
✓
Fact-PCI
✓
✓
X
✓
POI
✓
✓
X X
Str. indep.
✓
✓
X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 28 of 49
for all
n∈ℕ
. Consequently f≺
T∗
Ng
.11
Now let
N′
be the network in which the claim (t, h) is removed. Since
𝗌𝗋𝖼N(f)=𝗌𝗋𝖼N
�
(f)={s}
and
𝗌𝗋𝖼N(g)=𝗌𝗋𝖼N
�
(g)={t}
, both POI and Strong Independence
imply f⪯
T∗
Ng
iff f⪯
T∗
N
�
g
. Therefore assuming either of POI or Strong Independence we
get f≺
T∗
N
�
g
. However is is also clear that
for all
n∈ℕ
, so f≈
T∗
N
�
g
. This is a contradiction, so neither POI nor Strong Independence
are satisfied.
◻
To summarise this section, Table1 shows which axioms are satisfied by each of the
operators.
6 Variable domain truth discovery
So far, we have considered an arbitrary but fixed (finite) domain of sources, facts and
objects
(S,F,O)
. Our operators and axioms were defined with respect to this domain.
However, the operators do not depend on the domain: they can be defined for any choice
of
S
,
F
and
O
. In this section we generalise the framework so that these sets are no longer
fixed. This allows new situations to be modelled, such as new sources entering the network.
Adapting the definition of a TD operator to this case, we can then see how the ranking of
facts changes as new sources are added. Such variable domain operators are then analogues
of variable electorate voting rules in social theory.
Formally, let
𝕊
,
𝔽
and
𝕆
be countably infinite sets of sources, facts and objects respec-
tively. A domain is a triple
D=(S,F,O)
, where
S⊆𝕊
,
F⊆𝔽
and
O⊆𝕆
are finite, non-
empty sets. We think of
𝕊
,
𝔽
and
𝕆
as being the ‘universe’ of possible sources, facts and
objects, and a domain as the (finite) sets of entities under consideration in a particular TD
problem. Given a domain
D=(S,F,O)
, we define
D
-networks and
D
-operators as in defi-
nitions 2.1 and 2.2.
Definition 6.1 A variable domain operator T is a mapping which maps each domain
D
to
a
D
-operator
TD
.
Note that for a domain
D=(S,F,O)
and a
D
-network N,
⊑TD
N
and
⪯TD
N
are rankings only
over the set of sources
S
and
F
in the domain
D
, not all of
𝕊
and
𝔽
. If
D
is clear from con-
text, we write
⊑T
N
and
⪯T
N
without explicit reference to the domain.
Since all the axioms so far were stated with respect to a fixed but arbitrary domain, they
can be easily lifted to the variable domain case. For instance, we say a variable domain
operator T satisfies Coherence if
TD
satisfies the instance of Coherence for domain
D
, for
all
D
, and similar for the other axioms.
But we can now go further, and introduce axioms which make use of several
domains. First, we generalise Symmetry to act across domains. Say networks
N,N′
in
domains
D,D′
respectively are equivalent if there is a graph isomorphism
𝜋
between
them such that
𝜋(s)∈
S
�
,
𝜋(f)∈
F
�
and
𝜋(o)∈
O
�
for all
s∈S
,
f∈F
and
o∈O
.
Tn
N
�
(f)=Tn
N
�
(g)=1
11 Note that g ranks higher than f in this network simply because t makes more claims than s, and each fact
is claimed only by a single source. This questionable property of UnboundedSums is inherited from Sums.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 29 of 49 42
Axiom 9 (Isomorphism) Let N and
N�=𝜋(N)
be equivalent networks. Then for all
s1,s2∈S
,
f1,f2∈F
, we have
s1
⊑
T
N
s
2
iff
𝜋
(s
1
)⊑
T
N
�𝜋(s
2)
and f
1
⪯
T
N
f
2
iff
𝜋
(f
1
)⪯
T
N
�𝜋(f
2)
.
Like Symmetry, Isomorphism simply says that operators only care about the structure
of the network, not the particular ‘names’ chosen for sources, facts and objects. Symme-
try is obtained as the special case where N and
N′
are equivalent when seen as networks
in a common domain
D
. All the operators of Sects. 3 and 5.4 satisfy Isomorphism.
Next we introduce a new monotonicity property. In what follows, for a network
N=(V,E)
in domain
(S,F,O)
,
f∈F
and
S′⊆𝕊
finite and disjoint from
S
, write
N+(
S
�
,f)
for the network in domain
(
S
∪
S
�
,
F
,
O
)
with edge set
E∪ {(s,f)∣s∈
S
�}
,
i.e. the extension of N where a collection of ‘fresh’ sources
S′
each claim f. For example,
Fig.6 shows
N+(
S
�
,h)
for the network N from Fig.1 and new sources
S�={x1,…,x4}
.
Axiom 10 (Eventual Monotonicity) Let
D=(S,F,O)
be a domain and N a
D
-network.
Then for all
f,g∈F
,
f≠g
, there is a finite, non-empty set
S′⊆𝕊
with
SS=
and
g
≺
T
N+(
S�
,f)
f.
This axiom says that, given any pair of distinct facts f,g, it is possible to add enough
new claims for f to make f strictly more believable than g. Intuitively, this is less
demanding that Monotonicity, which requires that f can become strictly more believable
than g with the addition of just one additional claim. Note that Eventual Monotonicity is
not possible to state in the fixed domain case (e.g. consider N already containing claims
from all the available sources in
S
).
When paired with Isomorphism, Eventual Monotonicity takes on a form similar to
postulates for Improvement and Majority operators in belief merging [22, 23]: there is a
threshold
n∈ℕ
such that f becomes strictly more believable than g after n new claims
are added for f. That is, the identities of the new sources
S′
are irrelevant; it is just the
size of
S′
that matters. We require a preliminary lemma.
Fig. 6
N+(
S
�
,h)
, where N is
the network from Fig.1 and
S�={x1
,
…
,
x4}
.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 30 of 49
Lemma 6.1 Suppose a variable domain operator T satisfies Isomorphism. Let
D=(S,F,O)
be a domain, N a network in
D
and
f∈F
. Then for all non-empty, finite
sets
S′
1
,
S′
2
⊆
𝕊
disjoint from
S
with
|S�
1|=|S�
2|
,
Proof Write
D1
=(
S
∪
S�
1
,
F
,
O)
and
D2
=(
S
∪
S�
2
,
F
,
O)
. Then
N
+(
S�
i
,f
)
is a net-
work in domain
Di
(for
i∈{1, 2}
). Since
|
S
�
1|
=
|
S
�
2|
by assumption, there is a bijection
𝜑
∶
S�
1
→
S�
2
. Define a mapping
𝜋
from
D1
to
D2
by
and
𝜋(g)=g
,
𝜋(o)=o
for
g∈F
and
o∈O
. Then it is easily verified that
𝜋
is an isomor-
phism from
N+(S�
1
,f
)
to
N+(S�
2
,f
)
. For
g1,g2∈F
, we have
g
1
⪯T
N+(S�
1
,f)g
2
i ff
𝜋
(g1)⪯
T
N+(S�
2
,f)𝜋(g2
)
by Isomorphism. Since
𝜋(g1)=g1
and
𝜋(g2)=g2
, this shows
⪯T
N+(S�
1
,f)=⪯
T
N+(S�
2
,f
)
.
◻
Note that since
𝕊
is infinite and domains are finite, for any
n∈ℕ
and any domain
D=(S,F,O)
there is always some
S′⊆𝕊
, disjoint from
S
, wit h
|S�|=n
. For operators T
satisfying Isomorphism, write
⪯T
N+(n×f)
for
⪯T
N+(
S�
,f)
; Lemma 6.1 guarantees this is well-
defined (i.e. does not depend on the particular choice of
S′
). That is,
⪯T
N+(n×f)
is the fact
ranking resulting from adding n new claims for f from fresh sources. We obtain an equiva-
lent characterisation of Eventual Monotonicity, whose proof is almost immediate given
Lemma 6.1.
Proposition 6.1 Suppose T satisfies Isomorphism. Then T satisfies Eventual Monotonicity if
and only if for all domains
D=(S,F,O)
, all networks N in
D
and distinct
f,g∈F
, there
is
n∈ℕ
such that
g
≺
T
N+(n×f)
f.
Proof (sketch) ‘if’: To show Eventual Monotonicity, take any
S′⊆
𝕊
⧵S
of size n.
‘Only if’: Given that Eventual Monotonicity holds, simply take
n=|S�|
.
◻
We can now show that all operators studied so far—when lifted to the variable domain
case—satisfy Eventual Monotonicity.
Theorem6.1 Voting, Sums, SC-Voting and UnboundedSums satisfy Eventual Monotonicity.
Proof (sketch) Let
D=(S,F,O)
be a domain, N a network in
D
and
f,g∈F
. Given that
Isomorphism holds for each operator, we sketch the proof via proposition 6.1.
For Voting and SC-Voting, we may simply take
n=1+|𝗌𝗋𝖼N(g)|
. For Sums and
UnboundedSums, take
n=2|S|⋅|F|
. Write
N�=N+(S�
,f)
for some
S′⊆
𝕊
⧵S
with
|S�|=n
If
(
T
k
)
k∈ℕ
denotes Sums, one can show by induction that
Tk
N
�(f)=
1
and
T
k
N
�(h)≤
1
2
for
any
h
≠
f
and
k>1
, and thus
g
≺T
S
ums
N′
f.
Similarly, letting
(
T
k
)
k∈ℕ
denote UnboundedSums, one can show by induction that
Tk
N�
(f)>T
k
N�
(h
)
for
h
≠
f
, and thus
g
≺T
U
nboundedSums
N′g
.
◻
⪯T
N
+(
S�
1,
f
)
=⪯
T
N
+(
S�
2,
f
)
𝜋
(s)=
{
s,s∈S
𝜑(s),s∈S�
1
(s∈S∪S�
1
)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 31 of 49 42
To conclude this section, we show that the impossibility result of Theorem4.2 holds
in the variable domain case if one replaces Monotonicity with Eventual Monotonicity and
Symmetry with Isomorphism.
Theorem6.2 There is no variable domain operator satisfying Coherence, Isomorphism,
Eventual Monotonicity and POI.
Proof For contradiction, suppose T is an operator satisfying the stated axioms. Let N be
the network from Fig. 1, viewed as a network in domain
({s,t,u,v},{f,g,h,i},{o,p})
.
Applying Eventual Monotonicity with i and h, we have that there is
N′
with
i
≺
T
N′h
, where
N�=N+(S�
,h)
for some
S�⊆
𝕊
⧵{s,t,u,v}
. Since
N′
only adds claims for p-facts, POI
applied to object o and Isomorphism give f≈
T
N�g
(e.g. consider
𝜋
which simply swaps
s with t and f with g). From Source-Coherence we get
t
⊏
T
N
′
s
. But
𝗌𝗋𝖼N
�
(f)={s}
and
𝗌𝗋𝖼N
�
(g)={t}
, so Fact-Coherence gives
g
≺
T
N′
f : contradiction!
◻
7 Discussion
In this section we discuss the axioms and their limitations. First, the version of Mono-
tonicity we consider is a strict one: a new claim for f gives f a strictly positive boost in
the fact believability ranking. This is also the case for Eventual Monotonicity in the vari-
able domain case, where we require that some number of new claims make f strictly more
believable than any other fact g. As noted in Sect.4.3, this assumes there is no collusion
between sources. Indeed, suppose sources
s1
,
s2
are in collusion. For example,
s2
may agree
to unconditionally back up all claims made by
s1
. In this case a claim of f from
s1
alone
should carry just as much weight as the claim from both
s1
and
s2
. However, Monotonicity
requires that f becomes strictly more believable when moving to the latter case.
A natural solution is to simply relax the strictness requirement. We obtain the following
weak version of Monotonicity.
Axiom 11 (Weak Monotonicity) Let
N,s,f,N′
be as in the statement of Monotonicity.
Then for all
g≠f
,
g⪯T
N
implies
g⪯T
N
�f.
Weak Monotonicity only says says that extra support for a fact f does not make f less
believable. Clearly Monotonicity implies Weak Monotonicity, but not vice versa. In the
collusion example, an operator may select to leave the fact ranking unchanged when a new
report of f from
s2
arrives; this is inconsistent with Monotonicity but consistent with Weak
Monotonicity. The weak analogue of Eventual Monotonicity can be defined in the same
way.
In the same spirit, one could consider versions of Coherence only using weak compari-
sons. Say
𝖿 𝖺𝖼𝗍𝗌N(s1)
is weakly less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
iff the condition in definition
4.1 holds, but without the requirement that some
f∈𝖿 𝖺𝖼𝗍𝗌
N
(s
1)
is strictly less believable
than its counterpart
𝜑(
f)
in
𝖿 𝖺𝖼𝗍𝗌N(s2)
, and define
𝗌𝗋𝖼N(f1)
weakly less trustworthy than
𝗌𝗋𝖼N(f2)
in a similar way. The weak analogue of Coherence is as follows.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 32 of 49
Axiom 12 (Weak Coherence) For any network N,
𝖿 𝖺𝖼𝗍𝗌N(s1)
weakly less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
implies
s1
⊑
T
N
s
2
, and
𝗌𝗋𝖼N(f1)
weakly less trustworthy than
𝗌𝗋𝖼N(f2)
implies
f
1
⪯
T
N
f
2
.
Note that Coherence does not imply Weak Coherence. This is because Weak Coher-
ence relaxes both the consequent and the antecedent in the implications in the statement
of the axiom. Whereas Coherence imposes no constraint when
𝖿 𝖺𝖼𝗍𝗌N(s1)
is only weakly
less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
, Weak Coherence requires
s1
⊑
T
N
s
2
. Consequently, the
‘weakness’ of Weak Coherence refers to its use of weak comparisons between sources
and facts, not its logical strength in relation to Coherence.
A natural question now arises. Does the impossibility result of Theorem4.2 still hold
with these new axioms? We have an answer in the negative: the ‘flat’ operator, which
sets all sources and facts equally ranked in all networks, satisfies all the axioms of the
would-be impossibility.
Proposition 7.1 Define an operator T by
s1
≃
T
N
s
2
and f≈
T
N
f
2
for all networks N, sources
s1,s2
and facts
f1,f2
. Then T satisfies Coherence, Weak Coherence, Symmetry, Weak Mono-
tonicity and POI.
Proof Coherence holds vacuously since we can never have
𝖿 𝖺𝖼𝗍𝗌N(s1)
less believable than
𝖿 𝖺𝖼𝗍𝗌N(s2)
or
𝗌𝗋𝖼N(f1)
less believable than
𝗌𝗋𝖼N(f2)
. Since any weak ranking holds for T, the
other axioms are straightforward to see.
◻
This shows that (strict) Monotonicity is required for the impossibility result, since
the result is no longer true when relaxing to Weak Monotonicity.
We now consider the new axioms in relation to the operators. First, Weak Coherence.
Proposition 7.2 Voting, Sums and UnboundedSums satisfy Weak Coherence
Proof (sketch) Voting Since
s1
⊑T
V
oting
N
s
2
always holds, Weak Source-Coherence clearly
holds. For Weak Fact-Coherence, suppose
𝗌𝗋𝖼N(f1)
is weakly less trustworthy than
𝗌𝗋𝖼N(f2)
.
Then there is a bijection
𝜑∶𝗌𝗋𝖼N(f1)→ 𝗌𝗋𝖼N(f2)
, so
|𝗌𝗋𝖼N(f1)|=|𝗌𝗋𝖼N(f2)|
. By definition of
Voting, f
1
≈T
V
oting
N
f
2
. In particular, f
1
≺T
V
oting
N
f
2
.
Sums First, one may adapt definition 5.1 to a numerical variant of a set of facts Y being
weakly less believable than
Y′
, by dropping all references to
𝜌
. We then have an analogue of
Lemma 5.1, and Weak Coherence for Sums follows by an argument similar to the one used
to show Coherence using Lemma 5.1.
UnboundedSums The proof that UnboundedSums satisfies Coherence can be adapted in
a straightforward way to show Weak Coherence.
◻
Proposition 7.2 indicates that Weak Coherence may in fact be too weak to capture the
original intuition behind Coherence—namely, that there should be a mutual dependence
between trustworthy sources and believable facts—since it does not even rule out Vot-
ing. Instead, Weak Coherence can be seen as a simple requirement which only rules out
undesirable behaviour, and complements (strict) Coherence.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 33 of 49 42
Since Monotonicity implies Weak Monotonicity, it is clear that Voting satisfies Weak
Monotonicity. We conjecture that Weak Monotonicity also holds for Sums and Unbounded-
Sums, but this remains to be proven.12
8 Related work
In this section we discuss related work.
Ranking systems. Altman and Tennenholtz [2] initiated axiomatic study of ranking sys-
tems. First we discuss their framework in relation to ours, and then turn to their axioms.
In their framework, a ranking system F maps any (finite) directed graph
G=(V,E)
to a
total preorder
≤F
G
on the vertex set V. In their view this is a variation of the classical social
choice setting, in which the set of voters and alternatives coincide. Nodes
v∈V
“vote" on
their peers in V by a form of approval voting [26]: an edge
v→u
is interpret as a vote for u
from v. A ranking system then outputs a ranking of V, following the general intuition that
the more “votes" v receives (i.e. the more incoming edges), the higher v should rank. As
with the ranking of facts in truth discovery, this does not necessarily mean ranking nodes
simply by the number of votes received, since the quality of the voters should also be taken
in account. For example, a ranking system may prioritise nodes which receive few votes
from highly ranked nodes over those with many votes from lower ranked nodes.
Note that our truth discovery networks N are themselves directed graphs on the ver-
tex set
S∪F∪O
. However, naively applying a ranking system to N directly makes little
sense: sources never receive any “votes", and the resulting ranking includes objects, which
do not need to be ranked in our setting. Perhaps a more sensible approach is to consider the
bipartite graph
GN=(VN,EN)
associated with a network N, where
That is, we take the edges from sources to facts together with the reversal of such edges.
The edges in
GN
have some intuitive interpretation: a source votes for the facts which it
claims are true, and a fact votes for the sources who vouch for it. Any ranking system F
thus gives rise to a truth discovery operator, where
s1
⊑
T
N
s
2
iff
s
1≤
F
GN
s
2
, and similar for
facts.
However, some characteristic aspects of the truth discovery problem are lost in this
translation to ranking systems. Notably, the objects play no role at all in
GN
. Sources and
facts are also treated symmetrically, where they perhaps should not be. For example, a fact
f receiving more claims than g is beneficial for f, all else being equal (see Monotonicity),
but a source s claiming more facts than t does not tell us anything about the relative trust-
worthiness of s and t.
While other choices of
GN
may be possible to alleviate some of these problems, we
believe the truth discovery is sufficiently specialised beyond general graph ranking so that
a bespoke modelling is required to capture its nuances appropriately. Our framework pro-
vides this novel contribution.
V
N=S∪F,EN=
⋃
(s,f)∈N
{(s,f),(f,s)}
.
12 Indeed, we conjectured in Sect. 5 that the stronger axiom (strict) Monotonicity holds for Unbounded-
Sums. As with Monotonicity, experimental evidence from various starting networks N suggests that Weak
Monotonicity is likely to hold.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 34 of 49
In [2], Altman and Tennenholtz also introduce axioms for ranking systems. Their first
set of axioms deal with the transitive effects of voting when the alternatives are the voters
themselves. As mentioned in Sect.4, these axioms provided the inspiration for Coherence.
The core idea is that if the predecessors of a node v are weaker than those of u, then v
should be ranked below u. If v additionally has more predecessors, v should rank strictly
below. Coherence applies this idea both in the direction of sources-to-facts (Fact-Coher-
ence) and from facts-to-sources (Source-Coherence). A notable difference is that we only
consider the case where the number of sources for two facts (or the number of facts, for two
sources) is the same. For example, a source claiming more facts does not give it the strict
boost Transitivity would dictate. Under the mapping
N↦GN
described above, any ranking
system satisfying Transitivity induces a truth discovery operator which satisfies Coherence.
The other axiom in [2] is an independence axiom RIIA (ranked independence of irrel-
evant alternatives), which adapts the classical IIA axiom from social choice theory to the
ranking system setting, although in a different manner to our independence axioms of
Sect.4.4. We describe the axiom in rough terms, deferring to the paper for the technical
details. Suppose the relative ranking of
u1
’s predecessors compared to
u2
’s predecessors
is the same as that of
v1
’s compared to
v2
’s. Then RIIA requires
u1≤u2
iff
v1≤v2
. Infor-
mally, “the relative ranking of two agents must only depend on the pairwise comparison of
the ranks of their predecessors" [2]. While we do not have an analogous axiom, the idea
can be adapted to truth discovery networks. Intuitively, such an axiom would state that the
ranking of two facts depends only on comparisons between their corresponding sources
(and similar for the ranking of sources).
However, the main result of Altman and Tennenholtz is an impossibility: Transitivity is
incompatible with RIIA. Moreover, the result remains true even when restricting to bipar-
tite graphs, such as
GN
described above. Accordingly, we can expect a similar impossibility
result to hold in the truth discovery setting between Coherence and any analogue of RIIA.
PageRank. PageRank [33] is a well-known algorithm for ranking web pages based
on the hyperlink structure of the web, viewed as a directed graph. It has also been stud-
ied through the lens of social choice and characterised axiomatically [1, 40].13 In [1] the
authors propose several invariance axioms, each of which requires that the ranking of pages
is not affected by a certain transformation of the web graph. For example, the axiom Self
Edge says that adding a self loop from a page a to itself does not change the relative rank-
ing of other pages, and results in a strictly positive boost for a (c.f. Monotonicity). How-
ever, if we identify a truth discovery network N with the graph
GN
as described above, most
of the transformations involved do not respect the bipartite, symmetric structure of
GN
.
That is, the transformed graph does not correspond to any
GN′
, for a network
N′
. Conse-
quently, the PageRank axioms have no truth discovery counterpart in our setting. The only
exception is Isomorphism, where the transformation in question is graph isomorphism; this
axiom is analogous to our Symmetry and Isomorphism axioms. However, since PageRank
is similar to the Hubs and Authorities [21] algorithm on which Sums is based—which also
uses the link structure of the web to rank pages—we expect there may be additional axioms
which can be expressed both for general graphs and truth discovery networks, satisfied by
PageRank and Sums. We leave the task of finding such axioms to future work.
13 Wąs and Skibski [40] axiomatise the numerical scores of PageRank, whereas Altman and Tennenholtz
[1] axiomatise the resulting ranking. Moreover, Wąs and Skibski point out that Altman and Tennenholtz
in fact only consider a simplified version of PageRank called Katz prestige, defined only for strongly con-
nected graphs.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 35 of 49 42
9 Conclusion
In this paper we formalised a mathematical framework for truth discovery. While a
number of simplifying assumptions were made compared to the mainstream truth dis-
covery literature, we are able to express several algorithms in the framework. This
provided the setting for the axiomatic method of social choice to be applied. To our
knowledge, this is the first such axiomatic treatment in this context.
It was possible to adapt many axioms from social choice theory and related areas. In
particular, the Transitivity axiom studied in the context of ranking systems [2, 36] took
on new life in the form of Coherence, which we consider a core axiom for TD opera-
tors. We proceeded to establish the differences between our setting and classical social
choice by considering independence axioms. This led to an impossibility result and an
axiomatic characterisation of the baseline Voting method.
On the practical side, we analysed the existing TD algorithm Sums and found that,
surprisingly, it fails PCI. This is a serious issue for Sums which has not been dis-
cussed in the literature to date, and its discovery here highlights the benefits of the
axiomatic method. To resolve this, we suggested a modification to Sums—which we
call UnboundedSums—for which PCI is satisfied. However, while UnboundedSums
resolves axiomatic problems with Sums, it may introduce computational difficulties
(since the numeric scores involved grow without bound). We leave further investiga-
tion of such issues to future work.
A restriction of our analysis is that only one ‘real-world’ algorithm was considered.
Further axiomatic analysis of algorithms provides a deeper understanding of how algo-
rithms operate on an intuitive level, but is made difficult by the complexity of the state-
of-the-art truth discovery methods. New techniques for establishing the satisfaction (or
otherwise) of axioms would be helpful in this regard.
There is also scope for extensions to our model of truth discovery in the framework
itself. For example, even in the variable domain setting of Sect.6, we make the some-
what simplistic assumption that there are only finitely many possible facts for sources
to claim. This effectively means we can only consider categorical values; modelling an
object whose domain is the set of real numbers, for example, is not straightforward in
our framework.
Next, our model does not account for any associations or constraints between
objects, whereas in reality the belief in a fact for one object may strengthen or weaken
our belief in other facts for related objects. These types of constraints or correlations
have been studied both on the theoretical side (e.g. in judgment aggregation) and prac-
tical side in truth discovery [44].
The axioms can also be further refined to relax some of the simplifying assumptions
we make regarding source attitudes; e.g. that they do not collude or attempt to manipu-
late. Most notably, Monotonicity should be weakened to account for such sources.
Finally, it may be argued that truth discovery as formulated in this paper risks sim-
ply to find consensus among sources, rather than the truth. To remedy this, the frame-
work could be extended to model the possible states of the world and thus the ground
truth (c.f. [32]). Upon doing so one could investigate how well, and under what condi-
tions, an operator is able to recover the truth from a TD network. Such truth-tracking
methods have also been studied in judgment aggregation and belief fusion [16, 20].
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 36 of 49
Proofs
Proof oftheorem4.1
The following lemma is required before the proof.
Lemma A.1 Suppose a network
N=(V,E)
contains claims only for a single object
o∈O
;
that is, there exists
o∈O
such that
(s,f)∈E
implies
objN(f)=o
for all
s∈S,f∈F
. Then
for any Symmetric operator T and
f1,f2∈F
,
|𝗌𝗋𝖼N(f1)|=|𝗌𝗋𝖼N(f2)|>0
implies f
1
≈
T
N
f
2
.
Proof Suppose N has the stated property, T satisfies symmetry, and
|𝗌𝗋𝖼N(f1)|=|𝗌𝗋𝖼N(f2)|>0
. Then there is a bijection
𝜑∶𝗌𝗋𝖼N(f1)→ 𝗌𝗋𝖼N(f2)
. Note that since
f1
and
f2
are for the same object no source can claim both facts, i.e.
𝗌𝗋𝖼N(f1)∩𝗌𝗋𝖼N(f2)=�
.
Define a permutation
𝜋
by
and
𝜋(o)=o
for all
o∈O
. That is,
𝜋
swaps facts
f1
and
f2
, and swaps the sources of
f1
with their counterparts in
f2
. Note that
𝜋=𝜋
−1
.
Write
N�=𝜋(N)
. We claim that
N�=N
. Write
E,E′
for the edges in N and
N′
respec-
tively. First we will show
E⊆E′
. Suppose
(s,f)∈E
. There are three cases.
Case 1
f=f1
. Here we have
(s,f1)∈E
, so
s∈𝗌𝗋𝖼N(f1)
. Consequently
𝜋(s)=𝜑(s)∈𝗌𝗋𝖼N(f2)
, i.e.
(𝜋(s),f2)∈E
. By the definition of a graph isomorphism we
get
(𝜋(𝜋(
s
))
,
𝜋(
f
2)) ∈
E
�
. Noting that
𝜋(f2)=f1=f
and
𝜋(𝜋(s)) = s
(since
𝜋=𝜋
−1
), we
have
(s,f)∈E�
as desired.
Case 2
f=f2
. Similar to the above case, here we have
s∈𝗌𝗋𝖼N(f2)
and so
𝜋(s)=𝜑−1(s)∈𝗌𝗋𝖼N(f1)
, i.e.
(𝜋(s),f1)∈E
. As before, applying the definition of a graph
isomorphism and using
𝜋=𝜋
−1
, we get
(s,f)∈E�
.
Case 3
f∉{f1,f2}
. By hypothesis f relates to the same object as
f1
and
f2
. This
means
s∉𝗌𝗋𝖼N(f1)
and
s∉𝗌𝗋𝖼N(f2)
, since otherwise s would make claims for mul-
tiple facts for a single object. Hence we have
𝜋(s)=s
and
𝜋(f)=f
. This means
(s,f)=(𝜋(s),𝜋(f)) ∈ E�
as required.
To complete the claim
E⊆E′
, suppose
(f,o)∈E
. There are again three cases:
f=f1
,
f=f2
, or
f∉{f1,f2}
. In each case the definition of
𝜋
and
𝜋(N)
easily yield
(f,o)∈E�
.
Hence
E⊆E′
.
Now for the reverse direction: we must show
E′⊆E
. Let
(x,y)∈E�
. By definition of
a graph isomorphism, we have
(𝜋−1(x),𝜋−1(y)) ∈ E
. Using
𝜋
−1=𝜋
and the first part we
get
(𝜋(x),𝜋(y)) = (𝜋−1(x),𝜋−1(y)) ∈ E⊆E�
. The definition of a graph isomorphism then
yields
(x,y)∈E
and so
E′⊆E
. Hence
E=E�
and
N=N�
.
𝜋
(s)=
⎧
⎪
⎨
⎪
⎩
𝜑(s)if s∈𝗌𝗋𝖼N(f1
)
𝜑−1(s)if s∈𝗌𝗋𝖼N(f2
)
sotherwise
𝜋
(f)=
⎧
⎪
⎨
⎪
⎩
f2if f=f1
f2if f=f2
fotherwise
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 37 of 49 42
To conclude the proof, we apply Symmetry of T to get
and so f
1
≈
T
N
f
2
as required.
◻
Proof (Theorem 4.1) Suppose T is an operator satisfying Symmetry, Monotonic-
ity and POI. Let
N∈N
,
o∈O
and f
1
,f
2
∈𝗈𝖻𝗃
−1
N
(o
)
. We need to show that f
1
⪯
T
N
f
2
i ff
|𝗌𝗋𝖼N(f1)|≤|𝗌𝗋𝖼N(f2)|
.
Let
N′
be the network obtained from N by removing all claims for facts other than those
for object o; that is,
N�=(V,E�)
where E is the set of edges in N and
Note that the fact-object affiliations are the same in
N′
as in N, and the set of sources for
each fact in
𝗈𝖻𝗃−1
N
(o
)
is the same. Therefore POI applies, and it is sufficient to show that
f
1
⪯
T
N
�f
2
iff
|𝗌𝗋𝖼N
�
(f1)|≤|𝗌𝗋𝖼N
�
(f2)|
.
First suppose
|𝗌𝗋𝖼N
�
(f1)|≤|𝗌𝗋𝖼N
�
(f2)|
. If
|𝗌𝗋𝖼N
�
(f1)|=|𝗌𝗋𝖼N
�
(f2)|
, then we
have f
1
≈
T
N
�f
2
by Symmetry and Lemma A.1; in particular f
1
⪯
T
N
�f
2
. Other wise
|𝗌𝗋𝖼N
�
(f2)|−|𝗌𝗋𝖼N
�
(f1)|=k>0
. Removing k sources from
f2
to obtain a new network
N′′
,
we can apply the lemma to get f
1
≈
T
N
�� f
2
. We may then add these sources back to obtain
N′
again; k applications of Monotonicity then give f
1
≺
T
N
′f
2
as required.
To complete the proof we show that f
1
⪯
T
N
�f
2
implies
|𝗌𝗋𝖼N
�
(f1)|≤|𝗌𝗋𝖼N
�
(f2)|
. Indeed,
suppose f
1
⪯
T
N
�f
2
but
|𝗌𝗋𝖼N
�
(f1)>|𝗌𝗋𝖼N
�
(f2)|
. Then the argument above gives f
1
≻
T
N
′f
2
,
which is clearly a contradiction. Hence the proof is complete.
◻
Proof oftheorem4.3
The proof of this theorem is similar in spirit to that of Theorem4.1. Like in that case, a
preliminary result is required first.
Lemma A.2 Let N be a network and
f1,f2∈F
. Write
o1=𝗈𝖻𝗃N(f1)
,
o2=𝗈𝖻𝗃N(f2)
. Suppose
N has the following properties:
1. There is
o∗∈O⧵{o1,o2}
such that
f∈F⧵{f1,f2}
⟹
𝗈𝖻𝗃N(f)=o∗
; and
2.
𝗌𝗋𝖼N(
f
)=�
for all
f∈F⧵{f1,f2}
.
Then for any operator T satisfying Symmetry,
|𝗌𝗋𝖼N(f1)|=|𝗌𝗋𝖼N(f2)|
implies f
1
≈
T
N
f
2
.
Proof The proof is similar to that of Lemma A.1. Suppose
|srcN(f1)|=|𝗌𝗋𝖼N(f2)|
. Write
f1⪯
T
Nf2⟺𝜋(f1)⪯
T
N�𝜋(f2
)
⟺f2⪯T
N�f1
⟺f
2
⪯T
N
f
1
E�
=(E∩(S×𝗈𝖻𝗃
−1
N
(o))) ∪ (E∩(F×O
))
Q
1
=𝗌𝗋𝖼
N
(f
1
)⧵𝗌𝗋𝖼
N
(f
2
)
Q2
=𝗌𝗋𝖼
N
(f
2
)⧵𝗌𝗋𝖼
N
(f
1)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 38 of 49
Then
|Q1|=|Q2|
, so there exists a bijection
𝜑∶Q1→Q2
. Define a permutation
𝜋
as
follows:
That is,
𝜋
swaps
f1
and
f2
, swaps
o1
and
o2
, and swaps sources in
Q1
with their counter-
parts in
Q2
. Note that
𝜋=𝜋
−1
. Write
N�=𝜋(N)
. We claim that
N�=N
. Write
E,E′
for the
edges in N and
N′
respectively. First we show that
E⊆E′
. For this, first suppose
(s,f)∈E
for some
s∈S
,
f∈F
. By definition of E, either
f=f1
or
f=f2
.
Case 1
f=f1
. Here
𝜋(f)=f2
, and we have either
s∈Q1
or
s∈𝗌𝗋𝖼N(f1)∩𝗌𝗋𝖼N(f2)
.
In the first case,
𝜋(s)=𝜑(s)∈Q2⊆𝗌𝗋𝖼N(f2)=𝗌𝗋𝖼N(𝜋(f))
. In the second case
𝜋(s)=s∈𝗌𝗋𝖼N(f2)=𝗌𝗋𝖼N(𝜋(f))
. In either case,
(𝜋(s),𝜋(f)) ∈ E
.
Applying the definition of a graph isomorphism we get
(𝜋(𝜋(s)),𝜋(𝜋(f))) ∈ E�
. But
𝜋=𝜋
−1
, so this means
(s,f)∈E�
as desired.
Case 2
f=f2
. This case is similar. Here
𝜋(f)=f1
. If
s∈Q2
, then
𝜋(s)=𝜑−1(s)∈Q1⊆𝗌𝗋𝖼N(f1)=𝗌𝗋𝖼N(𝜋(f))
. Ot herwise
s∈𝗌𝗋𝖼N(f1)∩𝗌𝗋𝖼N(f2)
and
𝜋(s)=s∈𝗌𝗋𝖼N(f1)=𝗌𝗋𝖼N(𝜋(f))
. Again, we have
(𝜋(s),𝜋(f)) ∈ E
in either case, so
(s,f)∈E�
.
Note that these two cases cover all possibilities since by hypothesis
𝗌𝗋𝖼N(f)=�
if
f∉{f1,f2}
.
Next, suppose
(f,o)∈E
. If
f=f1
then
o=o1
, so
(𝜋(f),𝜋(o))=(f2,o2)∈E
. Similarly if
f=f2
then
o=o2
and
(𝜋(f),𝜋(o))=(f1,o1)∈E
. If
f∉{f1,f2}
then
𝜋(f)=f
and
o=o∗
,
so
𝜋(o)=o
. We see that in all cases,
(𝜋(f),𝜋(f)) ∈ E
. Applying the same argument as in
case 1 above, we see that
(f,o)∈E�
. This shows
E⊆E′
.
To complete the claim that
N=N�
we must show
E′⊆E
. This can be shown using the
same argument used in Lemma A.1. Indeed, suppose
(x,y)∈E�
. Then by definition of a
graph isomorphism,
(𝜋−1(x),𝜋−1(y)) ∈ E
. Using the fact that
𝜋=𝜋
−1
and
E⊆E′
we get
(𝜋(x),𝜋(y)) ∈ E�
, which then yields
(x,y)∈E
as required. Hence
E=E�
and
N=N�
.
Finally, using Symmetry of T we have
𝜋(s)=
⎧
⎪
⎨
⎪
⎩
𝜑(s)if s∈Q1
𝜑−1(s)if s∈Q2
s
otherwise
𝜋(f)=
⎧
⎪
⎨
⎪
⎩
f2if f=f1
f1if f=f2
fotherwise
𝜋
(o)=
⎧
⎪
⎨
⎪
⎩
o2if o=o1
o1if o=o2
ootherwise
f1⪯
T
Nf2⟺𝜋(f1)⪯
T
𝜋(N)𝜋(f2
)
⟺f2⪯T
N�f1
⟺f
2
⪯T
N
f
1
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 39 of 49 42
Consequently f
1≈T
N
f
2
.
◻
Proof (Theorem4.3) The ‘if’ direction is clear since Voting satisfies Strong Independence,
Monotonicity and Symmetry (see Theorem5.1). For the other direction, suppose T satis-
fies the stated axioms. Let N be a network and
f1,f2∈F
. We will construct a network
N′
where all claims for facts other than
f1,f2
are removed, and these facts are grouped under a
single object. To do so, let
o1=𝗈𝖻𝗃N(f1)
,
o2=𝗈𝖻𝗃N(f2)
and take
o∗∈O⧵{o1,o2}
. Define
an edge set
E′
by
Then let
N′
be the network with edge set
E′
. Note that
𝗌𝗋𝖼N
�
(fj)=𝗌𝗋𝖼N(fj)
. By Strong Inde-
pendence it is therefore sufficient to show that f
1
⪯
T
N
�f
2
iff
|srcN
�
(f1)|≤|𝗌𝗋𝖼N
�
(f2)|
. Note
also that
N′
satisfies the hypothesis of Lemma A.2.
Now, suppose
|𝗌𝗋𝖼N
�
(f1)|≤|𝗌𝗋𝖼N
�
(f2)|
. If
|𝗌𝗋𝖼N
�
(f1)|=|𝗌𝗋𝖼N
�
(f2)|
then by Lemma A.2
f
1
≈
T
N
�f
2
, and in particular f
1
⪯
T
N
�f
2
.
Otherwise,
|𝗌𝗋𝖼N
�
(f2)|−|𝗌𝗋𝖼N
�
(f1)|=k>0
. Consider
N′′
where k sources from
𝗌𝗋𝖼N
�
(f2)
are removed, and all other claims remain. By the lemma, f
1
≈
T
N
�� f
2
. Applying Monotonicity
k times we can produce
N′
from
N′′
and get f
1
≺
T
N
′f
2
as desired.
For the other implication, suppose f
1
⪯
T
N
�f
2
and, for contradiction,
|𝗌𝗋𝖼N
�
(f1)|>|𝗌𝗋𝖼N
�
(f2)|
. Applying Monotonicity again as above gives f
1
≻
T
N
′f
2
and the
required contradiction.
◻
Proof oftheorem5.1
Proof We will show that Voting satisfies Symmetry, Unanimity, Groundedness, Monoto-
nicity, POI, Strong Independence and PCI, and that Coherence is not satisfied. For Symme-
try and PCI we use the (stronger) numerical variants numerical Symmetry and numerical
PCI, introduced in Sect.5.2. T will denote the (numerical) Voting operator in what follows.
Symmetry. Suppose N and
𝜋(N)
are equivalent networks. Let
f∈F
. By definition of
equivalent networks we have
s∈𝗌𝗋𝖼N(f)
iff
𝜋(s)∈𝗌𝗋𝖼𝜋(N)(𝜋(f))
for all
s∈S
. Consequently
𝜋
restricted to
𝗌𝗋𝖼N(f)
is a bijection into
𝗌𝗋𝖼𝜋(N)(𝜋(f))
, and hence
Now let
s∈S
. Clearly we have
TN(s)=1=T𝜋(N)(
𝜋
(s))
. Hence T satisfies numerical Sym-
metry and therefore Symmetry.
Unanimity and Groundedness. Suppose
N∈N
and
f∈F
. If
𝗌𝗋𝖼N(f)=S
then for any
g∈F
,
so
g⪯T
N
f and Unanimity is satisfied. If instead
𝗌𝗋𝖼N(
f
)=�
, we have
so f⪯
T
Ng
and Groundedness is satisfied.
(
s,f
)∈
E
�
⟺f
∈{
f1,f2
}
and s
∈
𝗌𝗋𝖼N
(
f
)
(f,o)∈E
�⟺
(f∈{f1,f2}and o=𝗈𝖻𝗃N(f)) or (f∉{f1,f2}and o=o
∗
)
TN(f)=|𝗌𝗋𝖼N(f)|=|𝗌𝗋𝖼𝜋(N)(𝜋(f))|=T𝜋(N)(𝜋(f))
TN(g)=|𝗌𝗋𝖼N(g)|≤|S|=|𝗌𝗋𝖼N(f)|=TN(f)
TN(g)=|𝗌𝗋𝖼N(g)|≥0=|𝗌𝗋𝖼N(f)|=TN(f)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 40 of 49
Monotonicity. Let
N,N′
,s
and f be as given in the statement of Monotonicity. It is clear
that
|𝗌𝗋𝖼N
�
(f)|=|𝗌𝗋𝖼N(f)|+1
. Also, for any
g∈F
,
g
≠
f
, the set of sources in
N′
is the
same as in N but with s possibly removed. Hence
|𝗌𝗋𝖼N
�
(g)|≤|𝗌𝗋𝖼N(g)
. Therefore
g⪯T
N
f
implies
and so
g
≺
T
N
′f as required.
Independence axioms. Next we show Strong Independence, which implies POI. Sup-
pose
N1,N2∈N
,
f1,f2∈F
and
𝗌𝗋𝖼
N
1(f
j
)=𝗌𝗋𝖼
N
2(f
j
)
for each
j∈{1, 2}
. Clearly we have
Consequently
as required for Strong Independence.
For PCI we proceed as with Symmetry by showing numerical PCI. Let
N1,N2
have a
common connected component G. Let
f∈G∩F
. By definition of a connected compo-
nent,
s∈𝗌𝗋𝖼
N
1(f)
iff
s∈𝗌𝗋𝖼
N
2(f)
, so
𝗌𝗋𝖼
N
1(f)=𝗌𝗋𝖼
N
2(f)
. Hence
For
s∈G∩S
, we trivially have
TN1(s)=1=TN1(s)
. Hence numerical PCI is satisfied.
Coherence. The violation of Coherence follows from Theorem 4.2, since we have
already shown that Symmetry, Monotonicity and POI are satisfied.
◻
Proof oflemma 5.2
Proof The first statement follows easily from the definition of the limit. We shall prove
only the second one.
First we prove the ‘if’ direction. Write
D=T∗
N(f1)−T∗
n(f2)
. We need to show that
D<0
.
Write
dn=Tn
N(f1)−Tn
N(f2)
so that
D=limn→∞dn
. Take
𝜀=𝜌∕2>0
. Then for sufficiently
large n we have
dn≤−𝜌∕2<0
. Taking
n→∞
, we have
D=limn→∞dn≤−𝜌∕2<0
as
required.
For the ‘only if’ direction, suppose
D<0
. Let
𝜌=−D
. Then for any
𝜀>0
, by the
definition of the limit there is
K∈ℕ
such that
|dn−D|<𝜀
for
n≥K
; in particular,
dn<𝜀+D=𝜀−𝜌
as required.
◻
Proof oftheorem5.2
The following results will be helpful to simplify the Proof of Theorem5.2.
Lemma A.3
𝗇𝗈𝗋𝗆
has the following properties.
|𝗌𝗋𝖼N
�
(g)|
≤
|𝗌𝗋𝖼N(g)|
≤
|𝗌𝗋𝖼N(f)|<|𝗌𝗋𝖼N
�
(f)|
T
N
1(f
j
)=|𝗌𝗋𝖼
N
1(f
j
)|=|𝗌𝗋𝖼
N
2(f
j
)|=T
N
2(f
j
)
f1⪯
T
N1f2⟺TN1(f1)≤TN1(f2
)
⟺TN2(f1)≤TN2(f2
)
⟺f1⪯T
N2
f2
T
N
1(f)=|𝗌𝗋𝖼
N
1(f)|=|𝗌𝗋𝖼
N
2(f)|=T
N
2(f)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 41 of 49 42
1.
𝗇𝗈𝗋𝗆
preserves numerical Symmetry, in the sense that
𝗇𝗈𝗋𝗆(T)
satisfies numerical Sym-
metry whenever T does.
2.
𝗇𝗈𝗋𝗆
leaves rankings unchanged, in the following sense. For
T∈TNum
,
N∈N
,
s1,s2∈S
,
f1,f2∈F
,
Proof For part (i), suppose T satisfies numerical Symmetry, and write
T�=U(T)
. Let N
and
𝜋(N)
be equivalent networks. First note that
where the second equality follows since
𝜋
restricted to
S
is a surjection into
S
by the defi-
nition of equivalent networks. If this maximum is 0, then
T�
N(
s
)=
0
=
T
�
𝜋(N)(
s
)
for all
s∈S
.
Otherwise,
One can show that
T�
N(f)=T�
𝜋(N)(
𝜋
(f))
by an identical argument. Hence
T�=U(T)
satisfies
numerical Symmetry also.
Now we prove part (ii). First suppose
s1
⊑
T
N
s
2
. Write
T�=𝗇𝗈𝗋𝗆(T)
. We have
T�
N
(x)=𝛼T
N
(x
)
for some
𝛼≥0
and all
x∈S
(either
𝛼=1∕maxx∈S|TN(x)|
or
𝛼=0
). We
therefore have
as desired.
Now suppose
s1
⊑T
′
N
s
2
, i.e.
𝛼TN(s1)
≤
𝛼TN(s2)
. If
𝛼>0
then dividing by
𝛼
readily gives
s1
⊑
T
N
s
2
. Otherwise,
𝛼=0
. This means
maxx∈S|TN(x)|=0
, and thus
TN(x)=0
for all
x∈S
. In particular
TN(s1)=0
≤
0=TN(s2)
so
s1
⊑
T
N
s
2
.
The second statement regarding fact ranking may be shown using an identical argu-
ment.
◻
Corollary A.1
𝗇𝗈𝗋𝗆
preserves Coherence, Unanimity, Groundedness and PCI.
Proof (Theorem5.2) Throughout this proof,
(Tn)n∈ℕ
will denote the iterative operator Sums,
T∗
will denote the limit operator, and
U=𝗇𝗈𝗋𝗆◦USums
will denote the update function for
Sums.
Coherence. Source-Coherence was shown in the body of the paper. The proof that Fact-
Coherence is satisfied is similar, and uses Lemma 5.3. Suppose
N∈N
,
T=Tn
for some
n∈ℕ
,
𝜀,𝜌>0
, and
𝗌𝗋𝖼N(f1)
is
(𝜀,𝜌)
-less trustworthy than
𝗌𝗋𝖼N(f2)
with respect to N and
T
s
1⊑T
Ns2⟺s1⊑
𝗇𝗈𝗋𝗆(T)
Ns
2
f
1
⪯T
N
f
2
⟺f
1
⪯𝗇𝗈𝗋𝗆(T)
N
f
2
max
x∈S|T
N
(x)|=max
x∈S|T
𝜋(N)
(𝜋(x))|=max
x∈S|T
𝜋(N)
(x)|
T
�
N(s)=
TN(s)
max
x∈S|
TN(x)
|
=
T
𝜋(N)
(𝜋(s))
max
x∈S|
T𝜋(N)(x)
|
=T�
𝜋(N)(𝜋(s
))
s
1⊑
T
Ns2⟹TN(s1)≤TN(s2)
⟹𝛼TN(s1)≤𝛼TN(s2
)
⟹T�
N(s1)≤T�
N(s2)
⟹s
1
⊑T�
N
s
2
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 42 of 49
under a bijection
𝜑
, where
T=U(T)
. Let
s∈𝗌𝗋𝖼N(f1)
be such that
TN(
s
)−
T
N(𝜑(
s
))
≤
𝜀−𝜌
.
Write
T�=USums(T)
so that
T=𝗇𝗈𝗋𝗆(T�)
, and set
We may assume without loss of generality that
𝜀< 1
|
S
|𝜌
. Note that for
s∈S
,
TN
(s)=𝛼T
�
N
(s
)
and therefore
T
�
N
(s)=
1
𝛼
TN(s
)
. Writing
and applying a similar argument as for showing Source-Coherence, we find
Now we need to bound
𝛽∕𝛼
from below. Since we assume
T=Tn
for some
n∈ℕ
, for any
y∈F
we have
Therefore
Next, we claim there is some fact
f∈
F with
TN
(
f)≥1∕
2
and
𝗌𝗋𝖼N
(
f)≠
�
. Indeed, if
T=T1=Tfixed
then take any fact with at least one associated source.14 Otherwise, since
we assume not all scores are 0 in the limit, there is some
f
with
TN
(
f)=
1
due to the appli-
cation of
𝗇𝗈𝗋𝗆
. Clearly
𝗌𝗋𝖼N(
f)
≠
�
, since we would have
TN(
f)=0
otherwise.
Let
x
∈𝗌𝗋𝖼
N
(
f
)
. Then
𝛼
=
1
max
x∈S|
T�
N(x)
|
𝛽
=
1
max
y∈
F
|
T�
N(y)
|
T
N(f1)−
TN(f2)=
s∈𝗌𝗋𝖼N(f1)
T�
N(s)−T�
N((s))
=
s∈𝗌𝗋𝖼N(f1)
TN(s)−
TN((s))
=
TN(s)−
TN((s))
≤−
+
s∈𝗌𝗋𝖼N(f1)⧵{s}
TN(s)−
TN((s))
≤
≤
⋅
S
−
<0
|
T
�
N(y)
|
=
∑
s∈𝗌𝗋𝖼N(y)
T
�
N(s)
≤|
F
|
≤
|
𝗌𝗋𝖼N(y)
|
⋅
|
F
|
≤
|
S
|
⋅
|
F
|
𝛽
≥
1
|
S
|
⋅
|
F
|
14 Note that this is always possible since a truth discovery network contains at least one claim by definition.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 43 of 49 42
This means
and so, finally,
Combined with what was shown before, this means
and Fact-Coherence follows from Lemma 5.3.
Symmetry. As a consequence of Lemma 5.4, to show Symmetry it is sufficient to show
that
Tfixed
satisfies numerical Symmetry, and that
U=𝗇𝗈𝗋𝗆◦USums
preserves numerical
Symmetry. Since
Tfixed
is constant with value 1/2, it is clear that numerical Symmetry is
satisfied. Moreover, Lemma A.3 part (i) already shows that
𝗇𝗈𝗋𝗆
preserves numerical Sym-
metry, so we only need to show that
USums
does.
To that end, suppose
T∈TNum
satisfies numerical symmetry, and write
T�=USums(T)
.
Let N and
𝜋(N)
be equivalent networks and
s∈S
. Then
Note that
f∈𝖿 𝖺𝖼𝗍𝗌N(s)
iff
𝜋(f)∈𝖿 𝖺𝖼𝗍𝗌𝜋(N)(
𝜋
(s))
. Rephrased slightly, we have
y∈𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(s))
iff
𝜋−1(
y
)∈
𝖿 𝖺𝖼𝗍𝗌
N(
s
)
. Hence we may make a ‘substitution‘
f=𝜋−1(y)
and sum over
𝖿 𝖺𝖼𝗍𝗌N(s)
, i.e.
Applying numerical symmetry for T, we get
Following the same tactic, one may also show that
T�
𝜋(N)(
𝜋
(
f
)) =
T
�
N(
f
)
for all
f∈F
.
Hence
USums
preserves numerical Symmetry, and we are done.
Unanimity and Groundedness.
Unanimity and Groundedness can be proved together using Lemma 5.5 and corollary
A.1. By these results it is sufficient that
Tfixed
satisfies Unanimity and Groundedness—this
is trivial—and that
USums
preserves them.
|
T�
N(x)|=T�
N(x)= TN(
f)
≥1∕2
+
∑
f∈𝖿𝖺𝖼𝗍𝗌N(x)⧵{
f}
TN(f)
≥0
≥
1
2
1
𝛼
=max
x∈S|
T�
N(x)
|
≥
|
T�
N(x)
|
≥
1
2
𝛽
𝛼
≥
1
|
S
|
⋅
|
F
|
⋅
1
2
T
N(f1)−
TN(f2)≤
1
2
⋅
|
S
|
⋅
|
F
|(|
S
|
𝜀−𝜌
)
T
�
𝜋(N)(𝜋(s)) =
∑
y∈𝖿𝖺𝖼𝗍𝗌
𝜋(N)
(𝜋(s))
T𝜋(N)(y
)
T
�
𝜋(N)(𝜋(s)) =
∑
f∈𝖿𝖺𝖼𝗍𝗌N(s)
T𝜋(N)(𝜋(f
))
T
�
𝜋(N)(𝜋(s)) =
∑
f∈𝖿𝖺𝖼𝗍𝗌N(s)
TN(f
)
=T�
N
(s)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 44 of 49
Suppose T satisfies Unanimity and Groundedness and write
T�=USums(T)
. Assume
without loss of generality that
T=Tn
for some
n∈ℕ
so that
T′
N
≥
0
. Suppose
N∈N
,
f∈F
and that
𝗌𝗋𝖼N(
f
)=S
. Let
g∈F
. We must show that
g
⪯
T�
N
f . We have
i.e.
g
⪯T
�
N
f as required for Unanimity. For Groundedness, suppose
𝗌𝗋𝖼N(f)=�
. We must
show f⪯
T�
Ng
. Indeed, the sum in the expression for
T�
N(
f
)
is taken over the empty set,
which by convention is 0. Since
T�
N
(g)≥
0
, we are done.
◻
Proof oftheorem5.3
Proof Here we give only the technical details for the argument showing SC-Voting satisfies
Symmetry, since the results for the other axioms were given in the main text.
Symmetry. Since Voting satisfies Symmetry, it is clear that f
1
⪯T
SCV
N
f
2
i ff
𝜋
(f1)⪯
T
SCV
𝜋(N)
𝜋(f2
)
for any equivalent networks N and
𝜋(N)
. We need to show that
s1
⊑T
SCV
N
s
2
iff
𝜋
(s1)⊑T
SCV
𝜋(N)
𝜋(s2
)
.
First we will show that
⊲N
and
⊲𝜋(N)
have a similar symmetry property:
s1⊲Ns2
iff
𝜋(s1)⊲𝜋(N)𝜋(s2)
. Indeed, suppose
s1⊲Ns2
. Then there is a bijection
𝜑∶𝖿 𝖺𝖼𝗍𝗌N(s1)→ 𝖿 𝖺𝖼𝗍𝗌N(s2)
with f⪯T
SCV
N
𝜑(f
)
, and there is some
f
with
f≺T
SCV
N
𝜑(
f
)
.
It can be seen that
𝜋
restricted to
𝖿 𝖺𝖼𝗍𝗌N(si)
is a bijection into
𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(si))
. Let
𝜋1
and
𝜋2
denote these restrictions for
i=1, 2
respectively. Set
𝜃
=𝜋
2
◦𝜑◦𝜋
−1
1
, so that
𝜃
maps
𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(s1))
into
𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(s2))
. As a composition of bijections,
𝜃
is itself bijective.
Let
g∈𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(s1))
. Write f=𝜋
−1
1
(g)∈𝖿 𝖺𝖼𝗍𝗌
N
(s
1)
. By the property of
𝜑
, we have
By the symmetry property of the fact-ranking (which follows from symmetry of Voting),
we can apply
𝜋
to the above to get
Since
f∈𝖿 𝖺𝖼𝗍𝗌N(s1)
and
𝜑(f)∈𝖿 𝖺𝖼𝗍𝗌N(s2)
, we have
𝜋(f)=𝜋1(f)
and
𝜋(𝜑(f)) = 𝜋2(𝜑(f))
.
Using this fact in the above inequality and recalling
f=𝜋−1(g)
we get
i.e.
g
⪯T
SCV
𝜋(N)
𝜃(g
)
. Applying the same argument with
g
=𝜋
−
1
1
(
f
)
we get
g
≺T
SCV
𝜋(N)
𝜃(g
)
.
This shows that
𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(s1))
is less believable than
𝖿 𝖺𝖼𝗍𝗌𝜋(N)(𝜋(s2))
with respect
to SC-Voting (whose fact-ranking coincides with Voting) in
𝜋(N)
under
𝜃
. Hence
𝜋(s1)⊲𝜋(N)𝜋(s2)
.
We have shown
s1⊲Ns2
⟹
𝜋(s1)⊲𝜋(N)𝜋(s2)
. For the converse implication, apply
the same argument starting from
𝜋(s1)⊲𝜋(N)𝜋(s2)
with the
𝜋
−1
.
Next, we note that for
i=1, 2
and any
t∈S
,
T�
N(g)=
∑
s∈𝗌𝗋𝖼N(g)
T
�
N(s)≤
∑
s∈
S
T
�
N(s)=T
�
N(f
)
f⪯
TSCV
N
𝜑(f
)
𝜋
(f)⪯
T
SCV
𝜋(N)
𝜋(𝜑(f
))
g
=𝜋
1
(f)=𝜋(f)⪯
T
SCV
𝜋(N)
𝜋(𝜑(f)) = 𝜋
2
(𝜑(f)) = 𝜋
2
(𝜑(𝜋
−
1
1
(g))) = 𝜃(g
)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 45 of 49 42
Consequently
𝜋
restricted to
WN(si)
is a bijection into
W𝜋(N)(𝜋(si))
, which means
|WN(si)|=|W𝜋(N)(𝜋(si))|
. Finally, this means
as required for Symmetry.
◻
Proof oftheorem5.5
Proof Here we show that UnboundedSums satisfies Symmetry, PCI, Unanimity and
Groundedness, since the other axioms were dealt with in the main body of the paper.
Throughout the proof, let
(Tn)n∈ℕ
denote UnboundedSums,
T∗
denote the ordinal limit of
UnboundedSums, and for a network N let
JN
be as in Theorem5.4. Then the rankings in N
induced by
Tn
for
n≥JN
are the same as
T∗
.
Symmetry. In the Proof of Theorem5.2, we saw that the update function
USums
preserves
numerical Symmetry, in the sense that if T satisfies numerical Symmetry then
USums(T)
does also. Since it is clear that the prior operator for UnboundedSums satisfies numerical
Symmetry,
Tn
satisfies numerical Symmetry and consequently Symmetry for all
n∈ℕ
.
Now, let N and
𝜋(N)
be equivalent networks. Let
J,J�∈ℕ
be such that
T∗(N)
and
T∗(𝜋(N))
are given by
TJ
N
and
T
J
�
𝜋(N)
respectively and take
n≥max{J,J�}
. For
s1,s2∈S
we
have by Symmetry of
Tn
,
as required for Symmetry. Using an identical argument, one can show that f
1
⪯
T∗
N
f
2
i ff
𝜋
(f)⪯
T∗
𝜋(N)
𝜋(f2
)
. Hence
T∗
satisfies Symmetry.
PCI. As with Symmetry, we will show that
Tn
satisfies numerical PCI, and consequently
PCI, for all
n∈ℕ
. Let
N1,N2
be networks with a common connected component G. Let
s∈G∩S
and
f∈G∩F
. Note that
𝖿 𝖺𝖼𝗍𝗌N1(s)=𝖿 𝖺𝖼𝗍𝗌N2(s)
and
𝗌𝗋𝖼N1(f)=𝗌𝗋𝖼N2(f)
since
by definition a source is connected to its facts and vice versa. For
n=1
we have
so
T1
has numerical PCI. Supposing
Tn
has numerical PCI for some
n∈ℕ
, we have
t∈W
N
(s
i
)
⟺
t⊲
N
s
i
⟺𝜋(t)⊲𝜋(N)𝜋(si)
⟺𝜋(t)∈W
𝜋(N)
(𝜋(s
i))
s
1⊑T
SCV
Ns2⟺
|
WN(s1)
|
≤
|
WN(s2)
|
⟺
|
W𝜋(N)(𝜋(s1))
|
≤
|
W𝜋(N)(𝜋(s2))
|
⟺𝜋(s1)⊑TSCV
𝜋(N)
𝜋(s2)
s
1⊑
T∗
Ns2⟺s1⊑
Tn
Ns2
⟺𝜋(s1)⊑Tn
𝜋(N)𝜋(s2
)
⟺𝜋(s1)⊑T∗
𝜋(N)
𝜋(s2
)
T1
N1
(s)=1=T
1
N2
(s)
T
1
N1
(f)=
|
𝗌𝗋𝖼N
1
(f)
|
=
|
𝗌𝗋𝖼N
2
(f)
|
=T1
N2
(f
)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 46 of 49
and similarly
Hence, by induction,
Tn
has numerical PCI for all
n∈ℕ
, and we are done.
Unanimity and Groundedness. For Unanimity, suppose
𝗌𝗋𝖼N(
f
)=S
. For any
g∈F
and
n∈ℕ
we have
so
g
⪯
Tn
N
f for all
n∈ℕ
. Since the ranking of
T∗
corresponds to
Tn
for large n, we have
g
⪯
T∗
N
f as required
For Groundedness, suppose
𝗌𝗋𝖼N(f)=�
. Then
Tn
N(f)=0
for all
n∈ℕ
. For any
g∈F
,
we have
Tn
N(g)≥0=Tn
N(f)
. Consequently f⪯
Tn
Ng
for all
n∈ℕ
. As above, this gives
f⪯
T∗
Ng
as required.
◻
Declarations
Conflict of interest The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long
as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Com-
mons licence, and indicate if changes were made. The images or other third party material in this article
are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the
material. If material is not included in the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly
from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
References
1. Altman, A., & Tennenholtz, M. (2005). Ranking systems: The pagerank axioms. In Proceedings of the
6th ACM Conference on Electronic Commerce (pp. 1–8). ACM.
2. Altman, A., & Tennenholtz, M. (2008). Axiomatic foundations for ranking systems. Journal of Artifi-
cial Intelligence Research, 31(1), 473–495.
T
n
+
1
N1
(s)=
∑
g∈𝖿𝖺𝖼𝗍𝗌N1(s)
Tn
N1
(g)
=Tn
N2
(g)
=
∑
g∈𝖿𝖺𝖼𝗍𝗌N2(s)
Tn
N2
(g)=Tn
+
1
N2
(s
)
Tn+1
N+1
(f)=T
n+1
N2
(f
)
T
n
N(g)=
∑
s∈𝗌𝗋𝖼N(g)
Tn
N(s)
≤∑
s∈𝗌𝗋𝖼N(g)
Tn
N(s)+ ∑
s∈S⧵𝗌𝗋𝖼N(g)
Tn
N(s
)
=∑
s∈S
Tn
N(s)
=∑
s∈𝗌𝗋𝖼N(f)
Tn
N(s)
=Tn
N
(f)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 47 of 49 42
3. Andersen, R., Borgs, C., Chayes, J., Feige, U., Flaxman, A., Kalai, A., Mirrokni, V., & Tennenholtz,
M. (2008). Trust-based recommendation systems: An axiomatic approach. In Proceedings of the 17th
International Conference on World Wide Web, WWW ’08 (pp. 199–208). ACM. https:// doi. org/ 10.
1145/ 13674 97. 13675 25.
4. Arrow, K. J. (1952). Social choice and individual values. Ethics, 62(3), 220–222.
5. Axler, S. (2014). Linear Algebra Done Right. Springer International Publishing.
6. Balakrishnan, R., & Kambhampati, S. (2011). Sourcerank: Relevance and trust assessment for deep
web sources based on inter-source agreement. In Proceedings of the 20th International Conference on
World Wide Web (pp. 227–236).
7. Berti-Equille, L., & Borge-Holthoefer, J. (2015). Veracity of data: From truth discovery computa-
tion algorithms to models of misinformation dynamics. Synthesis Lectures on Data Management,
7(3), 1–155.
8. Brandt, F., Conitzer, V., Endriss, U., Lang, J., & Procaccia, A. D. (2016). Introduction to computa-
tional social choice. In F. Brandt, V. Conitzer, U. Endriss, J. Lang, & A. D. Procaccia (Eds.), Hand-
book of Computational Social Choice. Cambridge University Press.
9. Christoff, Z., & Grossi, D. (2017). Binary voting with delegable proxy: An analysis of liquid
democracy. In Proc. TARK 2017.
10. Ding, H., Gao, J., & Xu, J. (2016). Finding global optimum for truth discovery: Entropy based
geometric variance. In Proc. 32nd International Symposium on Computational Geometry (SoCG
2016).
11. Dokow, E., & Holzman, R. (2010). Aggregation of binary evaluations with abstentions. Journal of
Economic Theory, 145, 544–561.
12. Dong, X. L., Berti-Equille, L., & Srivastava, D. (2009). Integrating conflicting data: The role of
source dependence. Proceedings of the VLDB Endowment, 2(1), 550–561.
13. Dong, X. L., Berti-Equille, L., & Srivastava, D. (2009). Truth Discovery and Copying Detection in
a Dynamic World. Proceedings of the VLDB Endowment, 2(1), 562–573. https:// doi. org/ 10. 14778/
16876 27. 16876 91.
14. Du, Y., Sun, Y. E., Huang, H., Huang, L., Xu, H., Bao, Y., & Guo, H. (2019). Bayesian co-cluster-
ing truth discovery for mobile crowd sensing systems. IEEE Transactions on Industrial Informat-
ics. https:// doi. org/ 10. 1109/ TII. 2019. 28962 87.
15. Endriss, U. (2016). Judgment aggregation. In F. Brandt, V. Conitzer, U. Endriss, J. Lang, & A. D.
Procaccia (Eds.), Handbook of Computational Social Choice. Cambridge University Press.
16. Everaere, P., Konieczny, S., Marquis, P. (2010). The epistemic view of belief merging: Can we
track the truth?. In ECAI (pp. 621–626).
17. Galland, A., Abiteboul, S., Marian, A., & Senellart, P. (2010). Corroborating information from
disagreeing views. In Proceedings of the Third ACM International Conference on Web Search and
Data Mining, WSDM ’10 (pp. 131–140). ACM. https:// doi. org/ 10. 1145/ 17184 87. 17185 04.
18. Ghosh, A., Kale, S., & McAfee, P. (2011). Who moderates the moderators?. In Proceedings of
the 12th ACM Conference on Electronic Commerce—EC 11. ACM Press . https:// doi. org/ 10. 1145/
19935 74. 19935 99.
19. Gupta, M., & Han, J. (2011). Heterogeneous network-based trust analysis: A survey. ACM SIGKDD
Explorations Newsletter, 13(1), 54–71. https:// doi. org/ 10. 1145/ 20313 31. 20313 41.
20. Hartmann, S., & Sprenger, J. (2012). Judgment aggregation and the problem of tracking the truth.
Synthese, 187, 209–221.
21. Kleinberg, J. M. (1999). Authoritative sources in a hyperlinked environment. Journal of the ACM,
46(5), 604–632. https:// doi. org/ 10. 1145/ 324133. 324140.
22. Konieczny, S., & Pérez, R. P. (2002). Merging information under constraints: A logical framework.
Journal of Logic and computation, 12(5), 773–808.
23. Konieczny, S., & Pérez, R. P. (2008). Improvement operators. In G.Brewka, J.Lang (Eds.), Princi-
ples of Knowledge Representation and Reasoning: Proceedings of the Eleventh International Con-
ference, KR 2008, Sydney, Australia, 16–19 Sept, 2008 (pp. 177–187). AAAI Press. http:// www.
aaai. org/ Libra ry/ KR/ 2008/ kr08- 018. php.
24. Kotonya, N., & Toni, F. (2020). Explainable automated fact-checking for public health claims.
In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing
(EMNLP) (pp. 7740–7754).
25. Kruger, J., Endriss, U., Fernandez, R., & Qing, C. (2014). Axiomatic analysis of aggregation meth-
ods for collective annotation. In Proceedings of the 2014 International Conference on Autonomous
Agents and Multi-agent Systems, AAMAS ’14 (pp. 1185–1192). International Foundation for Auton-
omous Agents and Multiagent Systems, Richland, SC. http:// dl. acm. org/ citat ion. cfm? id= 26173 88.
26174 37.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
42 Page 48 of 49
26. Laslier, J. F., & Sanver, M. R. (2010). Handbook on Approval Voting. Springer Science & Business
Media.
27. Li, Y., Gao, J., Meng, C., Li, Q., Su, L., Zhao, B., etal. (2016). A survey on truth discovery. ACM
SIGKDD Explorations Newsletter, 17(2), 1–16. https:// doi. org/ 10. 1145/ 28973 50. 28973 52.
28. Li, Y., Li, Q., Gao, J., Su, L., Zhao, B., Fan, W., & Han, J. (2016). Conflicts to harmony: A frame-
work for resolving conflicts in heterogeneous data by truth discovery. IEEE Transactions on Knowl-
edge and Data Engineering, 28(8), 1986–1999. https:// doi. org/ 10. 1109/ TKDE. 2016. 25594 81
29. Ma, F., Li, Y., Li, Q., Qiu, M., Gao, J., Zhi, S., Su, L., Zhao, B., Ji, H., & Han, J. (2015). Fait-
Crowd: Fine Grained Truth Discovery for Crowdsourced Data Aggregation. In Proceedings of the
21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD
’15 (pp. 745–754). ACM. https:// doi. org/ 10. 1145/ 27832 58. 27833 14. Event-place: Sydney, NSW,
Australia.
30. Ma, F., Meng, C., Xiao, H., Li, Q., Gao, J., Su, L., & Zhang, A. (2017). Unsupervised discovery of
drug side-effects from heterogeneous data sources. In Proceedings of the 23rd ACM SIGKDD Inter-
national Conference on Knowledge Discovery and Data Mining, KDD ’17 (pp. 967–976). ACM.
https:// doi. org/ 10. 1145/ 30979 83. 30981 29.
31. Marshall, J., Argueta, A., & Wang, D. (2017). A neural network approach for truth discovery in
social sensing. In 2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems
(MASS) (pp. 343–347). IEEE.
32. Meir, R., Amir, O., Cohensius, G., Ben-Porat, O., & Xia, L. (2019). Truth discovery via proxy vot-
ing. CoRR. arXiv: abs/ 1905. 00629.
33. Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank Citation Ranking: Bring-
ing Order to the Web. Technical Report 1999-66, Stanford InfoLab. http:// ilpubs. stanf ord. edu: 8090/
422/.
34. Pasternack, J., & Roth, D. (2010). Knowing what to believe (when you already know something).
In Proceedings of the 23rd International Conference on Computational Linguistics, COLING
’10(pp. 877–885). Association for Computational Linguistics, Stroudsburg, PA, USA. http:// dl. acm.
org/ citat ion. cfm? id= 18737 81. 18738 80.
35. Singleton, J., & Booth, R. (2020). An axiomatic approach to truth discovery. In Proceedings of the
19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS ’20 (pp.
2011–2013). International Foundation for Autonomous Agents and Multiagent Systems, Richland,
SC.
36. Tennenholtz, M. (2004). Reputation systems: An axiomatic approach. In Proceedings of the 20th
Conference on Uncertainty in Artificial Intelligence, UAI ’04 (pp. 544–551). AUAI Press, Arling-
ton, Virginia, United States. http:// dl. acm. org/ citat ion. cfm? id= 10368 43. 10369 09.
37. Waguih, D. A., & Berti-Equille, L. (2014). Truth discovery algorithms: An experimental evalua-
tion. arXiv preprint. arXiv: 1409. 6428.
38. Wang, D., Kaplan, L., Le, H., & Abdelzaher, T. (2012). On truth discovery in social sensing: A
maximum likelihood estimation approach. In Proceedings of the 11th International Conference
on Information Processing in Sensor Networks, IPSN ’12 (pp. 233–244). ACM. https:// doi. org/ 10.
1145/ 21856 77. 21857 37. Event-place: Beijing, China.
39. Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., Su, L., & Gao, J. (2018). Eann: Event adver-
sarial neural networks for multi-modal fake news detection. In Proceedings of the 24th ACM SIG-
KDD International Conference on Knowledge Discovery & Data Mining (pp. 849–857).
40. Wąs, T., & Skibski, O. (2020). Axiomatic characterization of pagerank. https:// doi. org/ 10. 48550/
ARXIV. 2010. 08487. arXiv: 2010. 08487.
41. Xiao, H. (2018). Multi-sourced information trustworthiness analysis: Applications and theory.
Ph.D. thesis, University at Buffalo, State University of New York.
42. Xiao, H., Gao, J., Wang, Z., Wang, S., Su, L., & Liu, H. (2016). A truth discovery approach with
theoretical guarantee. In Proceedings of the 22nd ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining, KDD ’16 (pp. 1925–1934). ACM. https:// doi. org/ 10. 1145/
29396 72. 29398 16.
43. Yang, Y., Bai, Q., & Liu, Q. (2018). On the discovery of continuous truth: A semi-supervised
approach with partial ground truths. In Web Information Systems Engineering—WISE 2018 (pp.
424–438). Springer International Publishing. https:// doi. org/ 10. 1007/ 978-3- 030- 02922-7_ 29.
44. Yang, Y., Bai, Q., & Liu, Q. (2019). A probabilistic model for truth discovery with object correla-
tions. Knowledge-Based Systems, 165, 360–373. https:// doi. org/ 10. 1016/j. knosys. 2018. 12. 004.
45. Yin, X., Han, J., & Yu, P. S. (2008). Truth discovery with multiple conflicting information provid-
ers on the web. IEEE Transactions on Knowledge and Data Engineering, 20(6), 796–808. https://
doi. org/ 10. 1109/ TKDE. 2007. 190745
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Autonomous Agents and Multi-Agent Systems (2022) 36:42
1 3
Page 49 of 49 42
46. Yin, X., & Tan, W. (2011). Semi-supervised truth discovery. In Proceedings of the 20th Interna-
tional Conference on World Wide Web, WWW ’11 (pp. 217–226). ACM. https:// doi. org/ 10. 1145/
19634 05. 19634 39. Event-place: Hyderabad, India.
47. Zhang, D. Y., Han, R., Wang, D., & Huang, C. (2016-12). On robust truth discovery in sparse
social media sensing. In 2016 IEEE International Conference on Big Data (Big Data) (pp. 1076–
1081). https:// doi. org/ 10. 1109/ BigDa ta. 2016. 78407 10.
48. Zhang, L., Qi, G. J., Zhang, D., & Tang, J. (2018). Latent dirichlet truth discovery: Separating
trustworthy and untrustworthy components in data sources. IEEE Access, 6, 1741–1752. https:// doi.
org/ 10. 1109/ ACCESS. 2017. 27801 82
49. Zhi, S., Zhao, B., Tong, W., Gao, J., Yu, D., Ji, H., & Han, J. (2015). Modeling tr uth existence in
truth discovery. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, KDD ’15 (pp. 1543–1552). ACM. https:// doi. org/ 10. 1145/ 27832 58.
27833 39.
50. Zwicker, W. S. (2016). Introduction to the theory of voting. In F. Brandt, V. Conitzer, U. Endriss, J.
Lang, & A. D. Procaccia (Eds.), Handbook of Computational Social Choice. Cambridge University
Press.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center
GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers
and authorised users (“Users”), for small-scale personal, non-commercial use provided that all
copyright, trade and service marks and other proprietary notices are maintained. By accessing,
sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of
use (“Terms”). For these purposes, Springer Nature considers academic use (by researchers and
students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and
conditions, a relevant site licence or a personal subscription. These Terms will prevail over any
conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription (to
the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of
the Creative Commons license used will apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may
also use these personal data internally within ResearchGate and Springer Nature and as agreed share
it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not otherwise
disclose your personal data outside the ResearchGate or the Springer Nature group of companies
unless we have your permission as detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial
use, it is important to note that Users may not:
use such content for the purpose of providing other users with access on a regular or large scale
basis or as a means to circumvent access control;
use such content where to do so would be considered a criminal or statutory offence in any
jurisdiction, or gives rise to civil liability, or is otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association
unless explicitly agreed to by Springer Nature in writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a
systematic database of Springer Nature journal content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a
product or service that creates revenue, royalties, rent or income from our content or its inclusion as
part of a paid for service or for other commercial gain. Springer Nature journal content cannot be
used for inter-library loans and librarians may not upload Springer Nature journal content on a large
scale into their, or any other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not
obligated to publish any information or content on this website and may remove it or features or
functionality at our sole discretion, at any time with or without notice. Springer Nature may revoke
this licence to you at any time and remove access to any copies of the Springer Nature journal content
which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or
guarantees to Users, either express or implied with respect to the Springer nature journal content and
all parties disclaim and waive any implied warranties or warranties imposed by law, including
merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published
by Springer Nature that may be licensed from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a
regular basis or in any other manner not expressly permitted by these Terms, please contact Springer
Nature at
onlineservice@springernature.com
