Content uploaded by Sanel Halilbegovic
Author content
All content in this area was uploaded by Sanel Halilbegovic on Apr 05, 2020
Content may be subject to copyright.
Backtesting Value at Risk Models
ABSTRACT
This manuscript researches a concept of value at risk which represents the largest
loss of investment portfolio that is expected in a given reporting period with a given
confidence level. Secondary, audited data was used along with various back-testing
methods for examining exception frequency and results indicate that the VaR
models in this research are accurate at almost all levels of confidence with only a
slight presence of possible risks and problems. Five tests including Point of Failure,
Time until First Failure, Basel Traffic Light, Mixed Kupiec and Christoffersen's
Independence test were performed in order to evaluate if the respective method for
VaR calculation is consistent. Results obtained from this research are able to
provide an intelligence whether potential flaws or risks exist in company's core, so
that management can address those and perform a necessary risk mitigation
measures to protect the firm against potential and future risks.
Keywords: risk management, portfolio risk, market risk measure, capital
management, Kupiec, Christoffersen
JEL codes: D81; G11; G32
1..Introduction
In today's everchanging business environment, management is often in a position to
make decisions on a go, without a deep evaluation of risks and potentially negative
effects of such decision. In the world of financial securities the most ubiquitous type
of risk is actually market risk which is threat to a portfolio of financial instruments
derived from the actual movement and/or volatility of market prices that include stock
prices, forex (foreign exchange rates), option prices, swaps, commodity prices,
interest rates etc. On the top of the list of today's modern management of any
business has to to be an active management of risks which has become an
indigenous part of operations in which management defines, benchmarks and after
all controls the vulnerability to risk (Graham and Pal, 2014). Modern financial
institutions come face to face with two major challenges: management of risk (risk
mitigation) and maximization of profits. Financial institutions aim to increase revenue
by taking on the risk(s) and actively coping with it. Hence, in order to keep
profitability of the organization at the wanted level, management of the company
needs to constantly and consistantly manage risks (Emmer et al, 2015). Markowitz
(1991) initially introduced gauging risks by utilizing mean variance behaviour
(Markowitz, 1991). Later on, other dimensions of risks have arisen and those are
common value at risk and a conditional value at risk. The former has become an
elementary measure of risk in banks regulation and a prime measure of internatal
risk of banks and institutions alike (Pflug, 2000). Great thing about VaR is that it is
very user friendly compared to other risk measurement techniques, hence earning
the prime risk measurement spot in practice – what additionally confirmed VaR
measurement, is the fact that it has been accepted by the Basel Accord as the
preeminent and unsurpassed risk measure.
2. Literature Review
Riskiness of loss in investing, measure of it that is, represents value at risk or VaR.
This measure assesses how much can a given set of investments lose in value,
given a certain probability, in normal market conditions during a specified period of
time. This measure is often used by investment companies and financial markets in
order to indicate the amount of funds that can cover potential losses. This way VaR
serves as a quality estimate of potential loss of value of a given portfolio. As noted
above, importance of VaR lies in the fact that it is fairly easy to come up with and
interpret, as a number that is a risk or the level to which the institution's portfolio is
exposed to loss. Elemental methodology of VaR is actually a fusion of contemporary
portfolio theory, statistics and financial analysis, all of which assess risk factors
(Zhang and Nadarajah, 2017). VaR was widely implemented in banks starting with
a Chase Bank Inc back in 1998 when they started using VaR as a tool for controlling
and assessing daily risks and vulnerability of bank's portfolio. CEO of the Chase
bank at a time, sir Weatherstone, chartered bank's financial analysts to start
generating rapport on a daily basis, a rapport that would pivot around a number that
would show an implict loss of a portfolio (Campbell, 2005). Yamai (2002) stated that
if in a given prortfolio, proportion or contribution of open positions during a certain
period is unchanged, value at risk gives a high quality insight into the potential loss
of such portfolio (Yamai, 2002). This value of the estimated loss that Yamai (2002)
talked about, is gauged with a specific level of assurance hence when considering
expected loss we can only treat it as a „potential“ loss. This is important to note
since this potential loss is not a measure that indicates a maximum boundary of
attainable loss. Another important feature of VaR is that it is not a tool used as an
illustrant of potential losses when extreme market conditions occur. This can be
portrayed in the following example – If VaR is 95%, and VaR representing the
maximum amount in a portfolio expected to be lost over a given time span, at a
predefined confidence level - a 95% one-month (or some other period) VaR is
$100,000, 95% confidence exist that over the next month the portfolio shall not lose
more than $100,000, but doesn’t “explain” what may occur with the other 5%
eventualities.
Jorion (2001) stated that VaR is the new benchmark for managing financial risk as it
takes into account how volatilities in financial securities’ price affect each other, and
expressed VaR as:
VaR=a*
σ
*W
(1)
Where parameters of the above formula are: “a” – confidence interval,
σ
–
Standard deviation (volatility) and W – starting portfolio value (Jorion, 2001).
Christofferssen (1998) noted that since VaR accounts for changes in securities’ price
and the way they impact and affect each other, it can disrate the risk with procedures
that involve diversification techniques (Christofferssen, 1998). Over time we can see
periods with pronounced volatility and persistency of it. Because of this it is possible
to differentiate between two market conditions: one under normal market conditions
and one under extreme market conditions, bluntly said, not-normal market
conditions. In these abnormal market conditions, VaR becomes an inadequate
indicator, and if one wants a broader view and a better awareness of market risk,
VaR has to be combined with other indicators, like a stress test etc (Bams et al,
2017).
Prior to doing computations and arriving to any verdicts from them, it is crucial to be
receptive and conscious of all fundamental terms apropos the procedure of projected
VaR revision. In order to verify if outcomes collected from VaR computation are
coherent, harmonious and tried-and-true, each model must be supported, attested
and corroborated with the process that is popularly called backtesting that is done
with the support of statistical models. Brown (2008) confirmed the magnitude and
significance of back-testing process by stating that any VaR is as good as it’s
backtest “When someone shows me a VaR number, I don’t ask how it is computed, I
ask to see the backtest“ (Brown, 2008). Backtesting is an approach where the
prognosed VaR number is associated and equated to the losses/gains of a given
investment portfolio. If VaR valuation is not precise or finite, the model itself should
be probed and inspected, possibly for wrong assumptions, potentially unfit
parameters, faulty model itself etc. There are various methods that are
recommended for backtesting. It is important to mention an unconditional coverage
test and what features it, is that the unconditional coverage doesn’t factor in or
consider when the exception has occurred. Similarly important aspect is to attest
that observations that surpass and outreach VaR are independent, that is, to be
uniformly dispersed in time. The main feature of a well-founded and rational model
is that such model is capable of avoiding clustering of deviations, in such way that a
given model reacts quickly to increase or decrease in volatility of a financial
instrument or a portfolio and their respective correlation (Paseka et al, 2018).
We have to note that there could be very serious inconsistencies in VaR appraisals
for turbulent markets (Nelson, 1991). In its basic definition VaR helps us to compute
expected loss given that market conditions are normal, meaning that a well-
constructed VaR model would be able to give a precise number of deviations and
exceptions that are evenly distributed in time which means that they would be
independent of each other.
3. Methodology of research
Due to the fact that this research takes into consideration not only one test, the
hypothesis for each test will be presented individually. However, the null hypothesis
that summarize five tests for the back-testing of VaR model is simply stated as:
Ho: Value at risk model is statistically significant
The TUFF (time until first failure) is the test and measures the time until the first
exception occurs. The null hypothesis for TUFF test is expressed as:
H0:p=
^
p
=
1
v
(3)
Variables in formula (3) are p - The magnitude of failure,
^
p
- recognized failure
rate and v- Time until first exception. Translated into theTUFF test, the main
hypothesis would be that the probability of an exception is the inverse
probability of the confidence level for VaR. The conclusion is the same as for the
POF test hypothesis that in the case where the likelihood ratio is higher than the
critical value of the χ², the null hypothesis would be rejected and model would be
considered as inaccurate. Otherwise, the null hypothesis would be accepted.The
Basel Traffic Light Approach as the second test examines the model accuracy and
correctness by measuring the number of exceptions. Using Basel Traffic Light
approach null hypothesis states that the number of exceptions is between 0 and
32 at 90% level of confidence; 0 and 17 at 95%; and between 0 and 4 at 99%
level of confidence. If the number of exceptions does not fall into range, the
conclusion about model inaccuracy would be made. Otherwise, the null hypothesis
would be accepted. Christoffersen's Independence test, third test, examines if the
probability of today's exception depands on the outcome from the day before. The
null hypothesis is expressed as:
H0: π0 = π1 (4)
Where π is probability value. The null hypothesis states that an exception that
occured at some period P, does not depend on whether an exception
happened at the period before, or P-1. The conclusion is the same as for the POF
& TUFF hypotheses that in the case where the plausibility ratio is higher than the
crucial value of the χ², the null hypothesis about equal distribution of exceptions over
time would be rejected and model would be considered as inaccurate. Otherwise,
the null hypothesis would be accepted. Mixed Kupiec-test, test number four,
proposed by Haas (2001), examines the time between each exceptions, advocating
exception indepencency during the testing period (Haas, 2001)
The null hypothesis is expressed as:
H0: x0 = x1 (5)
Where x is the number of exceptions. The null hypothesis for the Mixed Kupiec-test
states that exceptions are independent of each other over time. The conclusion
is the same as for the the other test hypotheses that in the case where the likelihood
ratio is higher than the critical value of the χ², the null hypothesis about exceptions
independency over time would be rejected and model would be considered as
inaccurate. Otherwise, the null hypothesis would be accepted. One of the reason
why this research has been performed is due to ever-going discussions among
practicioners, whether or not VaR model is dependable or as practicioners say tried-
and-true. So, the final purpose of the research is to evaluate through many tests if
the respective method for VaR calculation is consistent. This research takes into
consideration, for its empirical part, the quantitative research due to the fact that it
involves data in numerical form which are further used for statistical calculations in
order to draw a conclusion about model accuracy. The empirical part of the research
consists of two sections: calculation of VaR amounts and back-testing those
amounts using different types of tests. Researchers used secondary data sourced
from the audited financial reports and valid stock price information system (Securities
and exchange commission, and www.wsj.com) for five “blue-chip” companies:
Procter & Gamble, Mc Donald’s, Microsoft, Caterpillar and Apple as a base for all
calculations, simulations and analyses. It is important to emphasize that part of the
calculations and graphs are done in excel, and the other part in SPSS, using the
adequate formulas or functions. The first step, which is used for both: VaR
calculation and back-testing purposes, is to calculate daily returns for each company
(without dividends). Daily returns have been calculated as the P0 – today market
closing price less the previous day closing price P-1, divided by the previous day
closing price, P-1. This data calculation is essential for all three VaR calculation
methods and further calculations and conclusions.
a) VaR Calculation Methods
Once the daily returns are calculated, the second step for parametric method
(variance-covariance) is to measure the company’s standard deviation (volatility) in
SPSS. As the name of the method says, variance-covariance, it is necessary to form
a covariance matrix as well in SPSS. The covariance matrix is needed in order to
calculate the standard deviation of the portfolio. The value of standard deviation,
which is calculated from covariance matrix and amount of vector of invested funds, is
then used to get the value at risk at 95% level of confidence. All five companies are
included in the process. In order to calculate the VaR according to historical
simulation method, expected daily return for the whole portfolio is mandatory.
Expected portfolio daily return is calculated by computing expected daily returns for
each company and then summing up the expected daily returns for those five
companies. After that, the standard deviation of portfolio’s daily returns is measured.
The VaR is got by plugging the necessary data in the formula (1) at 90%, 95% and
99% level of confidence. The process includes all five companies. The third method,
Monte Carlo simulation, is only applied for Procter & Gamble and it requires the
usage of function RAND. By using this function it is possible to get the random
numbers. The following formula is used in EXCEL
=NORMINV(RAND();average;standarddeviation), where the amounts of average and
standard deviation of the company are previously computed in parametric method.
Computation of price change is based on the formula from Geometric Brownian
Motion which will be in more detail explained in the Data Analysis part. To calculate
VaR at 90%, 95% and 99% level of confidence by using the formula (1), it is required
to compute the standard deviation of simulated portfolio’s daily returns.
b) Back-testing Process
First test, Time until first failure (TUFF), calculates the time when the first exception
occurs, and then by using the following formula, the likelihood ratio at 90%, 95% and
99% level of confidence can be computed:
LRTUFF= -2ln
(
p∗(1−p)v−1
)
1
v∗(1−1
v)
v−1
(7)
The so-called Christoffersen’s Independence Test is the second test which is used
for evaluating whether the exceptions that occur are equally spread during the
period. This test is for the first step based on contingency table. For the purposes of
the contingency table, it is necessary to define two indicators, first a) It=1 if the
exception occurs and It=0 if the exception does not occur. This means that if the
loss is greater than the forecasted VaR, the assigned indicator would be 1; otherwise
0. From the figure below, the “n” stands for the number of days when two conditions
(for today: t and for the day before: t-1) were met:
Figure 1: Contingency table
It-1=0 It-1=1
It=
0n00 n10 n00+n1
0
It=
1n01 n11 n01+n1
1
n00+n0
1
n10+n1
1N
Using the data information from contingency table (figure 1), the following formulas
are used for computing the probability values:
π0 =
n01
n00+n01
(8)
π1 =
n11
n10+n11
(9)
π =
n01+n11
n00+n01 +n10 +n11
(10)
The formula for likelihood ratio is expressed as:
LRind = -2ln
(1−π)n00+n10∗πn01+n11
¿
(
¿¿(1−π0)n00∗π0n01∗
(
1−π1
)
n10∗π1n11
)
(11)
Mixed Kupiec-test for evaluating exception independency is the last test that will be
calculated and presented in this research. In order to calcualate the required
likelihood ratio, tie time between two exceptions is required. After that, by plugging
the necessary data into the same formula as one used for TUFF test:
LRTUFF= -2ln
(
p∗(1−p)v−1
)
1
v∗(1−1
v)
v−1
(12)
Once all previously explained calculations are completed, the comparison of results
obtained and corresponding critical values is done. The comparison is done for each
test. These comparisons are necessary in order to make correct conclusions about
the validity of the model.
4. Data analysis
The empirical part of this research is divided into two sections: VaR calculation
methods and back-testing process.
a) VaR Calculation Methods
As is already written in the literature review and methodology part, there are three
methods for calculating forecasted Value at Risk. The first one is the Parametric
method or the so-called Variance Covariance model. This model considers that all
changes in the market are normally distributed and in order to apply this method, the
set of simple data information is required. So, because of this, for calculation
purposes, set of daily close prices are observed for five companies. The first step for
the calculation of VaR using the parametric method is to collect the data for five
companies and their closing prices during the period.
Figure 2: Daily prices
Once the daily share prices are collected from the official Yahoo Finance page, daily
returns for each firm were calculated using the following formula:
r=ln St
St−1
(12)
Variables in formula (12) are:
r - Daily return ln – Natural logatithm St- Price today St-1 - Price day before
Using formula 12, returns for each firm have been calculated and displayed in figure
3.
Figure 3: Daily Returns
As the daily returns (figure 3) for each company are calculated and the results are
presented through the graphs, the values of standard deviation (volatility) for daily
returns are calculated in SPSS. It can be seen, since the standard deviation of each
company is close to 0, that daily returns tend to be very close to the mean. Results
of this analysis are shown in figure 4.
Figure 4: Standard deviation
Descriptive Statistics
N Minimum Maximum Mean Std. Deviation
Procter&
Gamble 251 -.0212049830 .0296893820 -.000108153310 .0078917720328
McD 251 -.0146537679 .0369152133 -.000205812919 .0068952041537
Microsoft 251 -.0465770461 .0703298388 .001248809870 .0136383321064
Caterpillar 251 -.0625890936 .0577191633 .000787025476 .0112286341552
Apple 251 -.0833024982 .0787942579 .001412017339 .0145728769847
Valid N
(listwise) 251
The next step is forming the variance/covariance matrix based on daily returns,
which are presented in the figure 5. The variance/covariance matrix output is formed
in SPSS indicating that both company's daily returns are increasing or decreasing
together and can be seen in the figure 5:
Figure 5: Covariance Matrix
Inter-Item Covariance Matrix
P&G Return McD Return Microsoft
Return
CAT Return Apple Return
Procter& GambleReturn .0000623 .0000189 .0000297 .0000116 .0000075
McD Return .0000189 .0000475 .0000176 .0000270 .0000028
Microsoft Return .0000297 .0000176 .0001860 .0000437 .0000067
Caterpillar Return .0000116 .0000270 .0000437 .0001261 .0000243
Apple Return .0000075 .0000028 .0000067 .0000243 .0002124
These values from the matrix are necessary for the calculation of standard deviation
of the portfolio in a way that the covariance matrix is multiplied by the value of vector
of invested funds which in this case is taken as $2000:
After the matrix vector is being calculated, it is multiplied by the transposed vector of
invested funds:
From this, a standard deviation (volatility) of portfolio is calculated:
σ =
√
2.03
= $1.42
Now, having the amount of standard deviation of portfolio, the Value at Risk can be
calculated at the 90%, 95% and 99% level of confidence. Value at Risk at these
levels of confidence equals to the product of standard deviation and confidence
level, which in this case gives the following result suggesting that not more than this
amount would be lost at the respective level of confidence:
VaR 90% = $1.42 * (-1.65) = -$2.35 VaR 95% = $1.42 * (-1.96) = -$2.79 VaR 99%
= $1.42 * (-2.58) = -$3.67
The second method for calculating VaR is a non parametric method called a method
of historical simulation.
For the purposes of calculating VaR, the same data are necessary as for the
parametric method: prices for five companies are considered for the period. Daily
returns for each one of five companies are calculated in a same way as for the
parametric method, but now when daily returns of each firm are computed, the
expected daily yield for the entire investment portfolio is computed by totaling
expected daily returns for each company and then summing up the expected daily
returns for those five firms. Expected daily returns for the company are calculated in
a following way: daily return (%) * amount of invested funds in portfolio. As an
example, it is taken that the amount of invested funds in portfolio equals to $2000.
Using the method for obtaining the expected daily returns for each firm and for the
entire portfolio results are as follows:
Figure 6: Expected Daily Returns
Once the expected daily returns are calculated, the next step is forming the
correlation matrix for the share prices from which the degree, to which shares from
companies move in relation to each other, is visible. The correlation matrix is
calculated in order to get the coefficient of correlation based on the share prices
movements. The figure 7 presents the calculated coefficients whose values fall
between -1 and 1:
Figure 7: Correlation Matrix
Correlations
P&G McD Microsoft Caterpillar Apple
P&G Pearson Correlation 1 .345 .218 -.031 .167
McD Pearson Correlation .345 1 .509 .663 .524
Microsoft Pearson Correlation .218 .509 1 .857 .858
Caterpillar Pearson Correlation -.031 .663 .857 1 .799
Apple Pearson Correlation .167 .524 .858 .799 1
Considering that expected daily yields of portfolio have been computed, those results
can be used for further calculations. Firstly, volatility of the portfolio (expressed as
standard deviation) and the amount of average expected daily returns of the portfolio
are calculated in SPSS:
Figure 8: Average Return and Standard Deviation
Descriptive Statistics
N Mean Std. Deviation
Portfolio 251 6.267772905 63.68454042
Valid N (listwise) 251
Descriptive statistics parameters are needed in order to compute a one day VaR
estimation which is done by using the formula (1) and taking the value of initial
portfolio value (amount of invested fund) to be equal to $2000. So, having all the
necessary data, by plugging it in the formula (1), the VaR values are shown in the
figure 9:
Figure 9: VaR Amounts
Confidence Level 1-day VaR
90% -6.63
95% -7.88
99% -10.37
The third method for VaR calculation is Monte Carlo Simulation, which is also not a
parametric method as a historical simulation. However, it is a way more complex to
implement. This method will be applied for one Proctor & Gamble company. Since
daily returns and volatility (standard deviation) are already calculated in the
parametric method, the same data and results will be used for the further
calculations of Monte Carlo simulation. By applying the Monte Carlo simulation
method, the potential (simulated) scenario of share price movements for the period
of future 251 days is going to be calculated. The first step for getting the simulated
share price movement scenario is to use the RAND function in EXCEL in order to get
random 251 numbers ( ). The second step is to calculate the percentage ofɛ
simulated returns by using the combination of two functions in EXCEL: NORMINV
and RAND, and two already calculated values in parametric method: average return
and standard deviation of the portfolio. In order to calculate price change, the usage
of the following so-called Geometric Brownian Motion formula in essential:
dS = µSdt + σSdz (13)
Variables in formula (13) are S – Stock (share) price, µ - The drift, σ – Standard
deviation and t – Time where dz =ɛ
√
dt
- Random numberɛ
Since, random numbers are already simulated in the first step of the Monte Carlo
method, the third step is to plug the necessary data into the formula (13) and get the
price change per each day (251 price changes in total). The last step is to calculate
the “new”, simulated prices for the future period of 251 working/trading days. The
first “new” simulated price would be calculated by summing up the last available
price (in this case price from 04.08.2014) and the calculated price change from the
previous step. This process is done for the rest of 250 days, each time taking the
respective new calculated price and the price change for that day.
The figure 10 is the representation and the summary of the whole process of
simulating the new prices described in the paragraph above:
Figure 10: Price Simulation
Each time the simulation is started, new values and results are available. So, one of
the possible simulated scenarios represents the stock prices movement for the future
period of 251 days. After the simulation is finished, the standard deviation of the
simulated daily returns is needed to be calculated in order to get the VaR for three
levels of confidence. Again, by using the SPSS, standard deviation equals to:
Figure 11: Simulated Portfolio Standard Deviation
Descriptive Statistics
N Mean Std. Deviation
Returns 251 .000933709 .0079907121
Valid N (listwise) 251
By using the formula (1) and new standard deviation of the simulated daily returns
for the company Procter& Gamble, VaR equals to amounts as shown in figure 12:
Figure 12: VaR Amounts
Confidence Level 1-day VaR
90% -0.094
95% -0.111
99% -0.147
The process of simulation can be performed as many as possible times, and each
time the VaR calculations will be automatically updated.
b) Back-testing Process
The process of back-testing is the essential part when making any conclusion
whether the model is „good“ or „bad“, reliable or not. For the purposes of this
research, the back-testing process is done for the calculated VaR forecast using the
historical simulation method. There are many methods for the so-called backtesting.
In this research following methods will be covered:
1.) Unconditional coverage is the first group that consists of three tests:
Kupiec’s POF is a test that is very popularly used for estimating and valuing risk
models such as VaR. This test is used in order to see if the number of exceptions is
in symmetry and harmony with the level of confidence. First step in testing is to
compute daily yields for each firm as well as daily yields for the whole portfolio, and
after that average return and a standard deviation of the portfolio. After that it is
essential to compute a number of actual exception occurrences. An exception has
happened if the value of loss is higher than the anticipated VaR value. Figure 13
shows results for examining whether the exception or even the profit occurs:
Figure 13: Exception occurrence
Procedure continues by obtaining a one day VaR and corresponding exceptions as
shown in figure 14
Figure 14: One day VaR & Exceptions
Confidence Level 1-day VaR Exceptions
90% -6.63 9
95% -7.88 11
99% -10.37 12
Back testing process can begin since we now know the number of exceptions for
each level of confidence and information that is needed is shown in figure 15.
Figure 15: POF Test Data
Level of confidence Number of observations Number of exceptions
90% 251 9
95% 251 11
99% 251 12
A 95 percent level of confidence is taken as the critical value for each of the three
‘likelihood ratio’ computations that have been performed for back testing procedures.
This basically means that a strong confirmation and validation is needed for rejecting
the null hypothesis.
First we will test the portfolio that had a total of nine occurrences where daily
portfolio returns or losses were greater than the estimated VaR at the 90%
confidence level during the 251 trading days.
The likelihood ratio test in this case equals to:
LR POF90= -2ln
(
(1−0.10)251−9¿0.109
)
[
1−( 9
251 )
]
251−9
∗( 9
251 )
9
= 14.85
Secondly we test the LR POF for the portfolio that had a total of eleven occurrences
where daily portfolio returns or losses were greater than the estimated VaR at the
95% confidence level during the 251 trading days.
LR POF95= 2ln
(
(1−0.05)251−11 ¿0.0511
)
[
1−( 11
251 )
]
251−11
∗( 11
251 )
11
= 0.21
Finally, we test the LR POF for the portfolio that had a total of twelve occurrences
where daily portfolio returns or losses were greater than the estimated VaR at the
99% confidence level during the 251 trading days.
LR POF99= -2ln
(
(1−0.01)251−12 ¿0.0112
)
[
1−( 12
251 )
]
251−12
∗( 12
251 )
12
= 18.94
Kupiec’s TUFF is a test that quantifies the actual time it takes first exception to
transpire (Kupiec, 1995). Similarly to how it was done for Kupiec’s POF, values of
TUFF are derived for each level of confidence that was performed in figure 14.
Results are summarized in figure 16:
Figure 16: Number of days (“time”) until first exception
Confidence Level Number of days until first exception
90% 43
95% 43
99% 29
The second step in the PUFF test is to calculate the Likelihood ratio, by plugging in
the necessary data into the formula (7):
First, a likelihood ratio is calculated at the 90% level of confidence. In this case, the
amount of „p“ equals to 0.1 (p=1-0.90). The likelihood ratio equals to:
LR TUFF90 = 2ln
(
0.1¿(1−0.1 )43−1
)
1
43∗(1−1
43 )
43−1
= 3.9564
Secondly, the likelihood ratio is calculated at the 95% level of confidence. In this
case, the amount of „p“ equals to 0.05 (p=1-0.95). The likelihood ratio equals to:
LR TUFF95 = 2ln
(
0.05 ¿(1−0.05)43−1
)
1
43∗(1−1
43 )
43−1
= 0.8011
Lastly, the likelihood ratio is calculated at the 99% level of confidence. In this case,
the amount of „p“ equals to 0.01 (p=1-0.99). The likelihood ratio equals to:
LR TUFF99 = 2ln
(
0.01¿(1−0.01)29−1
)
1
29∗(1−1
29 )
29−1
= 1.0734
Basel Traffic Light Approach uses the same data set as the Kupiec’s POF test
uses for making a conclusion about VaR model accuracy, meaning a number of
exceptions at each of the levels of confidence for the same period.
Figure 17: Basel Traffic Light Approach
Level of confidence Number of observations Number of exceptions
90% 251 9
95% 251 11
99% 251 12
2.) Conditional coverage is the second group that consists of two tests:
Christoffersen’s Independence Test is applied in order to examine if the
exceptions are equally distributed in time period. For the purposes of this test, the
contingency table, which is already explained in the Methodology part, has to be set
up. By using the available and necessary data, in this case at the 90% level of
confidence, the contingency table is presented below (figures 18 to 20)
Figure 18: Contingency Table at 90%
Figure 19: Contingency Table at 95%
Figure 20: Contingency Table at 99%
Now, once the contingency tables are set up for 90%, 95% and 99%, the probability
values π0, π1, π2 can be computed by factoring in the data from contingency tables
into the formulas (8, 9, 10):
Figure 21: Independence Test
The calculations are summarized in the figure 21. The final step is to calculate the
Likelihood ratio by applying the formula (11).
Test 1:
LRind = -2ln
(1−π)n00+n10∗πn01 +n11
¿
(
¿¿(1−π0)n00∗π0n01∗
(
1−π1
)
n10∗π1n11
)
=
LRind = -2ln
(1−0.035857)233 +9∗0.0358579+0
¿
(
¿¿(1−0.037190)233∗0.037190 9∗
(
1−0
)
9∗00
)
= 0.6695
Test 2:
LRind = -2ln
(1−0.043825)230 +10∗0.04382510 +1
¿
(
¿¿(1−0.041667)230∗0.041667 10∗
(
1−0.090909
)
10∗0.0909091
)
= 0.4765
Test 3:
LRind = -2ln
(1−0.047809)223+11∗0.04780911+1
¿
(
¿¿(1−0.046025)228∗0.04602511∗
(
1−0.083333
)
11
∗0.0833331
)
=
0.2916
Mixed Kupiec Test was suggested by Haas (2001) and as a first step it takes into
consideration the time between two exceptions. (Haas, 2001)
Test 1:
At 90% level of confidence (which is already calculated and presented in figure 13:
One day VaR & Exceptions), there are 9 exceptions that occur, meaning that 9 times
the value of loss exceeded the estimated VaR amount. Figure 22 shows the time
between each exception at 90% level of confidence:
Figure 22: Time between exceptions (at 90%)
Now, by applying the formula, it is possible to get the values of likelihood ratio at
90% level of confidence for each exception as shown in figure 23
Figure 23: LR statistics for exceptions (at 90%)
The final step is to sum up the LR statistics for all exceptions by using the formula:
LRind = 8.26+0.25+0.04+0.01+0.56+1.27+0.74+0.04+5.47 = 12.33
Test 2:
At 95% level of confidence, 11 exceptions occur as shown in the figure 24.
Figure 24: Time between exceptions (at 95%)
After plugging in the data from the figure 24 into the Formula 12, the computed
likelihood ratios at 95% confidence level for each time the exception occurred is in
figure 25:
Figure 25: LR statistics for exceptions (at 95%)
The final step is to sum up the LR statistics for all exceptions by using the formula
LRind = 0.80+1.10.0.24+0.53+0+0.12+2.38+0.06+1.80+0.24+1.34 = 8.60
Test 3: At 99% level of confidence, 11 exceptions occur. Time between each
exception is presented below in figure 26.
Figure 26: Time between exceptions (at 99%)
As time between two exceptions is known, the next step is to calculate the likelihood
ratio at 99% level of confidence:
Figure 27: LR statistics for exceptions (at 99%)
Finally we sum up the LR statistics from figure 27 for all exceptions by using the
formula
LRind = 1.07+2.40+3.90+2.55+3.09+0+2.27+5.43+1.30+4.77+2.55+0.35 = 26.68
5. Results
As far as the Point of Failure results, we can say that calculated likelihood ratios
(those at 90% and 99% confidence) are considerably higher than the critical value.
Model is rejected at 90% level of confidence as the calculated LR test is 14.86 which
is significatly higher than the critical value 3.84. We can therefore say that the null
hypothesis is rejected with 90% of confidence. Similar results came out for 99%
confidence level as the LR test is 18.94 which is, again, much higher than the critical
value. We can conclude that after calculation of ratio levels at given levels of
confidence, percieved rates of failure are quite different from the ones suggested by
the confidence interval rate of failure – to simplify even more it can be said that for
these two tests estimation of value at risk significantly downplay and falsely minimize
the risk. Same can't be concluded for the portfolios at 95% confidence level where
we saw that likelihood ratio is definitely lower than critical value. Time until first
failure results showed that likelihood ratio at 95% and 99% level of confidence is a
significantly below the critical value (p=1-c p=1-0.95=0.05) 3.84. In other words,
calculated values of 0.80 and 1.07, are below the critical value. This indicates that at
the 90% and 99% level of confidence, the null hypothesis stating that the probability
of an exception is the inverse probability of the confidence level for VaR is accepted.
Since the null hypothesis is not rejected, that means the model is considered
accurate. On the other hand, at 90% level of confidence the calculated likelihood
ratio equals to 3.96 and is great than critical value for 0.12. Even though the
difference is not significantly above the critical value, the model is rejected with 90%
of confidence. Basel traffic light approach suggests underestimation of risk at 99%
level of confidence, since the number of observations equals to 12, which according
to the Basel Committee is an inaccurate model. Test does not have results in a
yellow zone which would indicate a possible problem with the model. However, at
90% level of confidence number of exceptions equals to 9 which falls between 0 and
32 resulting with the green zone. The same is with the 11 exceptions at 95% level of
confidence: model is in the green zone. Even though results are indicating a green
zone model, more tests would be needed in order to discuss the quality and
accordance of the model. Christoffersen’s independence test showed that computed
likelihood ratios at 90%, 95% and 99% confidence level are way below the critical
value indicating that the null hypothesis, which states that an exception today does
not depend on whether an exception occurred the day before, cannot be rejected.
Based on the results, it can be said that no dependence is present between the
exceptions at all of three confidence levels.
Finally, critical values used for Mixed Kupiec-test are assigned by the number of
exceptions for each level of confidence. As can be seen from the table below, the
calculated likelihood ratio at 99% level of confidence exceeds the critical value by
8.65. Since this computed value is higher than the critical value, the null hypothesis
is rejected. By rejecting the null hypothesis, the conclusion about model inaccuracy,
indicating that the independence property is not satisfied. Moreover, at 90% and
95% level, test statistics for likelihood ratios are smaller than the assigned critical
values and the model is accepted indicating that exceptions are independent of each
other.
Tabular format of results from all tests can be seen in figure 28:
Figure 28: Summarized results
Confidence
Level
Obser-
vations
Except-
ions POF Test TUFF
Test
Traffic
Light
Approach
Christof-
ersen's
Test
Mixed
Kupiec-
Test
90% 251 9 Reject Reject Green Accept Accept
95% 251 11 Accept Accept Green Accept Accept
99% 251 12 Reject Accept Red zone Accept Reject
6. Conclusions
Today’s firms are in constant cope with all kinds of risks and are on everlasting
venture to manage and mitigate it. In financial institutions it is very popular to use
value-at-risk in order to gauge cash reserves they might need to cover potential
portfolio losses. Value at risk forecast is a procedure that is widespread across the
world mostly in investment and commercial banks. Since the method which produces
a highly accurate forecast doesn't exist, certain backtesting procedures must be
performed in order to verify wether VaR results are adequate or not. Hence,
backtesting procedures are an absolute necessity, and we concluded that more
backtests must be performed in order to confirm a veracity and trustworthiness of
VaR model. Such fact suggests that backtesting must be an integral part of every
day VaR computations. Results that this research had obtained, are able to provide
an intelligence if potential flaws or risks exist in firm’s essence, so that management
can address those and perform a necessary risk alleviation procedures in order to
protect the firm agaist potential and future risks. Surely, back testing process
deserves a top spot in reporting regulation of any financial institution, as it is an
essential part of process that validates whether value at risk techniques that firm
uses, are in fact ensuring consistent and reliable forecasts.
References
Bams D., Blanchard G., Lehnert T., (2017), ‘Volatility measures and Value-at-risk’,
International Journal of Forecasting, Elsevier, vol. 33(4), pgs 848-863
Brown, A., (2008), 'Private Profits and Socialized Risk – Counterpoint: Capital
Inadequacy', Global Association of Risk Professionals, 42, pgs 10-26
Campbell, Sean D., (2005), ‘A review of backtesting and backtesting
procedures’, Finance and Economics Discussion Series 2005-21, Board of
Governors of the Federal Reserve System (US), revised 2005.
Christofferssen, P., (1998), 'Evaluating Interval Forecasts', International Economic
Review, 39, pgs 764-787
Emmer, S., Kratz, M. & Tasche, D., (2015), 'What is the Best Risk Measure in
Practice? A Comparison of Standard Measures', Journal of Risk Model Validation
18(2), pgs 31-60
Graham, A., & Pal, J. (2014). 'Backtesting value-at-risk tail losses on a dynamic
portfolio', Journal of Risk Model Validation, vol8 no2 pgs 59-96
Haas, M., (2001), 'New Methods in Backtesting', Financial Engineering, Research
Center Caesar, Bonn 2001
Jorion, P., (2001), 'Value at Risk, The New Benchmark for Managing Financial Risk',
2nd Edition, McGraw-Hill, United States
KPMG [online] http://kpmg.com/ „Financial Risk Management Backtesting Value at
Risk Models“ (Accessed 21 Sept 2019)
Kupiec, P., (1995), 'Techniques for Verifying the Accuracy of Risk Management
Models', Journal of Derivatives 3 (2) pgs 73-84
Markowitz, H., (1991). 'Foundations of Portfolio Theory', The Journal of Finance,
46(2), pgs 469-477
Nelson, D.B., (1991). 'Conditional Heteroscedasticity in Asset Returns: A New
Approach'. Econometrica 59 (2), pgs 347-370
Paseka A., Thavaneswaran A., Frank J., (2018), ‘Generalized Value at Risk
Forecasting’, Communication in Statistics – Theory and Methods 47 (25), pgs 189-
207
Pflug, G., (2000), ‘Some Remarks on the Value at Risk and the Conditional Value at
Risk’, Probabilistic Constrained Optimization, Kluwer Academic Publishers, ed 1, pgs
272-281
Yahoo Finance [online], http://finance.yahoo.com, Yahoo Finance - Business
Finance, Stock Market, Quotes, News, (Accessed Aug 9, 2018)
Yamai, Y., (2002), 'Comparative Analyses of Expected Shortfall and Value at Risk
under Market Stress', Monetary and Economic Studies, Institute for Monetary and
Economic Studies, Bank of Japan, vol. 20(3), pgs 181-237
Zhang Y., Nadarajah S., (2017), 'A review of backtesting for value at risk',
Communication in Statistics, Theory and Methods 47 (15), pgs 3616-3639