Figure 3 - uploaded by Dezon K Finch
Content may be subject to copyright.
Source publication
Software cost estimation remains a difficult challenge despite decades of attention by both researchers and practitioners. Predictions are often inaccurate and characterized by very wide confidence intervals. Direct approaches base "expert" estimates on detailed requirements, along with the experience and intuition of the estimator. The Delphi meth...
Citations
... Williams et al. [14] discuss the criticality of extended stakeholders to software delivery-including marketing, sales, services, strategy, support, legal, executives, and customers-and propose a framework for enterprise software development coordination. Berndt, Jones, and Finch [2] also outline the need for coordination support across the extended stakeholder community and discuss information markets as a possible solution. ...
... The company has also studied the trading behavior of its employees within the markets [15]. Google's studies suggest two research objectives for software development: 1) test whether market aggregation of stakeholder estimates can produce useful software project indicators [2]; and 2) test whether the dynamics of stakeholder participation in an aggregate estimate can reveal additional project indicators. These objectives represent methods for eliciting, collecting, and interpreting private information. ...
Coordination of project stakeholders is critical to timely and consistent software delivery. In this short paper we present the problem of private information as a guiding framework or lens through which to interpret coordination dynamics within software organizations. We provide evidence of this problem in the form of specific challenges, collected via interviews from a diverse set of extended (i.e., non-development) stakeholders in a globally distributed software development organization.
... Market (www.milestonemarket.org) at the University of South Florida is being deployed for software cost estimation and software project management where market contracts are defined for each set of milestones, and are tied to defined cost and time estimates (Berndt, Jones and Finch, 2006). ...
This dissertation employs both design science and behavioral science research paradigms to investigate an emerging form of technology-enabled human collective intelligence known as information markets. This work establishes a conceptual foundation for the study of organizational information markets and the design and use processes of information markets inside organizations.
This research conceptualizes markets from an information systems perspective and presents an information systems research framework for organizational information markets. This work develops a systems theory of information markets to facilitate investigation of the relationships and interactions between markets as systems and their context of use. It proposes a structuration model for design and use of IT artifacts in organizations and applies it to the study of information markets. A framework of market users is developed to guide market design to satisfy the different motivational and informational needs of market users. A design based solution is proposed to an important open question in the information markets literature; how to generate sufficient uninformed trades. This research extends structuration theory by developing the structuration model of technology-induced organization development.
A well-designed information market can generate several benefits to organizations that contribute to their growth and development. Due to the importance of software in everyday life, and the high costs and percentages of failure in software projects, this dissertation proposes an information market solution to help organizations better manage the risks facing software projects. It also develops a theoretical framework for the determinants of software project risk assessment accuracy and evaluates the market‘s efficacy in improving assessment accuracy via the use of controlled laboratory experiments.
The results of the experiments demonstrate the market‘s efficacy in improving assessment accuracy by increasing the currency, accuracy and completeness of reported status information about project main objectives such as cost, schedule, performance and functionality. The results also demonstrate the market‘s efficacy in increasing individual willingness to report negative status information by decreasing their perception of information asymmetry between them and management/clients, and by increasing their perception of both the anonymity of the reporting mechanism and their perceived self-interest in reporting negative status information.
... A market may be especially powerful if the traders are diverse in their backgrounds, independent of each other, and have local knowledge (Surowiecki, 2004). Inspired by Surowiecki, Berndt, Jones et al. advocate the use of decision markets in software effort estimation (Berndt et al., 2006). They stress that by allowing all project stakeholders to participate in the decision market, one ensures diversity in the input to the estimation process and aggregates the knowledge from all the project stakeholders. ...
... We have not managed to find any empirical research on the use of decision markets for software estimates. However, a recent paper by Berndt et al. (2006) describes an ongoing study. Studies on the combining of estimates for student tasks have shown some positive effects, both when combining estimates statistically (Höst and Wohlin, 1998) and in face-to-face discussions (Passing and Shepperd, 2003). ...
When producing estimates in software projects, expert opinions are frequently combined. However, it is poorly understood whether, when, and how to combine expert estimates. In order to study the effects of a combination technique called planning poker, the technique was introduced in a software project for half of the tasks. The tasks estimated with planning poker provided: (1) group consensus estimates that were less optimistic than the statistical combination (mean) of individual estimates for the same tasks, and (2) group consensus estimates that were more accurate than the statistical combination of individual estimates for the same tasks. For tasks in the same project, individual experts who estimated a set of control tasks achieved estimation accuracy similar to that achieved by estimators who estimated tasks using planning poker. Moreover, for both planning poker and the control group, measures of the median estimation bias indicated that both groups had unbiased estimates, because the typical estimated task was perfectly on target. A code analysis revealed that for tasks estimated with planning poker, more effort was expended due to the complexity of the changes to be made, possibly caused by the information provided in group discussions.
... Data for this study was collected during experiments using a web-based information market mechanism designed for cost estimation and other project management tasks (Berndt et al. 2006). The Milestone Market (www.MilestoneMarket.org) is structured like a futures market in which participants buy and sell contracts in project delivery dates or effort estimates. ...
Information Markets have been shown to accurately predict future uncertain events and could be used as decision support systems. The design features for Information Markets are far from established. By understanding trader behavior patterns we can evaluate their effect on user surplus, then we can make changes in the markets design to encourage those behaviors that add to the markets predictive accuracy. We show that there are differences in trader behavior patterns and demonstrate a systematic way of uncovering groups who share common trading strategies. We also demonstrate how those differences influence trader surplus.
Purpose
The purpose of this paper is to contribute to the growing body of research in prediction markets by using trading data as a means of characterizing trader behavior in these markets. Traders are placed in homogenous groups based on common Trading habits using clustering algorithms. Several behavioral themes are used to guide the analyses.
Design/methodology/approach
Several market experiments were run to collect trading data, which was then exported into a data warehouse. A secondary data analysis is performed on variables derived from the original trade data. In particular, k‐means clustering is used to form groups of traders that share common characteristics.
Findings
Participants can be classified into homogenous groups based on their trading behavior. Groups tend to differ based on the overall level of participation, how much of their trading activity is spent buying or selling, and how early they enter the market.
Research limitations/implications
More research should be done using a variety of variables to further determine the impact of various types of trader behavior on prediction markets.
Practical implications
Using insights gained from work like this, the design of prediction markets can be fine tuned to encourage behavior that contributes to trader participation and the overall accuracy of the market predictions.
Originality/value
Little research has been done to evaluate the impact of trader behavior on the accuracy of prediction markets. The authors used a new prediction market implementation to collect detailed trading data. This data was then used to derive higher level trading attributes that can he used to characterize traders. The k‐means clustering algorithm was shown to be an effective means of defining groups of traders who share common market behaviors.
