Content uploaded by Steven Cohen
Author content
All content in this area was uploaded by Steven Cohen on May 04, 2015
Content may be subject to copyright.
Defining and Measuring Effectiveness in Public Management
Author(s): Steven A. Cohen
Source:
Public Productivity & Management Review,
Vol. 17, No. 1 (Autumn, 1993), pp. 45-57
Published by: M.E. Sharpe, Inc.
Stable URL: http://www.jstor.org/stable/3381048
Accessed: 21/07/2010 10:07
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=mes.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.
M.E. Sharpe, Inc. is collaborating with JSTOR to digitize, preserve and extend access to Public Productivity &
Management Review.
http://www.jstor.org
Defining
and
Measuring
Effectiveness
in
Public
Management
Steven A. Cohen
Public management research shouldJocus on improving the performance
of public managers.
This article begins with a discussion of the concepts of management and lead-
ership. I seek to integrate these two concepts by specifying dimensions of
leadership and management in which a public manager strives to be effective.
Next, I will discuss the concept of effectiveness as it applies to public manage-
ment. For each dimension of management I will propose indicators for assess-
ing effectiveness. Finally, I will discuss how these indicators might be used in
empirical studies of management effectiveness.
I want to know what is meant by the terms effectiveness and management.
The unit of analysis is the manager. I am attempting to assess effective man-
agement performance, not effective organizational performance. This em-
phasis is best represented in the management literature by Henry Mintzberg
(1973) and his focus on the work of managers, Robert Behn's recent work
(1991, 1992) 7 andby Herbert Kaufman's (1981) similarwork on the behavior
of federal bureau chiefi. It is difficult to discuss effective management without
understanding what managers do. However, management cannot be under-
stood simply by studying and describing the behaviors of managers. The so-
ciology literature in organization theory and the business literature in
management present us with a number of different slants on the definition of
management and leadership (Stogdill, 1974) . Review of this literature makes
it clear that we are looking at a complex, multidimensional concept. Each par-
ticular scholarly emphasis provides useful insight on some dimension of man-
agement. I do not believe it is necessary to choose between these schools of
thought: by drawing on all of them we can develop a comprehensive defini-
tion of management (Cohen, 1991).
For some time now scholars of private sector management have pursued
a behavioral approach to the study of management. To date) however, they
45 PUBLIC PRC)DUCTIVITY & MANAGEMENT REVIEW, VOI XVII, no. 1, Falll993 C) Jossey-Bass Publishers
46 Cohen
have failed to relate managerial effectiveness to organizational effectiveness.
The additional complexity of public management environments makes it
even more difficult to study and understand the behavior of public managers.
Nevertheless, if we do not observe the work of public managers, we cannot
hope to understand and improve their performance.
Leadership and Management
In the management literature a distinction is often made between leadership
and management. According to this literature, management is the task of set-
ting up control structures and standard operating procedures (SOPs), while
leadership involves stimulating organizational change by articulating a vision
andinspiringa sense of mission.John Kotter (1990) has done an excellent job
of defining these two concepts. Kotter (1990, p. 103) observes that "leader-
ship and management are two distinct and complementary systems of action.
Each has its own function and characteristic activities." Kotter (1990, p. 107)
concludes by noting:
According to the logic of management, control mechanisms compare sys-
tem behavior with the plan and take action when a deviation is detected. In
a well-managed factory, for example, this means the planning process estab-
lishes sensible quality targets, the organizing process builds an organization
that can achieve those targets, and a control process makes sure that quality
lapses are spotted immediately....
For some of the same reasons that control is so central to management,
highly motivated or inspired behavior is almost irrelevant. Managerial pro-
cesses must be as close as possible to fail-safe and risk free. That means they
cannot be dependent on the unusual or hard to obtain. The whole purpose
of systems and structures is to help normal people who behave in normal
ways to complete routine jobs successfully, day after day. It's not exciting or
glamorous. But that's management.
Leadership is different.... Motivation and inspiration energizes
people, not by pushing them in the nght direction as control mechanisms do
but by satisfying basic human needs for achievement, a sense of belonging,
recognition, self esteem, a feeling of control over one's life and the ability to
live up to one's ideals.
Here Kotter is describing two functions that are basic to management: (1) es-
tablishing and maintaining production systems, and (2) inspiring people to
contribute the effort needed to implement and change work processes. The
analytic advantage to separating these two functions is that it ensures that each
receives separate focused attention. However, my own view is that control sys-
tems and motivational factors must be connected in both analysis and practice.
People frequently have greater or lesser comfort with either the "system"
Defining and Measuring Effectiveness 47
side of the shop or with the "human interaction" part of the operation. How-
ever, control structures and SOPs only perform at peak efficiency when the
people within the organization are motivated to implement and change them.
Otherwise they quickly become outmoded. Indeed, Kotter agrees that both
"leadership" and "management" are needed for a successful organization.
My fear is that by not subsuming them under a single concept, we run the
risk of emphasizing one over the other in both analysis and practice. I have
seen people who hide behind their elegant control structures complaining
about the "damn fools" in the organization who simply will not play their des-
ignated role in the system. I have also seen managers who focus on vision ex-
clusively, leaving the less glamorous implementation tasks to "lesser" staff
who inevitably allow the vision to be compromised in the day-to-day world of
real work. Finally, 1 have also observed "touchy-feely" human relations ex-
perts so caught up in perfecting group processes that they ignore the impor-
tance of productionsystems and organizational outputs. In terms of analysis,
formal structures and control systems are more amenable to quantitative
study and development than are soft topics such as motivation and vision.
Academics tend to focus research on phenomena that enable them to use so-
phisticated methodologies. After all, they have invested considerable effort in
learning these techniques, and frequently enjoyusing them.
But it is possible to integrate "soft" management with "hard" manage-
ment. Total quality management (TQM) is an effort to integrate the "soft"
management techniques of organizational development and human resource
management with the "hard" quantitative techniques of statistical process
control (Cohen andBrand, 1990, pp. 100-102) . While TQM aims to improve
an organization's systems and SOPs, it utilizes staffenergy and enthusiasm to
do so. Systems development and human relations are not divorced because
SOPs are developed by staff for their own use.
In this article I am seeking to develop a multidimensional definition of
management, and then attempting to develop a method for measuring man-
agement effectiveness. I am not seeking to distinguish management from
other concepts, but instead to build a more comprehensive definition.
Defining Management
To define management I suggest we focus on two distinct units of analysis.
The first is the functions of management; the second is the attributes of man-
agers. Managers must perform the following functions:
1. Learn what the organization is doing.
2. Determine what the organization needs to do.
3. Encourage innovative ways of doing what the organization needs to do.
4. Deploy incentives to convince the organization's members to conduct the
activities that the organization needs to do.
48 Cohen
S. Represent the aspirations and fulfill the social and psychological needs of
the organization's members to the extent necessary to maintain the orga-
. .
nlzatlon.
6. Ensure that the organization accomplishes what it needs to accomplish.
To perform these functions, managers require the following skills or attributes:
1. The ability to listen, to learn, and to distinguish critical from peripheral
facts, values, and concepts.
2. A personality that matches the needs of the particular organization's so-
cial structure and internal composition, environment, and goals.
3. Skill and sensitivity in distributing resources, deploying inceIltives and
exercising power. This requires an ability to read people, interpret behav-
ior, act intuitively, and correctly mix the use of emotional appeals, reason,
incentives, and one's own power to influence organizational behaviors.
4. The ability to identify and achieve appropriate goals. In the public sector,
goal setting is often a complex act requiring the manager to juggle con-
{licting claims and ambiguous policy directions. As incremental theory
indicates, sometimes public managers are not trying to accomplish any-
thing new, but simply trying to keep a bad situation from getting worse.
S. The ability to clearly communicate roles, tasks, goals, and missions.
6. Creativity.
7. The ability to inspire.
Defining Effectiveness
According to Lynn ( 1987, p. 18), "Success in public management is a product
of the interaction of personality and circumstances: the skillful use of many
approaches structural, political, symbolic-to produce movement toward
personal goals in specific situations. Executives' beliefs and temperaments
define and impart direction to ambiguous reality. " As Lynn indicates, the criti-
cal point to understand in defining effectiveness is that it is always situational.
The definition of effectiveness will vary with different organizational environ-
ments, different organizational types, and different organizational goals. Each
of the functions and the attributes needed to perform those functions I just
listed will vary in different organizational settings. One can easily imagine a
manager in a crisis situation who has excellent listening skills, but who listens
too much and does not actwhen he or she should. Or perhaps we might imag-
ine another manager running a directionless organization: this particular
manager has terrific communication skills but has poor skills as a strategist-
the wrong situation for this person's talents.
However, I have observed some general traits shared by effective public
managers. In the Effective Public Manager (Cohen, 1988) I discussed the gen-
eral behaviors exhibited by these managers. In this section 1 will summarize
Defining and Measuring Effectiveness 49
some of the points I made in my book to provide an overview of the concept
of effectiveness.
Effectiveness, as Lynn indicates, requires craftsmanship and an ability to
give shape to vague and ambiguous policy environments. An effective public
manager seeks to shape events rather than be shaped by them. The most effec-
tive public managers learn to adjust programs rapidly to reflect changed pri-
orities. They anticipate changes in policy direction and build organizations
capable of rapid redirection. Developing management systems and under-
standing budget formulation and financial management are only part of the
job. To achieve effective management, a manager must be able to innuence
human beings to act in a desired manner. An effective manager must have
both the most up-to-date information and an excellent staff to respond to and
act quickly in a changing environment.
The most effective public managers are careful people who understand
that actions that are poorly thought through can result in unanticipated bad
consequences. However, they do not allow wise caution to degenerate into
inertia. They must learn to judge when caution is the prudent course and
when risk is necessary. This kind of judgment takes intuition, guesswork,
wisdom, and experience. Since very little is ever achieved in conflict-free
environments, a good manager must not only accept conflict as an inevitable
part of accomplishing goals, but she or he must learn to feel comfortable
working with it. The effective public manager must have flexibility, resilience,
and persistence.
Effective public management requires managers who are not afraid to take
aggressive and even risky actions to overcome constraints and obstacles. The
entrepreneurial public manager assumes risk for the sake of organizational
gain. However, the risks are made in a calculated fashion and as part of a realis-
tic (possibly unwritten) organizational strategy. Good managers must take
risks to assure performance. Effective public managers try to make things hap-
pen; they pursue programmatic goals and objectives by thinking and acting
strategically. They attempt to understandwhy things are happening andhow
things can be changed. This requires constant contact with an informal net-
work of informants who provide feedback on ideas and initiatives, and who
themselves are constantly learning, teaching, experimenting, and changing.
Effective management requires defining and redefining the mission and
activities of the organization. The manager must keep in close contact with
the needs and views of both staff and customers while designing and execut-
ing programs. It is the responsibility of the manager to ensure that the
organization's performance results in accomplishments and products that
serve public needs. This responsibility requires entrepreneurial risks. The ef-
fective public manager works continually to interpret the public's changing
needs and design creative responses to fulfill those needs. Put simply, the ef-
fective public manager has a can-do attitude and believes that all problems
have a solution; the secret is to persist until the solution is found.
so Cohen
Indicators of Effectiveness
These general sentiments are useful in providing a feel for the nuances of ef-
fectiveness, but they lack the type of specificity required for research. To pro-
vide the type of specificity I am looking for I will take each dimension of
management its functions and the attributes of managers and propose
indicators of effectiveness or at least an approach for determining effective-
ness for that particular dimension. I should mention that these indicators do
not imply any assumptions about their relative weight in assessing effective-
ness. One of the first tasks I would recommend in utilizing these indicators is
to assess the situation (the nature of the organization, its goals, and its envi-
ronment) and decide what indicators are most important. Let me also note
that I look at these management dimensions and indicators as a first step in
operationally defining effective management. They will need to be modified
in the light of empirical research on actual public managers.
Indicators of Effective Performance of Management Functions
1. Signs that indicate a manager is learning about what the organization is
doing include:
Presence of a management information and/or financial control system,
as well as evidence of actual use of the system in decision making.
Presence of processes for describing and reporting work, such as regular
staff meetings.
Evidenceofinformalcommunicationpatterns managementbyapparent
wandering around, social events, travel, phone contacts, and the like.
Evidence of changed management behavior in response to information
about what the organization is actually doing.
2. Signs that indicate that a manager is determining what the organization
needs to do include:
Presence of a process for formulating organizational strategy. This need
not (and probably should not) be a formal process resulting in a writ-
ten document.
Evidence of frequent contact with external parties for example, other
units within the organization, legislatorss other executive branch orga-
nizations, the media, academics, industry, and interest groups.
Documentation or evidence of an evolving strategy in response to
changed external conditions.
3. Signs that indicate that a manager has established systems and proce-
dures to ensure that the organization's work gets done and encourages in-
novative ways of doing what the organization needs to do include:
Presence and use of SOPs or rules governing work processes.
A record of efforts to change SOPs.
Evidence of efforts to change SOPs.
Defining and Measunng Effectiveness 51
Evidence of pilot proj ects to test innovations and documentation of adop-
tion of new policies, processes, and technologies.
4. Signs thatindicate that a manageris deployingincentives to convince or-
ganizational members to conduct the activities that the organization
needs to do include:
Documentation of methods for describing and assessing staff perfor-
mance; again, we are not looking for a formal written performanee as-
sessment system.
Evidence that the manager understands the specific incentives most ef-
fective at motivating particular staff.
Documentation of deployment of resources for incentives to encourage,
discourage, or reward specific behaviors; such incentives might in-
clude awards, new equipment, higher pay, promotions, exhortation,
praise, admonitions, warnings, and assignments.
Evidence of changed behavior as a result of the deployment of incentives.
5. Signs that indicate that a manager is representing the aspirations and ful-
filling the social and psychological needs of the organization's members
to the extent necessary to maintain the organization include:
Evidence of high morale among those directly supervised, for example7 in
terms of staff attitudes toward the organization, or staff willingness to
interact with other staff and managers after normal work hours for
both work and socializing.
Data on staff turnover among those directly supervised. There are no
quantitative rules here, but a O percent turnover rate and a 100 percent
turnover rate are equally negative indicators of effectiveness.
6. Signs that indicate that a manager is ensuring that the organization ac-
complishes what it needs to accomplish include:
Data on organizational outputs. Do standard performance measures in-
dicate a relationship between the organization's strategy and its ac-
complishments? How much of what the organization sets out to
accomplish is completed?
Evidence that the organization has anticipated problems, or otherwise
prevented damage to the organization's functions or mission. For ex-
ample, in regard to the organization's media coverage, Are negatives
effectively countered? Are the organization's problems exaggerated,
ignored, or accurately reported?
If we are to build a body of research on the effectiveness of public manag-
ers, we must first study the impact that managers have had on their organiza-
tions. The indicators I have just discussed provide a framework for that type
of research. I assume that the actions of managers are the independentvariable
and the behavior of their organizations are the dependent variables. However,
we must move beyond assumptions and directly study the behavior of manag-
ers. To directly study managerial behavior, we must identify other indepen-
52 Cohen
dent variables and seek to explain variation in managerial behavior. The unit
of analysis for this inquiry is the effectiveness of managers, not the effective-
ness of organizations. Organizational behavior is a surrogate measure for
management effectiveness; it is not the focus of this article.
Indicators That Managers Possess the Skills Needed to Be Effective
1. Signs that a manager has the ability to listen, learn, and distinguish criti-
cal from peripheral facts, values, and concepts include:
Evidence that a manager regularly changes his or her views in response to
communications from staff.
Evidence of efforts to integrate disparate facts into central themes or
findings.
Examples of incidence in which the manager communicates to staff the
organization's core values and mission, and evidence of efforts to apply
these values to organizational tasks and to use them in discarding tasks
and information seen as peripheral.
Direct observation of efforts to elicit the views of staff. Does the manager
dominate all meetings, or are other voices heard? Does the staff feel
that their input is considered when the manager makes decisions?
2. Signs that a manager has a personality that matches the needs of the par-
ticular organization's social structure and internal composition, environ-
ment, and goals include:
Measures of the manager's professional reputation among peers and
staff-especially as related to the current job; evidence of support and
opposition among the staff.
Analysis of manager personality-perhaps via Meyers/Briggs or similar
classification scheme matched to an analysis of the organization's
needs. (For example, at this phase in the organization's life, does it
need an inspirational visionary, or someone good at minding the
organization's basic "nuts-and-bolts" routines? Are there external
threats requiring someone who performs well in front of TV cameras,
or would strengths as a legislative insider be more helpful? )
3. Signs that a manager has skill and sensitivity in distributing resources,
deploying incentives, and exercising power include:
Measures of the manager's reputation for overall competence among
peers and staff. Evidence that the manager is perceived as a skillful
practitioner. Is the manager perceived as judicious, prudent, timid, ag-
gressive, or a "hothead"? Does the manager inspire staffand peers' re-
spect and admiration, fear and loathing, or scorn and pity?
Evidence of the manager's ability to correctly predict events and the be-
havior of people. Does the manager attempt to anticipate reaction to
his or her actions, for example, and how often are the manager's
guestimates correct?
Defining and
Measuring Effectiveness 53
Evidence
of the manager's
ability to correctly
predict the effect
of the de-
ployment
of resources on the behavior of key staff.
4. Signs that
a manager has the
ability to identify
and accomplish
appropri-
ate goals
include:
Evidence
that the manager
is directly involved
in setting goals and
direct-
ing activities
that contribute
to their accomplishment.
This
evidence
might include analysis of calendars and formal memos to identify
activities
related to goal
setting and tracking,
or detailed case
analysis
following
the articulation
of a goal and identification
of specific
man-
agement activities that led to staff behaviors that contributed to
accomplishing
that goal.
Evidence that demonstrates
that the manager
takes into account the
organization's
environment
and internal resources
when setting
goals
and monitoring
accomplishments.
Evidence
of external support
for the organization
directly attributed
to
the direction
the manager
has set for the organization.
Evidence
that the manager
was able to change
the organization's
direc-
tion and
outputs in response
to specific external
stimuli.
5. Signs that
a manager has the ability to facilitate
development
of systems
of work processes and to clearly communicate
roles, tasks, goals, and
missions
include:
Analysis
of the manager's ability
to organize and
structure work.
Does the
manager
have good skills at planning and organizing projects
(mea-
sured
by means of self-perceptions,
staffperception,
and documented
project
plans)? If the manager
lacks these skills, does he or she have
the self-awareness
to delegate
these functions
to someone else?
Analysis
of overlap of manager
and staff understanding
of roles, tasks,
goals,
and missions.
Analysis
of internal formal
memoranda articulating
and implementing
roles, tasks,
goals, and
missions to assess the quality of the communi-
cating
process. Does staff
understand what
the manager is trying
to say
and do? What types of questions are raised
in formal communica-
tions?
Do they indicate
resistance, poor communication,
or simply a
need for
more detail?
The amount
of training and
experience the manager has had in spoken
and written
communication.
Staff and
peer perceptions
of the quality of the
manager's communication
skills.
Expert analysis
of the manager's
communication
skills. Does the
manager
have the technical skills
in speaking and
writing needed to communi-
cate clearly?
Is the manager
able to compensate
for any lack
of formal
technical
communication
skills, for example,
through more
frequent
communication
efforts,
through an engaging
informal style,
or by us-
ing subordinates
as surrogate
communicators?
54 Cohen
6. Signs that a manager has creativity include:
The frequency that the manager's advice is sought when developing new
programs, policies, and procedures. Does staffseek the manager's in-
volvement to prevent a veto or to tap into his or her ability to frame cre-
ative proposals?
Evidence that the manager encourages experimentation to modify estab-
lished practices.
Examples of new ideas that can be directly attributed to the manager.
Staffand peer perceptions of the quality of the manager's creative powers.
7. Signs that a manager has the ability to inspire include:
Evidence that the manager makes an effort to communicate his or her vi-
sion of the organization and its role in the world.
Evidence that the manager's vision is expressed without prompting by
members of the organizations, and that it is accepted as an important
and credible guide to action.
Staff and peer perceptions of the manager's ability to inspire.
Evidence of changed organizational behavior that can be directly attrib-
uted to the manager's ability to inspire, for example, staff statements
such as "I wasn't too sure this would work, but X says it will and 1 really
believe in what she says and stands for. So I did it her way."
Studying Management Effectiveness
Obviously the concept of management effectiveness and indicators of effec-
tiveness open up a thicket of definitional and measurement issues. My pur-
pose here is not to attempt to conclude a discussion of management
effectiveness but to contribute to a dialogue. I believe that if we are to improve
management performance, we must study that performance empirically.
However, we should not build a body of research that simply describes and
categori7es management behavior. Such scholarship may have its place in less
applied disciplines, but in the field of public management we need to study
effectiveness for a purpose: that purpose is to improve the performance of
public managers. As workers in the field, we should be very interested in stud-
ies that attempt to define what we are looking for in a manager what I am
calling effectiveness and then assess management perforrnance in the light
of explicit criteria.
In some sense this is a retreat from efforts to assess organizational effec-
tiveness. Some evaluation experts, such asJoseph S. Wholey (1983), have de-
veloped practical tools for evaluating organizational effectiveness and for
obtaining what Wholey terms itresults-oriented management. " lt is clear that
studies of organizational effectiveness and operational indicators of organiza-
tional effectiveness have an important place in professional practice and
scholarship. In particular, those responsible for running organizations need
tools for measuring the performance of the entity they are responsible for.
- - -
DeQning and Measuring Effectiveness ss
However, in his recent work, Understanding and Managing Public Organiza-
tionsJ Hal Rainey (1991, p. 208) reviewed the literature on organizational ef-
fectiveness and noted that organizational experts '4emphasize the difficulty of
defining and determining organizational effectiveness.... [T]hose who set
out to study effectiveness soon realized that assessing whether an organiza-
tion does its job well involved numerous technical, economic, ethical and
ideological issues."
Studying the effectiveness of an entire organization presents many daunt-
ing practical measurement problems, although a number of evaluation ex-
perts believe that this task is feasible. An accurate empirical organizational
study must describe a large number of people and their behaviors. Research-
ers must develop practical summary measures of organizational outputs. We
must also seek methods of summarizing the behaviors that lead to particular
outputs. I believe that these measurement tasks can and will be done and as-
sume that the organization will remain an important unit of analysis in pub-
lic management research.
However, while indicators of-organizational output can be useful in un-
derstanding current levels of organizational performance, I am not certain
that there is any benefit in measuring the overall effectiveness of an organiza-
tion. How can this information be used? If you know that one organization
meets its targets more often than another) what do you know about the orga-
nization? You may know that its leadership is good at setting low targets. Or
you may know that it is good at meeting the numbers itwishes to be measured
against But you have no way of knowing which output indicator is most
important, unless you have conducted a qualitative assessment of the orga-
nization's operations.
Even if you were able to figure out that one organization performed bet-
ter than another, an emphasis on output and impact indicators would not tell
you whatled to those differentperformance levels. If you then decide to study
organizational processes in order to understand the causes of various perfor-
mance levels, you end up observing the behavior of managers in instituting
and changing SOPs. In the end you still find yourself describing and analyzing
the behavior of managers. By emphasizing the manager as the unit of analysis,
we may be able to understand the ingredients of a manager's relative effective-
ness (or ineffectiveness) and therefore teach managers how to perform better.
While studying the behavior of managers within organizations presents
some of the same problems as studying organizations, one can at least focus
observations on a finite discrete case. Identifying causal links between an in-
dividual and an organization is a difficult, but feasible research task. One can
observe different managers running the same organization at different points
in time. The same manager might be studied in a variety of settings. I am argu-
ing that public management scholars should increase their efforts to borrow
methodologies developed for individual and clinical psychological studies
and in-depth anthropological fieldwork rather than utilize the macrolevel
Cohen
56
methodologies more characteristic of sociology and political science large-
scale survey research. A manager's performance must be assessed in light of
information about a large number of subtle and complex environmental fac-
tors. What is the organization's history? What type of issue areas is it working
in? What kind of political and media visibility does the issue area generate?
What is the constellation of political support for the organization? What type
of programs does the organization implement? Is the organization's work
technically and socially ambitious (for example, space travel or homeless
families) or simple and straightforward (for example, highway construction
or processing Social Security checks)? What type of personnel and orga-
nizational capacity did the manager inherit? There are hundreds of similar
contextual variables that must be understood to measure management effec-
tiveness. This requires a methodology that allows the researcher to live and
breathe the organization he or she is studying. The methodologies appropri-
ate to this study are those that enable a scholar to become immersed in the
organization and its work. RobertBehn has termed this approach 'icase analy-
sis research." In his view, theory on management effectiveness must be built
on empirical research. According to Behn (1991, p. 11):
Case analysis research is based on the assumption that if you want to develop
a theory about effective practice you have to understand a lot of observations
of effective practice. What makes an individual an effective manager in par-
ticular situations is not at all obviousr logical. If you reallywant to know
the best way to lead an organization to the top of sand dunes, you have to
observe a lot of organizational efforts to climb sand dunes. And then you
have to cull out the commonalities of success at least of particular kinds of
success in particular conditions and attempt to develop a theoretical
framework to explain such limited and conditional successes.
Public management research to date has focused on organizational per-
formance and developing indicators of organizational success. While I believe
that these indicators are useful as one method (hopefully among many) for
measuring current levels of organizational performance, such indicators have
been overinterpreted and misinterpreted. The effort to develop output and
impact measures that serve as a surrogate bottom line for public sector orga-
nizations can lead to artificialbean counting. Given the incremental nature of
public sector problem solving, the results we sometimes manage for are diffi-
cult to measure using the type of management indicators I have seen used in
government. I worry about goal-setting efforts that result in meaningless or
misleading organizational output and process indicators. This is not to say
that management information systems (MIS) do not have a role in public or-
ganizations. Tracking the current level of an organization's performance is an
essential element of any effort to improve performance. However, the use of
this data to set and manage against numerical goals seems to distort organiza-
Defining and Measuring Effectiveness 57
tional performance by driving out less quantifiable priorities and creating the
type of last-minute rush typically seen in Russian factories.
Finally, as an educator of future public managers, I am attracted to the
manager as a unit of analysis because I teach future managers. I do not teach
organizations. Research on the effectiveness of managers may provide me
with insights I can use in teaching future public managers. I believe that this
type of research focus has greater potential for helping to create a research tra-
dition more closely linked to professional practice.
In a strict behavioral sense, organizations do not 4'behave" but managers
and staff do. Improving organizational performance is not simply a matter of
manipulating organizational and control structures, legal authority, and fi-
nancial resources. Each of those factors can be seen as independent variables
that influence the behavior of management and staff. If we are attempting to
improve management effectiveness, we are attempting to stimulate manage-
ment behaviors, and those behaviors should be the ultimate dependent vari-
able in our analysis.
References
Behn, R. D. "Management by Groping Along."Journal of Policy Analysis and Management, 1988,
7 (4), 643-663.
Behn, R. D. "Case-Analysis Research and Management Effectiveness: Learning How to Lead Or-
ganizations Up Sand Dunes." Paper presented at the National Public Management Research
Conference, Maxwell Schoolt Syracuse University? Syracuse, N.Y., Sept. 19-21, 1991.
Behn, R. D. ';Management and the Neutrino: The Search for Meaningful Metaphors." Public Ad-
ministration Review, 1992, 52 (5), 409-419.
Cohen, S. The Eflective Public Manager: Achieving Success in Government. San Francisco: Jossey-
Bass 1988.
Cohen, S. "Measuring Effectiveness in Public Management." Paper presented at the 1991 annual
research tneeting of the Association of Public Policy Analysis and Management, Bethesda, Md.,
Oct. 24-26, 1991.
Cohen, S., and Brand, R. "Total Quality Management in the U.S. Environmental Protection
Agency.' Public Productivity andManagement Review, 1990, 14 (1), 99-113.
Kaufman, H. TheAdministrative Behavior of Federal Bureau Chiefs. Washingtonw D. C.: Brookings
Institution, 1981.
Kotter,J. P. "What Leaders Really Do.'l Harvard Business Review, l99Q, 68, 103-111.
Kotter,J. P. Corporate Culture and Performance. New York: Free Press, 1992.
Lynn, L. Managing Public Policy. Boston: Little, Brown, 1987.
Mintzberg, H. TheNatureofManagerial Work. NewYork: HarperCollins, 1973.
Rainey,H. G. UnderstandingandManagingPublicOrganiz.ations. SanFrancisco:Jossey-Bass, 1991.
Stogdill, R. M. Handboole of Leadership. NewYork: Free Press, 1974.
Wholey,J. S. Evaluation and Effective Public Managemenf . Boston: Little, Brown, 1983.
Steven A. Cohen is associate dean of the School of International and Public Affairs and direc-
tor of the Graduate Program in Public Policy andAdministration at Columbia University.
