Tag: Policy Analysis

SNAFU

The Army doesn’t like RAND’s study on postwar planning. Too bad.

The Army has buried a comprehensive study of the planning for postwar Iraq, conducted by RAND.

The study chided President Bush — and by implication Secretary of State Condoleezza Rice, who served as national security adviser when the war was planned — as having failed to resolve differences among rival agencies. “Throughout the planning process, tensions between the Defense Department and the State Department were never mediated by the president or his staff,” it said.

The Army’s explanation sounds weak:

“After carefully reviewing the findings and recommendations of the thorough RAND assessment, the Army determined that the analysts had in some cases taken a broader perspective on the early planning and operational phases of Operation Iraqi Freedom than desired or chartered by the Army,” Mr. Muchmore said in a statement. “Some of the RAND findings and recommendations were determined to be outside the purview of the Army and therefore of limited value in informing Army policies, programs and priorities.”

This is a familiar story, and redounds entirely to RAND’s credit. When I worked at RAND I had a similar experience with a research sponsor that didn’t like the results and tried to sweep the report under the rug. RAND’s leadership stood its ground, and the report was published. This is not the first time that service brass or OSD or JCS has been irked by RAND’s findings (most famously, the gays in the military study.) And still, most otherwise well-informed people I meet regard RAND as a right-wing propaganda shop. It’s nothing of the sort. It’s so non-partisan and non-ideological that it’s not even centrist. It’s a national treasure.

End of rant.

Let Them Eat Cake (but don’t let them reheat it)

Don’t kill LIHEAP-fix it.

Much has been made of John McCain’s perhaps too-candid admission that

The issue of economics is not something I’ve understood as well as I should….I’ve got Greenspan’s book

and that Phil Gramm is his principal advisor on economics. That’s reassuring.

And he’s been especially vocal on spending cuts:

The campaign has also said Mr. McCain would consider cutting the programs that the White House has identified as ineffective, which together make up 10 percent of the budget. The campaign has not specified which ones it would cut. In addition to Amtrak, the list includes various programs dealing with Defense Department communications, veterans’ disability and low-income heating assistance.

That’s a curious way of phrasing it: Under “not performing,” the White House includes those programs found to be “ineffective” and those for which “results not demonstrated.” (You can find out what the White House thinks of the cost-effectiveness of various federal programs at http://www.whitehouse.gov/omb/expectmore/index.html.) Heating assistance is in the latter category.

The GOP’s distaste for Amtrak is longstanding and well known. Less well publicized is their animus towards poor, largely old, cold people. The Low Income Home Energy Assistance Program (LIHEAP) is a $2B program that helps with heating bills and home weatherization, through block grants to the states. It has a complicated legislative history, and there are legitimate criticisms of the formulae for inter- and intrastate distribution of funds. All of which are amenable to fixing, absent congressional horsetrading and White House intransigence.

For what it’s worth, the White House’s assessment is that

As a block grant that provides maximum flexibility to States, the program faces challenges in developing meaningful performance measures that States can track. While energy trend data indicate that the net effect of this assistance has been to move low income household heating burdens closer to that of all households, the program lacks performance data to assess the outcome(s) of this trend.

So? Collect the performance data and assess the outcome(s). Oh, right, the White House also finds that results are “not demonstrated” for the Energy Information Administration.

I’ve yet to see a call for killing LIHEAP that offers any alternative solution to “let them freeze” other than (a) the states would take care of this on their own if not for federal handouts, (b) it’s a matter for private charity, or (c) if you can’t afford heating oil, don’t live in upstate NY.

Perversity Training

Mandatory diversity training can feel pointless, but does it actually reduce workplace diversity?

In my experience, which I’ve been perfectly willing to generalize, mandatory workplace sensitivity training is a scam and a waste of time. The people who don’t need it are checking their watches, the people who do need it are fuming and not listening, and the trainers are speaking to the employees as if they were dull-witted third graders. It hadn’t occurred to me, however, that it would have perverse unintended consequences.

Shankar Vedantam writes (in his typically informative column on social- and behavioral-science research findings) that

A comprehensive review of 31 years of data from 830 mid-size to large U.S. workplaces found that the kind of diversity training exercises offered at most firms were followed by a 7.5 percent drop in the number of women in management. The number of black, female managers fell by 10 percent, and the number of black men in top positions fell by 12 percent. Similar effects were seen for Latinos and Asians.

The analysis did not find that all diversity training is useless. Rather, it showed that mandatory programs — often undertaken mainly with an eye to avoiding liability in discrimination lawsuits — were the problem. When diversity training is voluntary and undertaken to advance a company’s business goals, it was associated with increased diversity in management.

I’ve felt the same way about mandatory sexual-harassment training, but I don’t find any studies on whether it’s followed by increased sexual harassment.

The academic estate and the public debate

Just because your first name is “Professor” doesn’t make you an expert on everything.

In general, academic specialists in foreign policy, strategy, and Middle Eastern affairs made much better guesses about what would happen if we invaded Iraq than did politicians and pundits. (Yeah, yeah, I shoulda listened. Sorry, sir! Won’t happen again, sir!)

And yet “ivory tower” remains an unanswerable insult in political discourse, as if journalists and politicians were proud of their ignorance. Many academics don’t speak out much in public fora, even in areas of their expertise. Why doesn’t the academic estate do more to claim its rightful voice in public affairs, and why, when it does, is it so little heeded?

I think there are two key distinctions here that are often lost: the distinction between an expert’s proper authority in his own field of expertise and a general claim by people with faculty appointments to opine about public affairs, and the distinction between research and policy analysis.

Academics are, by nature, specialists. In general, the claim of specialists to offer expert opinion outside their specialities is to be treated with skepticism. (Socrates made that point, if I recall correctly.)

Back in 2003, the UCLA Faculty (or, rather, the 200 people who bothered to show up for the meeting) voted its opposition to the pending invasion of Iraq. That conclusion was arrived at by vote after a short and chaotic debate (mostly among people with no scholarly credentials relevant to the choice at hand), and was not subjected to the sort of peer review or careful analysis that we require in our scholarly lives. I thought then that the resolution did not deserve the attention that, in fact, it didn’t get. By passing it as a faculty, we were illicitly claiming for our political opinions the authority that properly belongs only to our scholarly views. I still think so, though the proponents of that resolution turned out to be right in concluding that an invasion would end in tears.

That’s not to say that academics, per se, have no proper public role. Someone who studies Iraq or climate change or taxation professionally is entitled to a hearing &#8212 and, elitist though it may be to say it, to a more respectful hearing than a non-expert &#8212 if he claims that the factual premises underlying some policy are incorrect or that some likely effect is being ignored or misrepresented. Non-experts, including other academics, ought to disagree with experts, or disregard expert views, only cautiously and tentatively, unless there are comparably credentialed experts on the other side.

But, even if someone is a genuine expert in a relevant subject area, his claim to dictate the correct policy has much less force than his claim to describe what is the case and predict what is likely to happen, unless that person is also an expert in thinking about choosing good policies: an intellectual activity distinct from determining facts or framing theories to hold them together. It would be nice if the scrupulous fair-mindedness that characterizes the best scholarship also characterized the political arguments offered by scholars, but I doubt it’s so.

Read the “policy implications” section of a typical social-science paper. It rarely reflects the sort of cautious judgment about the relationship between observation and inference displayed in the “methods and results” section. That’s partly because many social scientists haven’t thought about the very different methods appropriate to policy analysis, and partly because the conventions of academia allow a certain amount of editorial freedom at that point in the paper. (Mark Moore has been making this point for years.)

To earn respectful attention to our opinions about what ought to be done, we need to learn to make those opinions intellectually respectable, which means, among other things, both carefully distinguishing what we know from what we prefer and accurately representing the limits of our knowledge.

I’m not saying that, if we do so, we will get such attention; we probably won’t. But I am saying that the attempt to use intellectual prestige, separated from serious and dispassionate critical truth-seeking, as a weapon in political struggle is no more legitimate than the use of money or celebrity as a weapon in political struggle, and less so if that attempt falsely claims the respect due to actual expert relevant knowledge.

Since academics have some capacity to lead opinion, some leisure, and some money, and since they’re mostly on the right side of the current major political divide, I’d like to see them more active in politics. I think that mobilizing the professoriate (and especially the hard scientists) in support of Democrats is a good, and possibly practicable, idea.

But when I saw an ad in the New York Times in October 1968, with a bunch of professors’ signatures under the headline “A Thousand People Who Think for a Living Think You Should Vote for Hubert Humphrey,” I thought that was arrogant bullsh*t: “elitism” in the legitimately pejorative sense of the term. And I still think so.

Update Mark Thoma is right to say that experts in one area can establish credentials to be heard in another area. More on that here.

Just to clarify for the benefit of one of Thoma’s commenters: I worked my heart out for Hubert Humphrey in 1968. But the ad I criticized implicitly said to voters “Since we’re professors, we’re smarter than you are. So we won’t bother to argue that Humphrey is the better candidate; just take our word for it. That was, and is, a lousy, arrogant argument.

Creating Competitive Markets

The name of this blog comes from a now-infamous statement by a senior Republican staffer that Republicans were part of the “faith based community,” as distinct from the Democrats’ “reality based” foundation. This faith went far beyond foreign policy, extending into vast reaches of domestic policy. From school choice to Medicare Part D to the president’s abortive Social Security privatization plan, Republicans have pushed ambitious programs to inject markets into new areas of social policy. Some of these made sense-others didn’t. But what characterized many of them was not analysis, but faith-a faith that markets, once “unleashed” (choose your alternative jargon, if you wish) would actually serve the goals of the programs better than state provision. That is, having failed to dismantle the programs, conservatives accepted (rhetorically at least) their goals and claimed that market-like mechanisms would be more effective means to achieving them.

I have a chapter (with Alan Jacobs of UBC) in a new book entitled Creating Competitive Markets, which looks at market making as a policy design problem. Using a wide range of cases, and with a pretty impressive set of authors (more impressive, that is, than me!) the book carefully interrogates when and under what conditions market making actually works. One of the findings of my chapter with Alan is that a key obstacle to effective market-making is excessive belief in markets on the part of policy-makers. That is, the same faith that leads market-makers to want to privatize or deregulate in the first place may cause them to fail to engage in skeptical and serious analytical design work to ensure against the risks of market-making. The consequence of this is that “faith-based” market makers are more likely to create policies that cause unintended consequences, thereby leading to substantial (often heavily market-distorting) changes after the fact, when the public is demanding that they clean up the mess.

While my chapter is in this book, and obviously I am not without substantial conflict of interest in this matter, I have to say that this is one of the most coherent edited books I know about. Policy-makers looking for insight as to when and how to apply markets to public policy problems would be well-advised to read it closely. What is more, the editors made certain that the chapters were concise and written with the non-specialist in mind (my chapter got 1/3 cut out of it!). There’s a copy of the introductory chapter on-line here.

Concerning organizational recklessness

Tom Schelling explains the Enron affair, the war in Iraq, and the Foley Follies: an organization may act recklessly not because it’s full of reckless people, but because it’s so full of cowards that no one dares say, “We can’t get away with this.”

Tom Schelling’s many friends are taking the occasion of his Nobel Prize to celebrate him in various ways. Last week was the Schelling Symposium organized by Peter Reuter at the University of Maryland, where I am spending the fall as the (ahem) first Thomas C. Schelling Visiting Professor of Public Policy. Tonight was a Forum event and dinner at the Kennedy School. Both were populated by star-studded groups of thinkers, and it’s been fascinating to hear about the many different lessons the speakers have taken away from talking to, and reading, Tom.

The central intellectual themes seemed to be:

* The common interest of adversaries in keeping conflict within limits;

* The problem of credible commitment and the advantage of closing off options;

* Tipping;

* Failures of cooperation in multi-player games;

* Focal points;

* The importance of small incentives;

* Mixing and sorting;

* The divided self and the strategic problem of self-command.

That’s an impressive roster of general ideas; personally, I’d sell my soul to Satan to have come up with even one idea that size. But equally impressive were the multiple flashes of illumination about wide ranges of behavior.

My own favorite Schelling insight is from an essay, which I read years and years ago and whose title I’ve forgotten, on the topic of organizational command and control. I can’t quote it precisely, but I find the thought coming up again and again: it applies, for example, to the invasion of Iraq, the collapse of Enron, and the failure of the House Republican leaders to get a handle on the Foley Follies before the situation blew up in their faces. I’m confident that what follows is a fairly close paraphrase:

When you see an organization acting recklessly, don’t leap to the conclusion that the people running the organization are themselves reckless. Perhaps they’re all such cowards that none of them dares to say to the others, “We can’t get away with this.”

What Can Ethnography Contribute to Public Policy, And How?

In partial response to Mark’s question below about anthropology, I’d like to suggest that our policy-oriented readers take a look at Robin Rogers-Dillon’s fine book, The Welfare Experiments: Politics and Policy Evaluation. Robin is an ethnographically oriented sociologist (that is, she’s kind of at the end of the discipline where it starts to morph into anthro) who was a part of the MDRC team that evaluated Florida’s 2 year time limited welfare program. In an innovation for MDRC, Robin was hired to do ethnographic work on the evaluation, which cashed out to doing a lot of hanging out in welfare offices. To make a long story short, Robin found out that the program on the ground was very different than the program on the books. For example, there was supposed to be a work program at the end of 2 years for all “compliant” clients. Well, the number of people in the work program was zero. Why? They were all administratively declared non-compliant. There was supposed to be an appeals process for the non-compliant, but it was staffed by local citizens who, it turned out, really don’t like exercising state power, so they acted primarily as a cheering squad, telling women who were cut off that “you can do it!”

To put it in technical terms, ethnographical methods (and this would certainly include anthropology) are essential, where there is any ambiguity as to what the “treatment” is in a particular policy experiment. In some cases, ethnographic methods are also valuable in helping to figure out the mechanisms that connect treatments to effects, which is highly relevant if you want to figure out how to replicate the program under other conditions. This is another case (I could list others) where well-designed policy analysis requires multiple, quite different methods in order to be serious and complete.