Sunday, August 30, 2015

How Many Seagulls Does it Take to Create A Pooptastrophe? Updating Counting Crows

One way to know how many crows are on a telephone wire is to count them. If fact, it may be the only objective way to know how many crows are on the wire. What do you know about the crows once counted? You know how many crows there are. Do you know if they are wonderful for people too look at? Are they tormenting some poor kitty cat? Really all you know is the number of crows. What about the impact of the crows? You would laugh if I told you that counting them tells you anything about their impact, importance, usefulness, or whatever.

In the most recent effort to rescue citation counting as a measure of the importance of legal scholarship from being completely disregarded by all but a group of people who teach law, the gang at St. Thomas has published another work that purports to measure the impact of scholarship by counting the number of citation regardless of what what a work is cited for. The jump from counting to impact is a hard one since there appears to be no separate definition of impact. In the tautological world  of citation counting, counting equals impact and impact equals counting. And, impact is reserved for citations by other legal scholars only. Citations or impact on courts (it's all the same thing to counters) is irrelevant. Your school could have 27 Supreme Court citations and 13 citations in the Bosco State Law Review and your schhool would be ranked higher in impact than a school with 0 Supreme Court citations and 14 Bosco State citations.

I thought initially that the authors were pretty good at counting but I am not so sure. For example,  if you are the editor of a book of, say, 30 articles, you (and your school)  will be cited every time one the articles is cited, regardless of your contribution, including if it was strictly administrative. If you an editor or coauthor of a prexisting treatise or book of any kind, you are cited and, thus, have had an impact whether or not you had anything to do with the material that is cited. In fact, I am beginning to think cite counting may be the worse measure of impact.

In some ways, the St. Thomas effort may reflect too much time at the counting punch bowl. First, in an effort to illustrate that scholarship does not interfere with teaching,  they cite studies showing the absence of a negative correlation between writing and teaching "quality" which is measure by -- you guessed -- student evaluations. My personal hunch is that writing does not detract from teaching but,  I wonder if they missed the numerous studies showing that teaching evaluations by students are rarely correlated with actual learning. What I took from their analysis is that writing was consistent with being a good entertainer in the classroom.

Second, as a demonstration of their objectivity they select the most recent 5 year period completely arbitrarily. As best I can tell this is done for ranking the schools as well as individual faculty. I am not sure it makes sense for ranking schools. For individual faculty, it is to reduce the impact of oldsters like me. In a sense this makes individual rankings more current although one wonders about the difference between one or two citations without regard for judicial citations or the nature of the citation -- in text, an aside etc. (This is all the less excusable since WestLaw now includes cites by scholars, judges, administrative agencies and even documents filed with courts.)

 The problem with simply counting arises more importantly when they include  new lateral hires who have written nothing or very little at their new schools. What does this mean exactly?  The only thing that it can mean is that the school that was left has less scholarly impact (but how could it?) even though every or most citations are to  works produced at that school. At the same time, the new school gets impact credit even though that particular scholar has written nothing (and may never) at the school to which the impact is now assigned -- so much for the accuracy of counting and ranking the scholarly impact of school or even its current faculty.

Since they must defend the citadel of counting,  they are obliged to take on a recent study by Amy Mashburn and myself attempting to determine not counts (which are highly correlated with where you teach, where you published, and where you went to school) but whether you were cited for anything that seemed to influence another author. I guess you might all this "actual impact" as opposed to "faith based impact" on which citation counting is based. No doubt our effort is fair game, as is any subjective effort. And, as you would also expect from legal scholarship cheerleaders (all of whom are  on the team and almost none of whom are not law teachers)  we are accused of being too conservative in our labeling. For example, if someone quoted someone else as saying "the common law is complex," we did not regard the author of that incisive statement as having a significant impact on the citing author. We were even criticized for selecting a sample composed of the articles most likely to be cited and to have an impact. That critic, who was uncomfortable with our findings, actually suggested a sample of articles that would result in a worse outcome in terms of "impact."
In our work we were particularly worried about hearsay and appeals to authority.  For example, how about this: "Citation counts objectively measure impact," with the following footnote:"See David L. Schwartz & Lee Petherbridge, The Use of Legal Scholarship by the Federal Courts of Appeals: An Empirical Study, 96 CORNELL L. REV. 1345, 1354 (2011) ( saying in study of citations of legal scholarship in court decisions, “measuring the use of legal scholarship by measuring citations in opinions has the benefit of being a fairly objective measure”); Arewa, Morris & Henderson, supra note 7, at 1011 (referring to “objective criteria such as citation counts and the Social Science Research Network (SSRN) downloads” for peer review  of faculty scholarship, although acknowledging these “are not perfect measures either”)."

This is from the St. Thomas article. In a citation count, these citations will be as important as ones noting works the authors actually grappled with. But what are we to make of these citations. Did they influence the St. Thomas authors? I doubt it. Instead it looks like an appeal to authority without any real examination of whether the "authority" is "authoritative." At also represents, unfortunately, a norm in legal scholarship and, also why it gets so little attention outside a small world. What it means is "you should believe me because I found someone who agrees with me."But who knows if they know what they were talking about?  It is, at best, a substitute for real research.  Indeed, when my coauthor and I tracked down some citations, those cited were citing a third party who also cited someone farther down the line. That amounted to a hearsay appeal to authority.

The Mashburn/Harrison work has received a fair amount of attention and, surprisingly, most of it has been favorable. (In their hearts law professors know.)  In that article we challenged defenders of counting to redo our study. The St. Thomas group did not do this opting instead to decide it cannot be right because -- well, just because. Moreover,  no one challenging our work has opted to prove we are wrong by selecting an article and indicating how each cited work was influential or did not fall in the hearsay category.

The bad news is that that the authors may be right. Citation may equal impact. If so, based on the research methods widely used by law professors, we are in even bigger trouble.

And, finally, and this time I mean it. Suppose citations do equal scholarly impact. One law professor influences another and so on. There is hardly anything useful going on unless that impact is felt somewhere outside their closed group. Unless you assess that, you have no meaningful measure of anything.

Wednesday, August 19, 2015

What Do I Tell My Friend?

A friend of mine just took job as the head of a corporation which is owned by another corporation. The parent corporation knew its subsidiary needed help and my friend understood this. No one mislead anyone.

But when my friend arrived at the new job here is what he found:

1. Employees were responsible for the product lines to be offered and they always selected the products they personally liked  whether or not there was a market for them.

2. Some employees were paid for full time work but were, in fact, only working part time.

3. Some were paid who actually were never at work.

4. Prior management knew all this but did not address it for fear of a revolt among employees.

5. Instead the prior management was weak and made secret side payments to employees for "extra work" as defined by the employee.

6, If  prior management made any effort to address issues, the employees closed ranks and defended each other.

7. When employees feared one of there product lines might be examined they formed into bands designed to spread disinformation about management and its motives.

8. The subsidiary was consistently out of compliance with some of the basic rules of the parent corporation.

9. The corporation kept no written records.

9. It was impossible to fire anyone regardless of their lack of production.

What can he do?

Monday, August 17, 2015

How Not to Lose Your Tenured Position

[Rerun] Over on Faculty Lounge and to some extent on the Caron Tax Blog a discussion has broken out about what to do with all the old (and privileged)  folks in law teaching who are clogging up the system so all the young (and privileged) folks who want to be law teachers have a hard time getting those plum jobs. My view? I'm just happy to observe the intraclass warfare.  As the John Revolting character said to the Christian Slater character in "Broken Arrow" when he set off a small nuclear device, "Ain't it Cool."

In the course of the discussion (I'd link to it but have no idea how to link to a comment) someone made a comment like "Young or old, let's have standards and if people cannot keep up, out they go."  Orin Kerr then writes: "Can you say more about how/if you would do that in a way consistent with tenure protections? Would you say that failure to reach a minimum productivity level is "good cause" to fire a tenured professor, and if so, what kind of standard would you propose? Or would you end tenure protections first?"

 And, right there we have the issue, don't we. I like the ways he phrases it -- "good cause." I think of it as "for cause."  When we think about tenure the running joke is to lose it you have to do something really awful -- kill someone, rob a bank, sexually harass a student,  Simply punching a student is not enough. In fact that gets you a paid vacation. Being a God-awful teacher is not enough; stealing from the school is not enough. In fact, and here is my punch line:

There is nothing connected to actually doing the job or not doing it that can be a basis for dismissal.

Think of this in the context of other jobs. If you delivered mail, you could be fired for punching the boss but never ever for putting the wrong mail in the wrong box. If you were a meat inspector, you could get fired for shop-lifting from WalMart but never ever for simple stamping as OK, rotten meat that then makes 1000 people sick. A physician could be canned for getting drunk and smashing his Porsche into a pedestrian but not for performing an accidental lobotomy.

I'm thinking. What kind of job security is it that says the main thing you can never be fired for is screwing up on the job you are being paid to do? 

Wednesday, August 05, 2015

Laughter and Respect

I am experiencing a relatively rileless summer. Some have noted that this is correlated with the departure of side-deal dean. I tell them correlation is not causation but, on the other hand, it does not rule out causation.

But this is not really what this post is about. It is about two different things. Yes, two for one and if you call now, you will get a pocket sized fly fishing outfit. Operators are standing by.

Laughter Lately I have been thinking a lot about laughter. It's that odd hacking sound people make when they find something funny. Surely evolutionary biologists have looked into this. What is the  function of genuine laughter? Why didn't we evolve to click our teeth  or flex our toes.

An aspect that is particularly interesting is laughter inflation.  It has devalued real laughter. I am talking about social laughter which is not an involuntary reaction at all.  It's a way to recognize others, make yourself seem impressed or non threatening, or just a way of saying I am jovial, fun loving, or what ever. It is a form of signaling. So, instead of saying "Hi, I am Phil, this what I want you to think I am like," you act it out.

I notice this at faculty meetings where there is a fair amount of nervous laughter -- "yes Phil, I am getting ready to say in coded terms that you are a idiot but I am just a nice guy." See, hear me laugh --ha, snort, ha, ha!

I was on a ship the other day and seating near a group of 10 guys just getting to know each other. It was either a laugh riot or a laugh off. Everyone seemed to signal to others that they were good guys by laughing at whatever anyone said that was within a mile of being funny. I think there must have been prizes for first laugh, loudest laugh, longest laugh, and last laugh.

After several days of meals and several hours of hanging out with each other, I noticed the laughter was infrequent. Did they all use up all their jokes? Naw, after about 40 hours of eating together the personality "marking" was over. Each had done what he could to communicate why he was likable and the other had bought it or not. The "laughter" had served its purpose

I also see it with my wife. It's  never in response to anything I say but in response to someone she does not know that well. They can say,  "Do you want a nickel, go get a pickle" and the deep laugh comes on and goes on forever or maybe it just seems forever.

Now laughter is spoiled for me. Am I laughing because it is funny or is it signaling? Are you laughing because I am funny or because you are telling me something about yourself. If it is the latter, please keep it to your self or email it.

Respect: A little over two years ago I wrote a post in which I said I respect people in inverse relationship to their power, status, and income. Yes, there can be false positives and false negatives but it's a really good starting point.

One of the things that can help you spot false positives and negatives is the way people treat secretaries. This is not fool proof. Some will be sweet as can be because that is the way to get what they want. On the other hand, those who treat secretaries badly drop off my chart of people who deserve even an atom of respect. In fact, the first level of a**holes on your faculty are those who treat the secretaries in a disrespectful manner. You might be surprised about who is "outed" by the secretary treatment check list.