Posts Tagged ‘psychology’

testing, testing, one, two, three

April 26, 2011 2 comments

Testing.  It’s become so much part of the life of a learner or a teacher, at any age.  And it’s a fascinating topic.

Okay, I’m one of those weird people who thinks of test-taking myself as a sort of competitive athletic event, one at which I’m really quite good even while thinking that the vast majority of tests I’ve ever taken were nearly completely pointless.  No, that’s not my impostor syndrome kicking in.  It has to do with a central concept in test design, which I’ll explain below.

What I love most about assessment is how useful it can be when done well.  One of my colleagues says that testing doesn’t bring out the best in people, it doesn’t bring out the worst in people, but it brings out the most in people.  We put you in a situation where your normal compensatory strategies for getting along in the world aren’t going to work.  As Peter Ossorio says, when you ask a person to do something they can’t do, they’ll do something they can do.  You’ll figure out something to do, the best you can, and what you do will be a reflection, in some way, of you.  It’s like science — each test is an experiment that you and I do together.  No one bit of data proves anything by itself, but when we put things together and look for themes, consistencies, divergences, a story begins to emerge, and it often does so surprisingly quickly.

But what bugs me is how little most folks understand about tests of all stripes — most importantly, how they’re built, how they work, what they’re good for… and what they aren’t.  So what I’d like to do is to kick off a random-access series of posts on various aspects of assessment, including ordinary classroom tests, high-stakes testing for No Child Left Behind Allowed Ahead (also known as No Teacher Left Standing) and other similar “accountability” movements, bubble tests like the dreaded SAT and its ilk, and, of course, my favorite, the one-on-one kinds of tests used for special education and other diagnostic work, the kind that seriously geeky people like me give.  Those include cognitive tests, neuropsychological tests, academic tests, psychological tests, behavioral questionnaires, and other fun stuff. I’ll start there because, well, because I like them and I think they’re really pretty interesting.  I’ll try to chew off manageable chunks to talk about, and over time, I hope people learn something.

The most serious and popular misconception I encounter is a fundamental misunderstanding of what tests can do.  They’re not magic, and neither are those of us who give them magicians.  We’re just very observant (or at least we’re supposed to be!), and we’re using them to make a series of structured observations.

Again, this is like science.  When I was training as a molecular biologist, one of the things I had thwacked into my head (through reading in the literature some of the truly impressively weird things that happened when people didn’t remember it) was that no experiment ever tells you anything about the real world.  It tells you what happened on that day when that person did that experiment in that way.  You might use that information to conjecture about the nature of the real world based on your data, and over time, as you build up more data, you can get a better and better sense of what the real world might be like.  But you might see a different experiment, claiming to answer the same question, where you get different results.  Uh, oh.  Where do you look, to figure out what was going on to find the difference that made the difference?  In the Materials and Methods, the specifics of how the experiment was designed and constructed.  Very often, that’s where the difference lies.  You cannot separate data from the experiment that generated it.

Same with assessment.   No test, no matter how beautifully it’s designed, how skillfully it’s administered, and how insightfully it’s interpreted, can possibly tell you anything incontrovertibly true about the real human being.  The test tells you what that person did on that day on that test with that tester in that environment.  It might reflect something probably true about the person, but you have to stay humble with your interpretation.

Since you will always value what you measure, it makes sense to think very carefully about how to measure what you actually value. In education, we talk about the idea of “alignment” — we’d say that this test is or is not well-aligned to the skills we want the student to be able to demonstrate.  That’s what I was talking about above, why I don’t respect the very bubble tests that I tend to be able to blow out of the water.  They typically test what is easy to measure, but not what a thoughtful professional would consider all that valuable.  At the conclusion of many thousands of hours of clinical training, psychologists in most states have to take a detailed fact-recall bubble test covering basically the entire field.  We to prove that we know which classic theorist suggested that you were running from the bear because you were afraid, versus which one suggested that you were afraid because you were running from the bear.  But we don’t have to demonstrate the capacity to actually manifest any clinical competencies with actual, oh, I dunno, human beings in distress.  In test design, we talk about the very-closely-related concept of “validity,” which comes in many flavors.  In this case, the construct validity of the test — how it defines what it is that it’s trying to measure — is awful.  Fact knowledge within a domain is a useful thing, and might be a good prerequisite to beginning clinical work.  But the public is not protected from incompetent psychologists by choosing only those who can remember the facts printed in their textbooks.

I think the best-aligned test I ever took was the qualifying exam for the Ph.D. I didn’t get in cancer biology.  I was required to dive in to fields I was unfamiliar with, learn about the prior research in those fields, and propose new lines of research that would answer important unanswered questions.  Minus the speed with which I had to do it (three of these, in completely different fields, within a single week!), this test was testing very much what I would need to do if I became a principal investigator running my own lab someday.   Of course, the alignment/construct validity of that test wasn’t perfect either.  What it didn’t explore was the personality traits which set me up to be a very sad and bored and frustrated person in the lab, the precise difference between thinking about science, which I love and am good at, and doing bench science on a day-to-day basis, which I don’t and am not.

What I find most concerning about the high-stakes testing (aka “accountability”) movement in education is that it tends to use tests with poor validity in a variety of domains (construct validity, content validity, and predictive validity being the most notable), and that it tends to ignore other underlying methodological differences between comparison groups (most notably, differences in the populations being served and the resources available to teachers and administrators to serve them, but also differences in how various jurisdictions define their goals and standards).  When science teachers teach kids about experimental controls, we start with the idea of a “fair game.”  But there’s no way on earth that these “games” are fair.  There’s nothing truly “standardized” about these experiments, and almost every interpretation that is made of them is a massive overinterpretation from inadequate data.  Gives serious testing a bad name.  Harrumph.

Okay, so my plans for this series of posts right now involve topics like the various types of validity and reliability (the twin pillars of assessment for people who actually want usable data!), and a sort of overview of each of the major types of clinical testing (e.g., cognitive, academic, neuropsychological, behavioral, projective) and what they are and aren’t good for.  I’ll do classroom and educational and high-stakes stuff later, but I’d rather start with what I do the most of.  If there are specific ideas or questions you’d like me to address, feel free to drop them in the comments area here.

I’m not usually political on this blog, but… (thoughts on mental illness and culpability of public figures)

January 9, 2011 12 comments

Oh, look, here we go again.

Public figures, folks in positions of leadership and authority, present their violent fantasies, talking about their political enemies as evil, dangerous, deserving of death, etc.  They often suggest methods that would be way-cool, too, and talk about how great it would be to “take decisive action” or somesuch.  Over and over again.  In loud voices, using all of their charisma.

(I will refrain from calling out specific speakers and specific incidents here, because it is wrong no matter who does it, and because I do not want this blog to degenerate into a pointless debate about the minutiae of precisely who said precisely what and whether that counts.  The specific tragedy that occurred today is just the latest instantiation.)

Lots of folks say, “Oh, that’s awful!  Don’t go around inciting violence!”

The speakers respond, “Oh, come on, can’t you take a joke?  I was only speaking figuratively!  That’s just political rhetoric!  I’m not really telling people to go out and do those things!  And besides, other people are doing it, too!  And I’m not really an authority figure anyhow, because, after all, I’m just an entertainer, or a candidate, or a humble religious leader, or a Citizen Just Like You.”  Right, because we all happen to have hundreds, thousands, or millions of people listening to us on the TV/radio/internet/lectern/pulpit.

Then someone does something awful: blows up a building, shoots a bunch of people, or otherwise takes violent action that looks rather like what that authority figure fantasized about.

And everyone is shocked! shocked, I say!  Everyone, especially including the public figures themselves, decries the violence and does their best to distance themselves from the person who actually did the bad thing.  The good news is that when we learn more about that person, we find out that they were a “crazy” “lone wolf.”  They’ve often posted rambling and incoherent monologues on the internet or left other clear evidence of serious and persistent mental illness.  Everyone titters and points and does their level best to say, “That person is nothing like me,” because thought disorders are scary. (I’m not being sarcastic.  The idea that your own brain might turn on you is legitimately terrifying.  But when we’re scared of something, one of our normal and natural defenses is to try to make it be as separate from us ourselves as possible.)

So hooray!  The public figures are off the hook!  They couldn’t possibly have predicted that some whack-job would have taken them seriously and done that awful thing.  Those people aren’t like us.  They can’t be held responsible for what those not-like-us-people might do, even if it was disturbingly like what they were talking about on the TV/radio/internet/lectern/pulpit.

Folks, the population incidence of schizophrenia is approximately 1%.  One. per. cent. Think about that for a minute.  Think about going into a movie theatre… or a house of worship… or a football game…  Now think about the population of the country (or the world).  Move the decimal place two spots to the left.  That’s a lot of people struggling with serious and persistent mental illness, typically in overburdened systems that rarely manage to provide the kind of help they need.  I’ve worked with folks who are seriously affected by these disorders — I have a great deal of compassion for them.

Let me be crystal clear — not all people with thought disorders are violent.  In fact, the research data is quite clear that the overwhelming majority are not.  (In fact, they are no more likely to be violent than the general public, although most people massively underestimate how violent the general public is).  Even those few who are dangerous are rarely dangerous to people they don’t know.  (That’s true of the general population, too, by the way — the overwhelming majority of victims of violence know their attackers well… they’re often closely related to them.)

But “knowing” someone in this case can include being introduced to them by, say, a public authority figure who talks about them a whole lot and tells you that they know all about this person who is evil and they know that this person should be killed and they create a concrete image of how that could be done and they repeat the message over and over again or have lots of friends who repeat similar messages.  Especially if part of your thought disorder includes the relatively common symptom of believing that the TV/radio/internet/movie/music has a special message just for you.  When you’re having a hard time holding onto reality and making it make sense, then those nice, simple, consistent messages getting repeated over and over can feel comforting.

While I’m on the subject of relatively-common-symptoms of thought disorders, let me also point out that the belief that you are somehow important, special, have a special mission to carry out, have to sacrifice yourself, have to save the world, etc… is also on the list.  Ahem.

Folks like to hide behind the “abstract language” thing.  I can’t agree.  You may speak abstractly, sure, but you’re speaking to lots of people in the population who are not abstract thinkers.  (Think about how frustrating it can be at the DMV, or with the TSA, or on a telephone support line, or any of the other situations that provoke the typical Xtranormal video.)  And when someone has a thought disorder, they often become highly concrete and not-quite-logical in how they process language.  Words don’t quite mean what they usually mean, sounds start meaning more than the words, sentences can start in one place and end someplace very different, language and logic can start to feel like one of those water-snake toys that keeps slipping out of your hands.  Understanding the niceties of figurative language and hyperbole and rhetorical flourish from the public authority figure on the TV/radio/internet/lectern/pulpit, figuring out what they really mean…?  Go ahead and look up the writings of any of these “lone crazies” and tell me if you think a person who has that little control over language and thought can tell when an authority figure’s comments about the nobility of sacrifice and the necessity of violence and all that are really just entertaining and clever words, and when they’re concrete calls to real-world action.  I like to think I’m a good consumer of the subtleties of language, and I am often unsure of what the shouting heads really mean.

The saving grace is that most of us, most of the time, have an observing ego.  We think about what we’re going to do at least a little bit before we do it, and we judge our planned actions in light of whether they conform to the usual rules of the culture, what the likely outcomes would be, and so forth.  And most of us, most of the time, have enough executive functioning to inhibit the acting out of the planned actions we judge to be poor choices.  When I hear about something despicable, when I’m angry or scared, I might think, “Gee, I wish I could just… <fill in some random violent fantasy>.”  But I don’t then actually do it… at least not most of the time (grin).  But serious mental illness can wreak havoc with those self-controls.

What really concerns me is when the same authority figures who claim that they couldn’t possibly have known that they were sooo powerful… seem to feed their own sense of power by watching other people (who have less in the way of observing ego and executive function) carry out their own violent fantasies.  I have known a few individuals who seemed to thrive on the chaos they caused within a community.  Metaphorically, they would throw bricks high up in the air.  When the “brick” came down and hurt someone or caused some other form of contention, they’d be as shocked as everyone else — perhaps more so.  But there was also the sly smile, the subtle recognition of their own power to have caused that. Heh.  I wonder if some of these folks who seem so often to step over the lines of appropriate authority-figure behavior are being repeatedly reinforced by how much crisis, both actual and feared, that they cause.  That’s a problem, because even telling them how powerful they are and reminding them that with power comes responsibility feeds the narcissism.

I’m not sure how we as a civil community can address that effectively, not when the ranting is what makes money.  To a great extent, I think we all have to get serious with ourselves about how our own fantasies are being fed by the violent talk.  Personally, I’ve noticed a few shows that I enjoy and typically agree with politically, but I start to feel that I am getting too much pleasure out of the implied combat.  When I stop liking who I become when I listen to them, I vote with my ears and whatever ratings statistics I might happen to be contributing to.  I have stopped listening to them.

So all this is why I can’t accept that “no one could have predicted.”  No one could have predicted precisely which person would react to precisely which turn of violent-fantasy speech or imagery in precisely which way at precisely which time and kill precisely which people.  The population is too large to be keeping tabs on every person to the level that would permit experts to make such predictions, and I personally would not want to live in a country that kept tabs on its citizens in such a fashion.  But the idea that someone would react in some violent way at some point was extremely predictable.  And in fact, in most of these cases, as in the tragedy this morning, the prediction was in fact made and ignored, made and pooh-poohed, made and shouted down.  And in most of these cases, there was that little “heh” coming from the background as those who agreed with the violent fantasy got the pleasure of having their fantasy gratified by someone else.

Those who have the attention of the public, on the TV, shouting on the radio, posting on the internet, ranting from the secular lectern or preaching from the religious pulpit, all have a responsibility.  The more people who listen to you, the more you tell them what to do, the more responsibility you have when they act on what you tell them.  Some of the blood spatters on you, too.


my dissertation is available!

September 20, 2010 5 comments

Okay, so it was already available, for those of you who looked in the ProQuest / UMI database.  But I’m making it easier for people to find, at the same lovely price:  free.  I want it to be read, especially by people who think that doing therapy with gifted clients is just like doing therapy with anyone else and therefore they wouldn’t bother paying for access to something so silly.

However, since many folk have asked if they could pay what they felt was a fair price for the work, and one good friend (you know who you are!) was therapist enough herself to confront me quite pointedly on the topic, I have put it up on my own website with a PayPal donate button.  If it brought value to your life, or if you would like to place a value on what you think it has brought to the world, if you’ve suddenly become heir to a vast fortune you’d like to share with me, or if you’d just like to buy me a cup of coffee, pay what you will, or pay nothing at all and enjoy it without guilt (okay, maybe with a little tiny bit of guilt (grin)).

I’ve also split out the chunk at the end with the provisional clinical guidelines for therapists who work with gifted clients as a separate file, for those who don’t feel like plowing through the whole thing.  I’ll probably post excerpts, or articles based upon chunks of the thing, on this blog.

Whether you’ve read it already but would like to donate, or whether you haven’t yet seen my magnificent octopus (apologies to Blackadder), come on over and take a look!

a nice post from a fellow GT blogger

September 12, 2010 6 comments

I just read this post:

and I think it’s really worth reading.  So often, I think, parents have had their own educational or social traumas, or they’ve heard countless stories about the problems and pain other people have had, that they come in assuming that all teachers are dumb, hostile, clueless, nasty, evil, you name it.  In the shrink biz, we’d call this a transference phenomenon, where your prior experience creates a distorted lens through which you view your current experience.  Quite naturally, you expect that what happened before is going to happen again.  But that often leads people to behave in ways that create the very problems they are afraid of happening.  Thus, the trauma becomes re-enacted, proving, of course, that you were right to expect that sort of thing to happen, and continuing the cycle.

Are a lot of teachers clueless about giftedness?  You bet.  Are some of them hostile?  Yes.  But they’re not the norm.  The overwhelming majority of educators do it because they like kids and because they like teaching.  (After all, the pay is pretty crappy and the working conditions are terrible.)  They don’t get up in the morning thinking, “Okay, how do I ruin little Freddie’s life today?”  Build the relationship and see if you can find a different ending to the story.

Clues on finding a therapist for a gifted client

August 20, 2010 71 comments

This is another topic I get asked about a lot… “My kid is really struggling to adjust, to make sense of himself in a world that doesn’t quite understand him. I think therapy would help, but how do I find someone who really “gets” the whole gifted thing?”

This is a huge problem.   A recent publication from my esteemed mentor, Jean Peterson, showed that most accredited counselor training programs offer zero training in understanding giftedness, and those that do have only very tiny amounts of time spent on the topic.  And I worry, frankly, about what is actually being taught when anything’s being taught… whether they are just continuing to promulgate the same typical myths about how gifted kids “will be just fine on their own.”  There aren’t any studies yet looking at training programs for psychologists, but my sense of it is that, if anything, the situation is worse, since “gifted education” exists as a distinct field, while “gifted psychology” kinda basically doesn’t.  Basically, giftedness is not considered a relevant dimension of human difference or cultural experience.  Gifted folks are just like everyone else except that we got As in school and we’re all just fine.  Aren’t we?  <cough, cough…>

And it’s not just a problem for kids, either.  Gifted adults also struggle to make sense of ourselves in a world that often isn’t even remotely a good fit.  Giftedness isn’t just a school-bound phenomenon, and it doesn’t expire upon graduation — we are who we are across domains and throughout the lifespan.

I wish I had some easy answers.  My dissertation research, on the therapeutic working alliance between gifted clients and their therapists, was useful in that it showed where some of the major pitfalls were.  But it didn’t help with the basic problem that most therapists think they do get it (an example of the Dunning-Kruger effect, where the lack of knowledge and metacognitive skills in a domain interferes with accurate self-assessment of skill in a domain — basically, if you don’t know what you don’t know, you don’t know that you don’t know it).  So it’s like in education: if you ask someone if they understand giftedness, they’ll all tell you they do, but that doesn’t mean a whole heck of a lot.

Plus, a lot of providers seem to think that “gifted” means “fiscally gifted,” and they see us as high-functioning clients who are likely to provide a nice revenue stream (as Tom Lehrer said, they think they’re specializing in “diseases of the rich“), so they advertise claiming this as a specialty area.  We know that gifted folk are found in every social, ethnic, racial, and economic group, but, well, like I said, a lot of folks don’t know what they don’t know.

So what we need is a way to interview prospective providers to find out what they actually know, rather than what they’ll tell you they know.

I know, we’re all afraid to talk about it (I should really put up a rant about gifties as a closeted minority), but we have to.  If we don’t, they won’t either — remember, they think it’s not relevant.

So, I would advise specifically bringing up the topic of giftedness and multiple exceptionality with any prospective therapist in a nonthreatening but clear way, one which focuses on observable behavior — “What experience do you have working with folks who are highly intelligent?  What do you see as the major risk and resilience factors in this population?   Have you found it necessary to adapt your approach in working with gifted folks?  If so, how, and what is your rationale for that?”

Pay attention not just to the content of the responses, but also to the nonverbal signals and your gut feeling about how they’re responding to the notion. If you feel like you’re getting a dismissive or hostile reaction, go somewhere else and don’t feel bad about it.  Really.  You’re not crazy (okay, you might be, but I’m not going to diagnose you via a blog).  You might be a bit hypersensitive, sure (I see that a lot, especially from adult GT folks who had educationally or socially traumatic experiences in childhood), but in the shrink biz, they teach us to pay attention to those feelings.

Ideally, I would want to hear someone spontaneously identify issues of social isolation, intense imaginations and emotions, and asynchrony between cognitive and emotional development, as all being relevant things to think about when working with gifted folks.  Look up one of the many lists of myths and realities about giftedness (try this one, or this one, or this one), and if they start spouting any of these, try disagreeing gently.  If you get push-back instead of thoughtful dialogue, just thank them nicely and walk.

Second choice would be someone who can at least spontaneously admit that they are not knowledgeable in the area but would be interested in self-educating.  If they want to self-educate, the (btw, I would recommend the Models of Counseling Gifted Children, Adolescents, and Young Adults book, edited by Jean Peterson and Sal Medaglio, for a professional to self-educate on the topic).  There are some very good materials available on the SENG (Supporting the Emotional Needs of the Gifted) website, too.  Or they could call me for a short-term professional consultation — I’m thrilled to be able to spread the knowledge here.

Anyone who frames giftedness as being part of the problem, anyone who defines the intensity and drive and perceptiveness and differentness and postformal reasoning as “the thing that’s wrong with you,” leave and don’t look back.  The goal is not to get our kids (or us!) to act like everyone else.  The goal is to help us figure out who we are and how to act like ourselves, just in an adaptive way.

I hate to have to say it, but I really do think that therapists who work with smart folks probably should be at least reasonably smart themselves, if for no other reason than so they can follow the logic and metaphor, quick thinking, intuitive leaps, and so on.   Also, I would want someone who in your initial interview seems to have some level of personal empathy with those experiences — they don’t necessarily have to have had them themselves, but it helps if they’re close enough that they didn’t experience their own intelligence as always a positive thing.  Both of these showed up in my research as a very common theme in terms of what distinguished successful from unsuccessful alliances.  Someone who is at least moderately bright and who is a good listener is okay for a client who is looking mainly for symptom relief, but for a client who needs a long-term mentor relationship, the respondents in my study were pretty clear on the topic… they need to be pretty smart.  When you’re trying to figure out who you are and how you’re going to exist in the world, you don’t want to have to wait around for the therapist to catch up, or to feel like you’re doing all the work yourself.

Note, however, that the therapist who happens to be gifted needs to have done their own work and come to some level of acceptance and understanding of their own intelligence. If they’ve got unmetabolized “stuff,” it’s going to play out in the relationship.  Some of the nastiest and most invalidating responses I’ve personally experienced or heard about in my research came from professionals who were themselves very likely to be highly intelligent. I’m not saying “contrary,” I’m saying “downright nasty.” (in the biz, we’d call those countertransference reactions).

Another reason why it helps to have someone reasonably smart so that the kid will have a harder time snowing them if they try.  The research on client honesty is pretty solid across the board — therapists tend not to know what clients hide from them.   I’ve heard consistent reports from GT clients that if they weren’t able to leave therapy (many child or adolescent clients don’t feel they have much choice in the matter), they were highly successful not just in stonewalling, but in outright snowing therapists who they felt didn’t empathize with them effectively (my favorite was the tweenaged client who structured her play so as to make her therapist come to certain interpretations).

In general, gifted folk don’t do as well in rigidly manualized treatments (which are, sadly, becoming extremely popular).  Cognitive-behavioral techniques can be really useful as tools, but a flexible and collaborative approach is going to be key. Don’t believe the press about “evidence-based treatments.” All forms of therapy have evidence to support them, and some forms are better than others for some clients for some kinds of situations — it’s all very individual. And what the main body of evidence shows is that, as James Carville might have said, it’s the relationship, stupid.

Before you ask, no, I don’t have a giant network of people I can recommend.  You can try asking me privately if I know anyone in your geographical area, and I’ll try, but no guarantees.