Posts Tagged ‘introduction’

testing, testing, one, two, three

April 26, 2011 2 comments

Testing.  It’s become so much part of the life of a learner or a teacher, at any age.  And it’s a fascinating topic.

Okay, I’m one of those weird people who thinks of test-taking myself as a sort of competitive athletic event, one at which I’m really quite good even while thinking that the vast majority of tests I’ve ever taken were nearly completely pointless.  No, that’s not my impostor syndrome kicking in.  It has to do with a central concept in test design, which I’ll explain below.

What I love most about assessment is how useful it can be when done well.  One of my colleagues says that testing doesn’t bring out the best in people, it doesn’t bring out the worst in people, but it brings out the most in people.  We put you in a situation where your normal compensatory strategies for getting along in the world aren’t going to work.  As Peter Ossorio says, when you ask a person to do something they can’t do, they’ll do something they can do.  You’ll figure out something to do, the best you can, and what you do will be a reflection, in some way, of you.  It’s like science — each test is an experiment that you and I do together.  No one bit of data proves anything by itself, but when we put things together and look for themes, consistencies, divergences, a story begins to emerge, and it often does so surprisingly quickly.

But what bugs me is how little most folks understand about tests of all stripes — most importantly, how they’re built, how they work, what they’re good for… and what they aren’t.  So what I’d like to do is to kick off a random-access series of posts on various aspects of assessment, including ordinary classroom tests, high-stakes testing for No Child Left Behind Allowed Ahead (also known as No Teacher Left Standing) and other similar “accountability” movements, bubble tests like the dreaded SAT and its ilk, and, of course, my favorite, the one-on-one kinds of tests used for special education and other diagnostic work, the kind that seriously geeky people like me give.  Those include cognitive tests, neuropsychological tests, academic tests, psychological tests, behavioral questionnaires, and other fun stuff. I’ll start there because, well, because I like them and I think they’re really pretty interesting.  I’ll try to chew off manageable chunks to talk about, and over time, I hope people learn something.

The most serious and popular misconception I encounter is a fundamental misunderstanding of what tests can do.  They’re not magic, and neither are those of us who give them magicians.  We’re just very observant (or at least we’re supposed to be!), and we’re using them to make a series of structured observations.

Again, this is like science.  When I was training as a molecular biologist, one of the things I had thwacked into my head (through reading in the literature some of the truly impressively weird things that happened when people didn’t remember it) was that no experiment ever tells you anything about the real world.  It tells you what happened on that day when that person did that experiment in that way.  You might use that information to conjecture about the nature of the real world based on your data, and over time, as you build up more data, you can get a better and better sense of what the real world might be like.  But you might see a different experiment, claiming to answer the same question, where you get different results.  Uh, oh.  Where do you look, to figure out what was going on to find the difference that made the difference?  In the Materials and Methods, the specifics of how the experiment was designed and constructed.  Very often, that’s where the difference lies.  You cannot separate data from the experiment that generated it.

Same with assessment.   No test, no matter how beautifully it’s designed, how skillfully it’s administered, and how insightfully it’s interpreted, can possibly tell you anything incontrovertibly true about the real human being.  The test tells you what that person did on that day on that test with that tester in that environment.  It might reflect something probably true about the person, but you have to stay humble with your interpretation.

Since you will always value what you measure, it makes sense to think very carefully about how to measure what you actually value. In education, we talk about the idea of “alignment” — we’d say that this test is or is not well-aligned to the skills we want the student to be able to demonstrate.  That’s what I was talking about above, why I don’t respect the very bubble tests that I tend to be able to blow out of the water.  They typically test what is easy to measure, but not what a thoughtful professional would consider all that valuable.  At the conclusion of many thousands of hours of clinical training, psychologists in most states have to take a detailed fact-recall bubble test covering basically the entire field.  We to prove that we know which classic theorist suggested that you were running from the bear because you were afraid, versus which one suggested that you were afraid because you were running from the bear.  But we don’t have to demonstrate the capacity to actually manifest any clinical competencies with actual, oh, I dunno, human beings in distress.  In test design, we talk about the very-closely-related concept of “validity,” which comes in many flavors.  In this case, the construct validity of the test — how it defines what it is that it’s trying to measure — is awful.  Fact knowledge within a domain is a useful thing, and might be a good prerequisite to beginning clinical work.  But the public is not protected from incompetent psychologists by choosing only those who can remember the facts printed in their textbooks.

I think the best-aligned test I ever took was the qualifying exam for the Ph.D. I didn’t get in cancer biology.  I was required to dive in to fields I was unfamiliar with, learn about the prior research in those fields, and propose new lines of research that would answer important unanswered questions.  Minus the speed with which I had to do it (three of these, in completely different fields, within a single week!), this test was testing very much what I would need to do if I became a principal investigator running my own lab someday.   Of course, the alignment/construct validity of that test wasn’t perfect either.  What it didn’t explore was the personality traits which set me up to be a very sad and bored and frustrated person in the lab, the precise difference between thinking about science, which I love and am good at, and doing bench science on a day-to-day basis, which I don’t and am not.

What I find most concerning about the high-stakes testing (aka “accountability”) movement in education is that it tends to use tests with poor validity in a variety of domains (construct validity, content validity, and predictive validity being the most notable), and that it tends to ignore other underlying methodological differences between comparison groups (most notably, differences in the populations being served and the resources available to teachers and administrators to serve them, but also differences in how various jurisdictions define their goals and standards).  When science teachers teach kids about experimental controls, we start with the idea of a “fair game.”  But there’s no way on earth that these “games” are fair.  There’s nothing truly “standardized” about these experiments, and almost every interpretation that is made of them is a massive overinterpretation from inadequate data.  Gives serious testing a bad name.  Harrumph.

Okay, so my plans for this series of posts right now involve topics like the various types of validity and reliability (the twin pillars of assessment for people who actually want usable data!), and a sort of overview of each of the major types of clinical testing (e.g., cognitive, academic, neuropsychological, behavioral, projective) and what they are and aren’t good for.  I’ll do classroom and educational and high-stakes stuff later, but I’d rather start with what I do the most of.  If there are specific ideas or questions you’d like me to address, feel free to drop them in the comments area here.


Oh, no, my kid might be gifted! Where do I start?

August 3, 2010 23 comments

This is perhaps the most common reason people call me.  Maybe they always assumed that their kids would be gifted, but had also assumed that there would be clear and easy guidance and help available from the school system.  Or maybe giftedness just wasn’t on their radar, but a pediatrician or teacher or relative or someone else said something.  Either way, they started web-searching, and quickly found themselves confused and overwhelmed — there is so much information, so much jargon, all the information is contradictory, and so much of it is, well, highly opinionated.

[Okay, so I’m probably no exception to the “highly opinionated” thing, but I try not to go off the, “Your kid is a super-special superior being,” or the, “If you don’t do this One Right Thing and you don’t do it Right Now, you’re not taking proper care of your child,” deep ends.  If I do, please don’t hesitate to call me on it.]

But let me give you a few simple things to do.

First, breathe.  Your kid is wonderful and terrific and you are feeling the same awesome responsibility as the custodian of a young life as all good parents do.  But, as I tell my own kids, unless something is bleeding or on fire, chances are good that this isn’t an emergency.  I do get some emergency-type calls (school is in the process of throwing a kid out, kid is emotionally falling apart on a regular basis, etc.), and if that’s the kind of situation you’re in, absolutely send me a note or give me a call and I’ll talk you through.

But otherwise, relax.  Your kid is the same kid they were yesterday and is the same kid they’re going to be tomorrow.  A day or a week or a month here or there isn’t going to make the difference between a fulfilling life of scholarship, career, and love, and a kid who crashes and burns and ends up on Skid Row.  It’s okay to take the time to get your bearings.  And few decisions are truly permanent — you can usually change course later.

Okay.  There are a lot of books written about gifted kids and how to parent them.  I frankly don’t think that parents (who are probably pretty smart themselves!) need to read every book out there — there’s not going to be that much new information once you’ve read one.  My current go-to book — it’s comprehensive, realistic, calm, not worshipful or overblown or dismissive — is Webb, J.T., Gore, J.L., Amend, E.R., & DeVries, A.R. (2007).  A parent’s guide to gifted children.  Scottsdale, AZ: Great Potential Press.

If you’re looking for some specific suggestions for curricula, toys, books, contests, local groups, you name it, go to Hoagies.  Carolyn K curates probably the largest single pile of links to All Things Gifted.  You name it, it’s probably already there.  If it’s not, tell Carolyn, and she’ll add it.  It’s not the easiest site to browse through, just because there’s so much of it — use the search tool liberally.  Hoagies also has a wealth of articles with every possible opinion on every possible topic, but chances are good that’s what got you overwhelmed in the first place (grin).

One very strong theme in the research literature on giftedness is the idea of social isolation, both for kids and for parents.  Yeah, you.  Giftedness is treated with a lot of ambivalence in the USA and similar cultures — you’re supposed to have brilliant kids, but whatever you do, Don’t Talk About Them.  The single best thing parents can do for themselves is to break the isolation.  If you go to Hoagies (of course), there is a list of mailing lists and ways to get in contact with other parents:  What I usually recommend is to subscribe to GT-Families and TAGFAM, two general-interest mailing lists.  Volume can be high at times — try using your mail program to sort messages into a folder, and if you get behind by a week or more, don’t feel bad about deleting.  There are also related lists for parents of kids who are twice-exceptional, homeschooling, radically accelerated, etc.  The lists are *very* helpful for parents to get advice and feedback from each other, and even just to have a place to talk about what they’re going through without being assumed to be bragging.

The GT universe is also moving into social media — many of the major players have Facebook pages (you can find mine here), blogs, Twitter accounts, etc.  Hoagies again is a great one to follow, and allow yourself to bop around and enjoy the variety of viewpoints.  I don’t have favorites yet in this realm — it’s changing too fast.

Another great organization is Supporting the Emotional Needs of the Gifted:  Besides having some very good articles on the social-emotional experience of giftedness, they train facilitators to run local parent support groups.  Very helpful for breaking the isolation and processing one’s own experiences of growing up gifted.  Hint: the kid didn’t just fall out of the sky that way…

There is a National Association of Gifted Children, and every state has an association.  Associations vary widely in terms of what they actually provide — my experience has been that they are more focused around political advocacy than around direct service to parents or kids.  You can find a list of the state associations on that same Hoagies page.

Many parents who call me are looking to find out about what their school districts are obligated to do.  You can find a complete listing of state laws affecting gifted kids at Genius Denied.  The title might lead you to think that it’s a pretty depressing listing, and you’d be right.  Sigh.

Along with that, by the way, if you think your child might be twice-exceptional, or if you live in a state where gifted education is handled through the special education process, the place to go is Wrightslaw.

There are many state and regional talent searches; find the ones serving your location on the Hoagies Talent Search page. All offer low-cost methods for kids to take out-of-level achievement tests (because they’d hit the ceilings of grade-level achievement tests, of course!), qualifying them to take courses designed for gifted kids.  Courses can be expensive but financial aid is available.  The courses are generally terrific, and the chance to be with other gifted kids is one that many kids treasure.  If nothing else, having a qualifying score on these is a cheap-and-easy way to get something objective in hand that you can share with your child’s district when they say, “Oh, we have lots of kids just like that.”

To learn about what the research actually says about the various different forms of academic acceleration, you can download the A Nation Deceived and A Nation Empowered reports for free.  They’re from three of the leading researchers in gifted education, and, as the title might suggest, most of what educators think they know just isn’t so.  They typically get zero training in gifted issues while in teacher school, and what they do get typically just perpetuates the myths.  And don’t worry about that “social-emotional” thing, or the one story they can tell you about the kid who accelerated and was miserable — the most important social-emotional need for gifted kids is appropriate academic challenge and real peers.

The best single book to share with teachers is: Winebrenner, S., & Brulles, D. (2012). Teaching gifted kids in today’s classroom: Strategies and techniques every teacher can use (3rd ed.). Minneapolis, MN: Free Spirit Publishing.  She talks about how to actually implement techniques like curriculum compacting (test out of what you already know), tiered or differentiated lesson plans (have different kids do different things on the same topic), and independent projects, without spending lots of money or putting forth enormous effort.  There’s even a companion CD with customizable forms for the teachers to use.

Because we need to break the social isolation of the kids, too, I also strongly recommend sharing with administrators if they’ll let you:  Winebrenner, S., & Brulles, D. (2008).  The cluster grouping handbook: A schoolwide model: How to challenge gifted students and improve achievement for all. Minneapolis, MN: Free Spirit Publishing.  In cluster grouping, instead of spreading the gifted kids out among all classes in a grade (which seems “fair” to the teachers, but makes it very hard for the gifted kids to find each other), they get put in groups of about 5-6 kids within a few classrooms.  This costs nothing to implement, and not only does the book tell a principal how and why, it even includes sample letters to help manage the political stuff around it.

If you are considering a whole-grade acceleration (grade skip), check out the Iowa Acceleration Scales, 3rd edition.  It’s a research-based measure designed to help parents and administrators talk in a holistic fashion about a kid when making placement decisions.  If you’re talking with administrators about district-wide acceleration policy (hey, a girl can dream, eh?) the Institute for Research and Policy on Acceleration has guidelines.  Both of these are from the same folks as Nation Deceived.

If a kid has IQ and achievement scores both above 145, there are some great opportunities available through the Davidson Institute for Talent Development.  Consultation, financial support, classes for kids and parents, all sorts of nice stuff.  Even if the child is not eligible (I always have to remind people who come to me for testing that by the math of the normal curve, 96% of kids above 130 are *not* above 145), DITD has some public-access bulletin boards with useful information and discussions as well, and a database of articles on specific topics.

That should get you started.  If you need more help, ask me, ask on the mailing lists, ask on the bboards, ask wherever, and you’ll get lots of help.  Typically, folks introduce their questions with, “I don’t know if anyone has ever had this happen, but…” and everyone says, “Oh, no, we’ve all had that happen, here’s the different ways we dealt with it.”  Sure, you’ll probably end up with way too many ideas, and you’ll hear all sorts of, er, very strong opinions.  Remember that I’m here giving you permission to relax, take a breath, work with the school, don’t give up, accept that you don’t have to find a perfect solution right away, and that there’s no one right way to do anything.

This post is part of the Hoagies’ Gifted Blog Hop on Gifted 101.

gifted 101 button

Categories: getting started Tags:

Hello, world!

July 25, 2010 13 comments

Well, now that I’ve finally graduated and don’t have papers and a dissertation to write, I’m ready to start actually writing more for the general public.  I tend to have opinions on giftedness, learning disabilities, education, psychology, and the interface between all of those.  I have particular expertise in cognitive, academic, and psychological assessment, the psychological and social effects of high intelligence, theories of intelligence and personality, and the real world of various educational strategies.  I believe that most questions are far more complex and nuanced than most people give them credit for, and that it’s important to think in a rigorous way about the ramifications of various ideas.  So I tend to be in the mode of Blaise Pascal’s famous quote (often attributed to Mark Twain), “I wrote you a long letter, because I didn’t have time to write you a short letter.”

Some readers might be new to the whole giftedness thing, and might be overwhelmed.  I get a lot of phone calls and emails that amount to, “Help!  How do I get started?”  I’ll try to tag things in terms of whether something is a general-interest post (or even a GT 101 FAQ), or whether I think it’s mostly going to be of interest to the folks who are as geeky as I am.

I love to hear suggestions from people about specific topics or questions they’d like me to address, so feel free to comment here or to send me a private email (  I also like suggestions on how I can more effectively use this or other online forums — despite having been on the net for over 25 years (!), I tend not to be a particularly rapid adopter of the new and shiny.

Categories: getting started Tags: