Assessment in Math

Getting Ahead of the Curve

My school has recently begun an inquiry about how we assess. I have been struggling through the process for a number of reasons. I struggle most with trying to break away from assessing how I’ve been assessed. Mathematics lends itself to a certain type of grading – that where we think about the “skills” of the class.  I’m very good at this game we call school, but many of my students are incredibly bad at the game, yet great at mathematical thinking.

The main question I go back on forth on: What does it mean to earn an 80% in a Geometry class? Does it mean that you’ve mastered 80% of each of the objectives? Does it mean that you tried really hard, but did not understand all of the concepts? Or does it mean that you know everything, but were kind of lazy and missed deadlines? Where do expectations fit in?

Just like the Fed is supposed to control inflation and stimulate the economy with one lever (setting interest rates), so too does a teacher try to represent too much with its lever: setting grades.

I’ve spoken with some of my students about this, and we’ve come to two general conclusions. Historically grades represent ability (think skillsets and applications) and work ethic (meeting deadlines, trying one’s best).

One Assessment Model

I tried to graphically represent a model of this historical assessment:

image

This two-dimensional graph has a number of implications:

First, when we think of skills, we tend to think of “hard skills” such as multiplying fractions or using the chain rule correctly. We lose the “soft skills”, such as the highly touted 4 C’s: communication, collaboration, creativity and critical thinking. Each of those is difficult in and of itself to assess

Second, how do we weigh these two dimensions?

Should we base grades solely on the content that students mastered? This brings up a few thoughts for me: If you were really great at Calculus, you might be able to walk into Calculus 2, listen to the lectures, try a couple problems and understand all the content. If we weigh only the objectives of the class, how can we justify giving a 0 for a missed deadline? The grade will artificially be lower – the student understands the material but their grade does not reflect it.

And what of work ethic? If a student is really trying, improving, busting her ass on every assignment but still only masters 79% of the material, we give her a C. That somehow feels cold and unfair. But standards/objectives-based grading would imply that she earned exactly that grade. The counter would be: multiple studies indicate that humans with higher work ethics are more successful than humans with solid skill sets. So perhaps we should prioritize work ethic…

Third, we can think about the mathematics behind this model. On a 1-100 point grading scale, lacking a work ethic will destroy any representation of skills. Miss a couple of assignments, ace all your tests, your grade could be interpreted as skill-deficient. You can get by more with good work ethic/poor skills than poor work ethic/good skills. Either mindset lends itself to gaming grading.

Trying to Give Weight

I began playing around with what weighing these dimensions with grades might look like, and I came up with…

image

Yes, yes, I know the proportions are not exact. Feel free to create a better one.

And just for fun, I put some hypothetical students on the original scale. I started thinking how I approach those students, and the results were delightful and frightening.

image

I’m so drawn to people with high work ethic, but become too skilled, and I’m flustered on how to keep you engaged… One class wanted me to put a point where I thought each of them landed. I don’t advise you to try it at home.

Conclusions

I’m still at a loss for how I want to approach grading philosophically. I had a great professor at Baylor tell me that he did not want to give out any grades, but write two sentences about each student. I think that mindset is powerful, but rarely feasible. Maybe I’m approaching this the wrong way and equivocating grading with assessment. I would love to know some other math teachers’ (or just teachers) thoughts on this particular model.

Flipping Classrooms: A behaviorist by any other name?

If you only read the headlines, you might think that education is rapidly moving towards a monumental shift. Venture capitalists, tech start-ups, and Bill Gates himself hold the idea that with better data management, we can revolutionize education. Many of these ed tech companies employ some sort of “flipped classroom”, where students are autonomous beings who are given scaffolding to learn skills at their own pace. There are various reports to the efficacy of these practices, but they are drawn to the Mathematics world more than other domains.

Pursuing the Dream

I want to discuss the methodologies, pros and cons of one such system: Dreambox learning. Dreambox promises to “meet students where they are” with respect to their mathematical abilities and then to “accelerate” their mathematical learning. Their methodology for accelerating students is a simple GUI where students are to complete various tasks, such as adding fractions or comparing numbers with decimals.

Let me say first that I do not wish to single out Dreambox or any other “flipped” classroom approach such as watching videos and taking subsequent quizzes at Khan Academy.  However, the debates center around logical (and rarely empirical) arguments, and conclude that these approaches are good and that therefore we must spend money on them.

As I wrote last week, there seems to be a plethora of jargon on the systems used by many of these educational systems. Here are some jargon-y words from Dreambox’s FAQ: differentiated; adaptive; learning engine; gaming fundamentals; rewards; aligned (to Common Core standards); intervention; proven; and real-time reports.

Dreambox promises that for every minute that a student is engaged with their software, their engine analyzes 800 different pieces of information. Everything from the keys they press to the duration between those presses is analyzed and spit out to… somewhere. “The kids love it because Dreambox looks like a game”. I don’t really take issue with anything that they’re trying to do so far, with one large, glaring exception. The headline that Dreambox promises states explicitly that “Every child can think like a mathematician!”. Dreambox also markets itself like a teaching assistant with “unlimited patience and a perfect memory”.

The “game” I tried on Dreambox had students practicing identifying decimal values on a number line. Here’s a screenshot of the software in action.

image

And since I really am terrible at knowing my decimal places, I put the pin in the wrong place:

image

After I guessed incorrectly a couple of times, the narrator of the “game” said she would help me and gave me an entirely different problem. She led me through using the magnifier and basically solved a similar problem for me. I think as a teacher, this is exactly what I would want a teaching assistant to do: solve similar problems with the student giving scaffolding, assistance and problem solving. But how did this particular game meet me where I currently am? Did it analyze why I might have gotten a question wrong? As a human being that can ask questions that aren’t so binary, I still struggle with determining students misconceptions. Call it the experts’ (ha!) blindspot. But this system is no more adaptive than the “teaching machines” the behaviorists created.

(One more thought: Game like? I mean.. there’s definitely color? There’s a funny shaped character with what I believe to be a monocle. But I’m not sure where the playfulness of a game exists here.)

Behaviorist Approaches

In the above video, we can see the emergent technology of the behaviorist: the teaching machine. Although it might be comical to watch the antiquated technology of the 1950s, they hold striking parallels to the emergent technologies of today.

Listen to Skinner’s opening line. “These young people are studying in a new way. The class is in spelling, but it might as well be in arithmetic, or in algebra or grammar, or anything involving the use of words or symbols.” This approach of content agnostic technology is what gets at me the most. The idea that mathematics should be taught in the same way as spelling or grammar is beyond fundamentally flawed – it loses the idea of divergence completely. Forget the connections of divergence to GDP, or economy or even being happy. If we’re dealing with a dichotomous world – wrong or right answers only – we lose the reason that we studied any of these topics at all. As Paul Lockhart said, we don’t study Mathematics for its usefulness; we study Mathematics because it’s so damned interesting.

Second, listen to the words that Skinner uses to describe these teaching machines: effective study; immediate knowledge (feedback); leads to correct behavior; motivating; free of uncertainty or anxiety about being wrong; scene of intense concentration; and quick report. I do not know the history of teaching machines or why they failed, but the connection between yesteryear’s and today’s “teaching machines” is fairly obvious.

So if this technology has been around for over half a century, why is it just now that their use is being so proliferated? Is it a political issue (grandstanding on the idea of improving education) or an efficacy issue (only now do we have the processing power to truly utilize these machines). I am unclear which it is. However, I do have one question: how would a teaching machine of the past or present handle a divergent question like this:

Conclusions

Dreambox’s ultimate goal is to have every student think like a mathematician. Nowhere in their software could I find the use of thinking in extremes, solving a simpler problem or analyzing assumptions – all key components of how mathematicians think.

If there’s any message I could get across to these ed tech companies, it’s this: A skill-based approach will not a mathematician make. Mathematicians are interested in good questions, not how to follow instructions. Trying to predict what a student might say to a convergent question is easy – and you can get many data points. But is it useful? Will it help a student think creatively? These are the kinds of questions that will drive innovation in flipped classrooms and self-pacing. However, it doesn’t seem like we’ve made much progress there.

Divergent Question: Plastic Edition

https://i2.wp.com/farm3.staticflickr.com/2764/4028595631_35c17923c6_z.jpg

Recently, the city of Austin instituted a ban on plastic bags. The city found that about they spend about $850,000 collecting, transporting and disposing of the non-recyclable plastic bag. The city council also approved a $1.5 million educational program about the new law.

Research and give arguments both for and against the ban. Consider using logical, empirical, environmental and constitutional arguments.

Notes from #SXSW

Inspired, ignited and discombobulated describe best my mindset. I feel like I’ve travelled here from another planet even though SXSW is approximately 6 miles from my house. Big thanks to the Khabele School for getting a ticket for me. I have just recently discovered the vast bucket-turning-into-an-ocean that is the educational blogging/tweeting community. To call it rich is an understatement. SXSW Edu was also my first educational conference (not relegated to just math), and I wanted to share my thoughts from a novice perspective. All of the talks can be found here. Below are some of my initial assessments (no pun intended) of the event.

On Conference Life

First, I had an incredible time on twitter seeing what everyone was talking about. More often than I would like, I wanted to go to more than one presentation at a time. However, I gathered a lot of good information just by the hashtag of various talks. On the other hand, I was shocked to see how much blatant self-promotion and product hocking went on. Color me naïve. Even the Lego’s Robotics program gave a presentation regarding their latest/great work the EV3 would enrich schools, but very little evidence of what that really looked like.

On Big “Ed Tech”

Here’s a primer:

Keeping in mind what Audrey Watters called “the corporate-driven conference agenda and the lack of educators”, I was floored how few panels actually included real life teachers. I know we are an elusive species and difficult to find in the wild, but damn. With the influx of app developers, venture capitalists, and “non-profit learning companies”, I would have loved to see more actual teachers sharing best practices. I could not find a mission statement on SXSWedu’s site, but in their about page they mention creativity and innovation. Creativity and innovation seemed to be relegated primarily to developers of a platform, app or environment.

The ideas of divergence rarely came up – especially in the “personalized”, “just-in-time”, “adaptive” educational technology secret sauces. But Alfred Solis of the “8 Elements of Project Based Learning” asked a very simple and intriguing divergent question: When does 1 + 1 not equal 2?

Bill Gates keynoted the event. I understand he didn’t get famous for his speeches, but it was pretty hard to watch. He began his talk by comparing education to polio. The basic premise of all of these ed tech talks is that Schools are broken and therefore we must (something really cool with flashing screens). Half of the talks began with showing some graphic or another showing Finland and Singapore’s superiority w/r/t Math scores. I understand: you can’t get funding without painting a bleak picture, but that doesn’t make their case necessarily valid.

I’ve heard of NBA draft drinking games, but I’m worried if anyone starts one during an educational conference. Any of the following words would have caused SXSW to look like Jonestown: personalization, seamlessly, point of instruction, adaptive, or standards based.

One last point: “Content agnostic” should be a bad word when it comes to learning environments. To think studying Mathematics looks anything like studying History is a gross simplification. Yet “content agnostic” seemed to be a real swell idea.

I will post next week my comparisons of many of these ed tech companies to the work of B. F. Skinner.

On Classroom Life

But the conference had some extremely useful speakers. I’ll try to be brief

Jeff Sandefer at the Acton Academy might have changed my life forever. He started with flatly saying that most standards taught were useless – that’s pretty strong counter point to almost every other panel at the conference. He had 7 outstanding principles that guided his school (two of my favorites: Every child is a genius; Never answer a question, ever).

Karen Fasimpaur gave many technology specialists and teachers some great resources on OER’s (Open Education Resources). They might change what education looks like or they might be ignored completely. The biggest challenge for OER’s is the quality control issue: Who is going to manage this beast? This could be solved by giving grants out to young, tall enterprising people from Austin to write one.

Wesley Fryer dropped a reference to Seymour Pappert! I’m guessing constructivism would be viewed by technology specialists like the ancient Wiccans would be viewed by modern agnostics. But I appreciated that students should be given support to construct their own knowledge at some level.

Speaking of obscure references, Tomas Riley of the Children’s Creativity Museum dropped a reference to Paulo Freire and how he wanted his museum to be “of the streets” vs. “on the hill”. If there had been a panel dedicated to critical pedagogy, how many ed tech people would have shown up? The idea of access was dismissed by several panelists.

One of the most powerful lessons I learned was from Elisabeth Soep of Youth Radio at her “Making Apps with Youth” panel. She compared the decision to give students access to open-ended platforms as similar to the canonical Jimmy Wales “steak knife” story. I teach in a very loving, yet traditional environment. I still operate under a very basic pretense: the possible costs of using computers/laptops/phones outweighs the potential benefits of those modern devices. This is an **extremely** difficult transition for many teachers to make. Especially when students who are classically trained without calculators might fare better than those without on certain assessments.

And coming full circle, the tension between power and quality control is still at the heart of most educational conferences. I’m refreshed and ready to join the larger internet community in searching for best practices.