Academic Websites Ain't Got It--The A/B test

The most dismaying article I have read in recent months is The A/B Test:  Inside the Technology that is Changing Business  in Wired.  So here is the basic idea.  Brian Christian describes a process of changing updating websites that relies of data rather than what the designers or authors think is "right," "beautiful," or "useful."  Here is the basic idea.  Rather than make design and layout decisions based on hunches or design principles, the point is the make the decisions based on how users actually use the website.  I know, I know, .. use data to actual make decisions... seems elementary, right. 

My experience in developing websites in academic settings is that most people don't even know that you can collect analytic data on websites and most academic staff wouldn't know what to do with analytic data.  The typical academic website rarely even goes through any basic user testing to see how it works.  Yikes. 

The A/B test is simple.  There are numerous ways to design websites and multiple of ways of organizing information.  The A/B Test idea is the try things out and see what works.  Compare design A with design B and see what users like by randomly presenting some users with one version and other views with other versions.  Over time this approach this approach will refine the design of the website and provide a means to develop a design that fits the user rather than the designers.


Here are the basic principles that Christian outlines:

1.  Choose Everything  instead of having to make choices.
2. Data makes the call rather than the person at the top makes the call.
3.  The risk is making only tiny improvements rather than the risk is making a huge mistake.
4.  Data can make the very idea of lessons obsolete rather than experience teaches us lessons.

Scholarship and Klout

My colleagues won't appreciate this comment, but I can't help but think that Klout will matter in the future in scholarship.  In some sense it all ready does, but we measure Klout by way of citations in referred journals.  I can't imagine this will be sufficient going forward.  At present my citations on Google Scholar exceed my Wekcitations is the major scientific databases.  What does this mean?  Am I more citable online than in print?  Is this more important?  My citations in Google Scholar does not include my Facebook views, my blog posts discussions, my tweets (there are none incidentally).

In the recent issue of Wired, there is the suggestion that my Klout score my help me get better dinner reservations (have they ever lived outside a major metropolitan city,  think not?).  Although I can't imagine my Klout score earning me coupons and free refrigerators, I can't help but think that scientific scholarship will ultimately develop some version of the Klout-type scores for scientific publications that will capture the wider use and reference to scientific studies beyond the current level of citation indices. 

Martin Weller who writes about digital scholarship makes a similar argument in a recent issue of The Chronicle for Higher Education.  Weller has written a thoughtful book about the course of academic scholarship in The Digital Scholar.