Thoughts on the WAA Certification Exam
I’ve been pondering this blog post for a couple of weeks now since I took the WAA Certification Exam along with eight others in the inaugural proctored exam at eMetrics in San Jose.
To be totally honest, I probably didn’t need to take this test. For starters, I’m not a traditional web analyst that’s down in the trenches doing the hard work of analysis, reporting and translating the massive amounts of data we’re all so fond of collecting into insights and recommendations. While these web analysts have something to prove to their organizations about the value of their jobs and the expertise they posses – frankly I have nothing to prove.
Additionally, I work for a well established consultancy with a great brand reputation and I’m not planning on looking for a new job anytime soon. Our clients are most likely going to work with us regardless of our certification status. Yet, I wanted to take this test because I do advise my clients on what they should be doing with web analytics from a strategic perspective. I speak frequently about analytics and how to interpret and deliver data in the most effective ways. So my vantage point cannot be void of practical knowledge that dictates what’s possible in a realistic world.
Thus, I took the test in part to illustrate to myself that I not only talk the talk, but am willing to put my practical skills to the test. And yes…I passed, so you’ll be seeing the CWA (Certified Web Analyst) designation show up on my credentials.
Further, many of you voted recently to elect me to the Web Analytics Association Board of Directors; and I thank you for that. I took the WAA Certification Exam, so that I could lead by example and educate others about what I genuinely believe to be a valuable test of digital measurement knowledge. I encouraged all of my fellow board members to take the test as well and several have done so and more are sure to follow.
But because I went through the experience of taking this exam, I am uniquely qualified to share my experiences that stretch way beyond the speculation of any detractors that criticize this exam. Thus, I give you the Good…the Bad…and the Ugly of the WAA Certification Exam.
The Good…
This exam is a true test of analytical knowledge that requires both business acumen and a deep understanding of applied web analytics. Like all things analytics – it’s not easy. In fact, it’s downright hard. The guidance offered by the WAA regarding a recommended 3 years of practical experience is sound advice. And even then, this exam will require web analysts to dig deep into their skill set to come up with not just acceptable answers, but the best answer. Out of the initial nine exam-takers, seven passed the test, which is good. Yet, the minimum passing grade for the exam is 60% and the mean scores for our inaugural group was 61.7% (maybe I should have saved that for the ugly). The high score among all test takers thus far was 70%. While this may open questions about whether or not this test is too hard, to me it shows that there is plenty of runway for analysts to showcase their superstar skills with high scores. And if it was easy, where everyone could pass, then what validation of knowledge would that really be?
As my fellow WAA Board member Vicky Brock Tweeted: “As an employer I’d hire folk who ace this, as it tests analytical skills not recall”. Vicky also shared thoughts on her experience. Much like Vicky, I believe this exam is a good test of knowledge that requires prospective certified analysts to know their stuff, which in turn demonstrates that the credential holds distinction.
The format is a familiar multiple choice answer system with four possible answers. Like most diligent test takers, I relied on the process of eliminating the ones that I knew were incorrect and then sorting through the remaining choices. This typically left me with two answer choices that could work, but knowing that one was better than the other, I was largely going on instinct to make the right choice. There is also a word question section that offered business scenarios and data sets leaving you to solve problems within the context of a specific business. These questions were the real gems of the exam and guaranteed to make your head spin. I love these types of questions, but perhaps I’m a glutton for punishment.
The big elephant in the room is the price. Without question, taking this exam is a financial commitment. I shelled out the bucks from my own pocket to do it because I believe in the value of certification. We as an industry are gaining momentum so quickly that analytics and data-driven cultures are all the rage today. The use of data is permeating organizations from the tactical to the strategic and ending up on the boardroom table, and in some cases, in financial analyst reports that end up on Wall Street. Yet, despite these significant gains, we have no designation to acknowledge that our Web Analysts are qualified for the job. This certification exam is that designation that will identify the truly proficient practitioners. In my opinion, this exam is worth every penny and I strongly believe that as more and more professionals acquire the CWA accreditation it will become the gold standard by which job candidates, consultants and trusted advisors are selected. When we reach this critical mass, those who aren’t Certified Web Analysts will be questioned with just cause…So why aren’t you certified?
The Bad…
I’ll be the first to admit that their are still some kinks in the system so it’s not perfect. Yet, nobody is so I’m willing to offer some leniency. For me, just downloading the application to sign up for the test was a chore. I offered feedback, so hopefully a fix is in the works now [there is], but when I registered the editable PDF application only worked if you had Acrobat writer on your machine, which I don’t. So after filling out the entire form, I couldn’t save it. I ended up printing out the pages and then scanning them back in to submit my application. Now, that’s more than I’d expect from your average exam taker, but I was on a mission. Also, be prepared to dig out your resume because the application requires listing all of your previous employers, their addresses, manager names and phone numbers. I was toggling between the application and my LinkedIn profile just to complete the darn thing.
**UPDATE** There is now a web based form that serves as the application, so no more downloading the PDF.
Next, it was very challenging for me to prepare for this exam. I did utilize the documents offered by the WAA including the Knowledge Required for Certification and the practice questions. The practice questions were actually great. They helped me to decide whether I was going to take the test and did closely resemble the actual questions on the test. I just wished there were more of them. The Knowledge Required document also contained a great deal of useful information, but after pouring through the 37 pages of material, I was still left feeling unprepared. The document mirrors the UBC course material, so it is thorough in describing what will be offered in terms of knowledge, but the meat of the work isn’t included in this document. It was all menu and no entree. So essentially, the document tells you what you will be tested on, but doesn’t teach any of the concepts. While they clearly state that: “Taking these four courses is not required to sit for the certification test.” those that do will be much better prepared than I was. I know that these courses are incredibly valuable and students rave about their success, but most professionals like myself don’t have the time to endure them – despite their value.
The Ugly…
So, I already ranted about the preparation materials and the costs above, but the Ugly for me was determining if I would actually re-take this test if I failed. The feedback that I received from the WAA did contain results for the four sections that were included in the test (Analytical Business Culture, Case Studies, Marketing Campaigns, and Site Optimization) and my scores for each section. Yet, this was the extent of the feedback on my performance. It was up to me to decipher which questions may have been within each of the four categories and where I needed to focus my efforts to better prepare for a re-test. To the credit of the Association, most standardized tests are scored this way and offer similar amounts of feedback – but most tests of this magnitude also have test preparation courses that teach the skills of taking the test and offer extensive feedback on skills necessary to score well on the exam. Thus, it was ugly for me because I can sincerely admit that I wouldn’t have paid to retake this test because I do not know how I would have prepared for a second exam.
The bright spot in this potentially ugly situation is that the WAA Board is committed to endorsing organizations that choose to develop WAA Certification Exam training programs. Since this test is still very new, these programs have yet to emerge, but the opportunity is out there. I want the WAA Certification Program to succeed for the WAA and for our industry. If the test-takers are better prepared to take the test through the help of a training program, then that’s a win-win. This type of prep course would offer me the confidence I needed to take the test again if I had failed…or for those of you taking the exam for the first time. Stay tuned for more news on this front as it develops.
The Summary…
This post is already getting long in the tooth and I’ve said a lot. The bottom line for me is that this exam is a strong indication of the digital measurement skills that an individual brings to his or her organization. Passing the WAA Certification Exam means that an individual is an expert in the field of web analytics. It’s an accomplishment that anyone in our industry should be proud of, and one that should receive accolades on top of accolades.
But that’s enough of my rant…What do you think?
I look forward to starting a long-term dialog on this topic, so please comment, email me or otherwise shout your opinions from the rooftops.
