Unlike most other evaluations, I actually took the time to do a deep-dive evaluation for your site. (For those questioning my fitness to do this, I'm a graduate of one of the top business schools in the US. I know enough to be dangerous, as one of my professors liked to say.) Comparing findings, I saw that the community and I mostly agreed on what was/wasn't a good questions, with a couple of divergences.
The key takeaways from my analysis of your questions were:
- This site excels most when the questions are specific. I noticed that the specific questions received not only the highest amount of participation from our established users, but also proved to be the ones that had us showing up better in Google search results.
- We don't excel at definition questions. This is in large part due to the large amount of competition in the personal finance market. It's going to be impossible to beat Wikipedia, Investopedia, or any of the other static sites that provide basic financial terminology information.
- Mathematical formatting is difficult to read. While math formulas are rarely needed on PFM (usually they're brought up in relation to TVM questions) the occasions where they are needed are obvious. I would have loved to see the mortgage vs. invest question from the last evaluation have the top answer edited to have the formula formatted in LaTeX or even in MS Word's equation editor. It would have taken little time and would have made that question infinitely more readable.
- Comments aren't answers. I saw several times a succinct, excellent answer left as a comment! No! Don't allow this. Comment back and suggest to the person that they convert their comment into an answer. Alternatively, turn the comment into an answer yourself!
Community Engagement and Question Quality
Now, as far as your community is concerned, there are some things that came up.
- Questions aren't over-answered. With an answered rate of 98% and an average of 2.6 answers per question, this means that most questions get enough of an answer for the asker to be satisfied, but not so many that new users feel powerless to respond to anything at all without repeating themselves. (This is the opposite of a forum, where everyone is happy to repeat the same points made before.)
- Traffic is on the up and up. Over the last six months, your traffic numbers have increased significantly. I've been the person most consistently doing your evaluations; my observations indicate that this number is up from what it was at the beginning of this year. Part of what helped was most certainly the tax ad we put up on the network back in March/April.
- Question quality is good. For the most part, this evaluation revealed that, the majority of the time, we are doing as well or better than the rest of the internet at answering questions about personal finance topics.
- Your user retention is low. This isn't something you can directly affect (though the following two points should help with this) but it's worth bearing in mind. Your attrition rate is high: half of all users with less than 1,000 reputation are effectively one-and-done, "drive-by" users. By that, I mean they are people who will ask one question, nod in understanding (maybe even upvote), and then peace out. It's a challenge this site will face for the entirety of its existence.
- Needs more community moderation. This means more editing, as well as more closing, more voting to close/reopen, and more flagging. Take for example the answer here. Dilip's answer is excellent, but I would have loved to see someone link directly to the tax code. It would have pushed this answer past the edge a little bit to be as amazing as possible. I felt like adding citations to law or tax code, pictures, a chart, and all sorts of small extras to posts would have improved their readability and presentation immensely. I'd love to see more community collaboration on posts this way. I'd also love to see more of the usual moderation, too, as after 2-3 years of existence you've got a huge base of questions and answers and knowledge to draw from.
- Small scale promo work would be amazing. I'm not talking about renting billboard time in Times Square -- I'm talking something as simple as posting interesting questions/answers in
/r/money. Yes, Reddit isn't exactly the best way to get people on board, but I know more than a few people involved in finance (personal and professional) who spend hours each day surfing Reddit. It'd be a great way to bring in people who would otherwise be wasting time on The Motley Fool boards. It would also be a great way to see if we can push the boundary between QF and PFM.
I'd like to see this community's questions per day stat go up. I'd also like to see more community moderation. I'd also love to see people starting to take a hard look at older questions, see what needs some tidying. I'd love to see some low-level community promotion from things as simple as posting to Reddit or newsgroups or finance course forums and mailing lists. If you regularly read a finance blog, comment on their entries more and then maybe pass along an interesting questions link (using the list of greatest hits questions is always a great place to start!) could make a major thing happen.
Look, I won't pretend I haven't been extremely critical of this site in my previous evaluations. I've done my best to place myself in the shoes of Random Internet Person and look at this site from a demanding perspective. Before this evaluation, this community was consistently found wanting. This is the first evaluation where I can say that you all exceeded my expectations. Your growth has been consistent, your question quality improved significantly, and I was happy to see more people commenting and voting on this evaluation than on the last.
We're very happy to see how much progress and improvement you've all made in the last six months. I, for one, sincerely hope that you all choose not to rest on your laurels. Instead, strive to make this site 1% better in some way, large or small, every single day. I truly believe that, with time, this site could be an unparalleled resource for personal finance questions on the internet.