On the Math site I saw this post in the Meta section -
Can a Mod respond with data for Money.SE? Curious how our stats look in comparison.
So here is a chart that I hope satisfies your curiosity. SE Mod said I could post this one.
(Beta sites shouldn't obsess over data like this. When we graduate, our info gets moved into http://data.stackexchange.com/ and the stataholic among us can go bonkers. It really shouldn't be too much longer now.)
As I see it, the downvotes are such a small percent of our activity, they rarely spike. They are probably drowned out by the US tax time up-votes