Why you oughta be skeptical when reading or watching critiques of pedals.
Reviewing a musical device is perilous. Though I've conducted exactly one boatload of demos, I've always avoided doing reviews. Reviews require judgment.
A pedal is not a hammer. It's not a thing that succeeds or fails in any objective way (unless it were to catch fire). It does stuff to sound. You may like the stuff; you may dislike the stuff. The truth is we usually just compare new things to old things, and we like the old things because music history has proven they work. But there is a difference between proven and better.
For example, last week I was shopping for a microphone, and the review I was reading mentioned that it didn't perform overly well when recording a string section. Too bright. And by default, my brain subtracted a point from the overall goodness of the microphone. But then I remembered: I've never recorded a string section! Not even once. I bought the microphone and it's just right for my needs. The brighter profile sits well alongside the ribbon mic I typically use.
All musical objects fall somewhere on a spectrum that makes them more or less suited to review. A tuner is more reviewable than a preamp. A tuner's job is to be dependable and transparent—pretty quantifiable—but preamps are often prized for the character they impart. This is sketchier. Whether a preamp's influence is welcome depends on your mics, for example, as well as your taste.
Pedals are at the extreme end of the “less reviewable" spectrum. They exist to transform and mangle your instrument, and their success largely depends on context—the frequencies of the other instruments around you, the type of input signal, the intention of the music. But reviews are a valuable part of the hunt, so let's discuss some of the pitfalls and limitations, and boost your healthy skepticism.
Beware the rating. The only truly valid way to “grade" a musical instrument is with a list of pros and cons. Musical devices are not a math test. They can't be 4 out of 5 good. Each one has flaws and strengths and nothing else matters.
The problem with a rating system is that it casts a shadow over all other aspects of the review. Even if all the features line up with your needs, a pedal that scores 6/10 doesn't seem very appetizing. It's the final judgement, and it makes it seem like one pedal can be superior to another, when really it's a matter of preference. And if something truly stinks, a long list of cons tells the story just fine.
Trapped in the honeymoon. Reviews can, typically, only cover the getting-to-know-you phase of the person/instrument partnership. Sometimes this makes devices seem better than they are; sometimes it makes them seem worse. But since you're presumably buying for the long haul, at some point you will probably have a deeper understanding of the device than the reviewer.This is one reason it can be helpful to balance reviews alongside experience from friends or online forums.
The good kind of subjectivity. Subjectivity is the most important and most hazardous part of a review. With music there's just no way around it, so the best thing to do is lean in. Subjectivity is only hazardous when it's concealed.
I want to know everything about reviewers: their process, experience, and tastes. I want the review to unfold like a first-person narrative. How was the pedal tested? Was it played alongside something similar? How does it compare to past favorites?
If I have little to nothing in common with the reviewer's musical approach, there really isn't much for me to learn. Beginners can and should write reviews, and their outlook will often be the best fit for others with similar experience—but I need to know! Who the reviewer is matters more than anything.
Know your need. Many of us have a strong desire to buy the good thing. But knowing your personal aims is far more valuable than buying the 10/10 reverb. Most stuff out there is pretty good, and the things that really matter—that make a piece of gear a fixture in your music—are too personal to ever be captured in a review. They only come with experience. My collection of musical devices isn't logical, and yours probably isn't, either. It's made up of stuff that just happened to work for me, often for reasons that don't make sense.
So, more than anything else, try not to worry too much. Follow your curiosity. If something suits your needs and the reviews are decent, try it on and see how it fits. At worst, it will deepen your understanding, and clarify your needs.
And that is 8/10 good.
