The Average Google Star Rating of HVAC Contractors in Massachusetts is 4.6
Why sales pitches touting all these Google Reviews are kinda meaningless
Correction: it actually rounds down to like, 4.6. Sorry, email people.
So once upon a time, I was a data scientist. I still retain some of those skills to this day, and I used it this winter to retrieve Google star rating averages & counts for the entire HPIN catalogue (or at least as many as the Google APIs could match to an actual business).
Ever wondered what ratings inflation looks like? Here it is:
It looks like the vast, vast, vast majority of contractors getting 4.5+ stars.
This is a problem, because HVAC sales will always hit you with the
“Yes, this is expensive, but you’re paying for quality. Look at all the Google star reviews we have!"
And this is meaningless, because the vast majority of contractors will have averages above 4.5 stars anyway.
An Industry-Wide Problem
I’m not the first person to notice this. Take a look at this WSJ article:
It is, indeed, the spiritual cousin of the now-infamous
That has gone so far out of hand that apparently even self-checkout screens will sometimes ask for a tip.
But I digress. This rating inflation problem has been on the radar of companies like Airbnb for a while (I know because I talked to their data science team about it once) and it dilutes the value of that combined average.
What does needlessly overpaying look like?
Here’s one such example:
GF2222, you can and should find a better deal. I hope you did.
Okay, what if you filter out all the small, indie contractors?
I filtered out all contractors with fewer than 10 reviews. We get:
As you can see, this kinda tanks the count of 5 star contractors, which implies that there are a lot of indie contractors with like, 3 reviews and all of them are 5 stars.
But the 4.5 star to 4.9 star bucket remains mostly unchanged, indicating that the vast majority of mature HVAC contractors live somewhere in this range.
The takeaway here is that
Look at us. We have a 4.7 star rating on Google, with 150 reviews!
is NOT a selling point that should persuade you to pay significantly above market average.
How we use star ratings
See appendix for more, but we basically use star ratings as a first pass to filter down the universe of 1,000 HPIN contractors to a more manageable list of ~200 contractors who may be good candidates for further vetting/price benchmarking.
Within the world of 4.5+ star contractors, there are a lot more medium-sized contractors (with like, 50 to 250 reviews) than the ones with 1,000+.
The 1,000+ review ones are probably overpriced and typically have higher overhead, and also spend more money on marketing/sales
You may be able to find some gems in that 10 to 50 category, particularly like, the master-technician-who-started-their-own-HVAC-firm type of person
50 to 250 is kind of my favorite, but surprisingly, there’s no consistent cost trend here. There’s a lot of pricing variance within each bucket
The purpose of our research benchmarks at Laminar Collective is to dive deeper into select groups of promising contractors, so we can uncover the highest quality contractors with price discovery (~2 dozen, 3 dozen contractors per season) and deep vetting (~15 contractors per season).
Appendix
Actually, I’m a baller
I’m happy for you. Want to buy the luxury car of heat pump installations by dropping an extra $20k above market average on the biggest installer in town? Yeah, go for it. I won’t stop you. Just like how I won’t stop you from buying that Land Rover.
But for the money, I’d personally opt for one of these Quilt systems if I were you:
Yes, in my non-existent baller life, I would take a Rivian over a Land Rover.
You can use star ratings as a red flag
This is what we do. Though you may not want to pay above market average based on star ratings, you do want to filter out contractors with poor star ratings:
Filter out contractors with no star ratings, or minimal star ratings (e.g. <10)
Filter out contractors with lower than a 4.5 star rating
Based on our dataset, only about 25% of contractors (who have ratings) have an average lower than 4.5 stars. You’re still left with 75% of the contractors.
Immigrant Contractors
So I grew up in an immigrant family. And I know that out in California, you have some smaller contractors who basically only work with their local immigrant/ethnic community, and everything is word-of-mouth. They speak your language, they’re in your Whatsapp/Wechat/family networks, etc.
These are the kinds of contractors who see no need to curate Google reviews, or even have an online web presence.
And you know what? That is sometimes perfectly fine, particularly if they come personally recommended by a relative; their accountability mechanism isn’t the risk of getting a bad review on Google. It’s the risk of repetitional harm in a social community.
But if you don’t have that personal recommendation, just be diligent with the vetting. That’s my take.
Okay, that’s all.
I’m out. Peace!