We’ve been asked a few times now if we were being reviewed in this years Make Magazine 3d Printer Shootout. We are not, and I wanted to explain why. Last Year, we were very excited to be included in the review. I was a charter subscriber to Make, and credit them with a lot of my inspiration for being a Maker.
The only issue was we did not then sell assembled printers and Make specifically asked for an assembled printer for review. That was ok, we made one up and packed it up and shipped it off with a letter specifically explaining that we do not sell assembled printers, and advising them to contact us in the event of shipping damage. Unfortunately, when it arrived, the package had been dropped, causing extensive damage to the printer.
We promptly assembled a second printer and revised our packing material and process, and reshipped it, this time the printer arrived nearly perfect. The fan mount on the print head had broken (a 5 minute print to replace it) and a single screw had come loose. Otherwise, according to the email we received upon delivery there were no issues.
In spite of that, and in spite of the fact that they were specifically advised that we did not normally sell assembled printers, they rated our printer “Caveat Emptor” due to the damage in shipping.
At Maker’s Tool Works, we welcome constructive criticism. Based on the experiences we had shipping those two printers, when we did start offering assembled printers a short time later, we shipped them with nearly a zero damage rate. We ship our kits world wide, and even with the fragile glass piece, we rarely have any breakage. We take our shipping seriously. We strongly feel that the review was unfair. At the very least  we should have been given a chance to respond prior to the review being published.
In addition to the above specifics, we also have general issues with the review methodology.
  •  First, we sell kits. Reviewing our printer as if it were sold assembled gives an inaccurate perception of our printer, the buyer should understand what they are getting.
  • They use a different reviewer for each printer. Each reviewer can put as much or as little effort into working with the manufacturer to fix issues. Other than resolving the original issues with the shipping damage, our support department had only one brief contact with their reviewer, where we responded within 90 minutes of receiving their email.  We have heard rumors– and have direct knowledge from the 2012 review– that other vendors were given extensive time to address any issues that came up (at least in 2012, some vendors were on site and actually helping tune their machines for the review).
  • The other issue with different reviewers for each printer is just the difference in skill levels. An experienced printer user can work through minor issues where a beginner might struggle. This can lead to unfairly biased reviews.. I don’t think this was necesarily an issue with our review, but it is a big problem with the basic format of the shootout.
In the end, none of this is actually why we were not reviewed this year. The simple answer is Make did not even bother to contact us. That is fine, it saved us some soul searching about whether we wanted to participate or not.