Just over two weeks ago, we launched WooCommerce version 2.4, the Helpful Hedgehog. From our standpoint, we thought the release went well. However, while we thought this was a pretty good release, we think we can do better.
In the interest of openness, we thought we’d share some of our observations from this release. We also wanted to share what we learned as we deployed 2.4, and we’re planning to do better for the next release.
Let’s start with what went well.
What went well: testing 2.4 on WooThemes.com
The aspect of this release that I’m most proud of is WooThemes.com running the final release candidate of 2.4 before releasing it to the public.
We did this to hold ourselves accountable. If our development team doesn’t inspire enough confidence to run this on our own site, then we’re doing something wrong and we shouldn’t release it.
It’s up to us — the development team — to do whatever is necessary to inspire that confidence. That could be writing more unit tests, releasing more betas, running the release candidate on multiple test sites, creating more documentation, and so on.
Running the release candidate on WooThemes.com helped us catch some bugs with variations prior to release. It also put us in your shoes. Testing on our live site makes sure that if there ever is a bug with the cart, checkout, or payment process we’ll feel that pain as much as you.
We think that’s fair. If we’re not ready to use it ourselves, we shouldn’t put the release out.
We really liked this process. As a result, we hope to do this going forward with each of our major releases.
What didn’t go so well: backwards compatibility
While we did pretty well testing our own extensions and themes, we didn’t catch everything. As you might already know, we changed the way variations are handled.
While we didn’t notice any issues with several WordPress.org themes with built-in WooCommerce support, or our other WooThemes like Storefront and its child themes, there were some issues with third party themes sold elsewhere.
We’d like to get better about ensuring backwards compatibility with third party theme developers who are working to make their themes compatible with WooCommerce. This would have prevented some of the issues with variations that were brought up in comments on the release blog post and in support tickets.
While we can put out more documentation about the changes we’re making, why we’re making them, and how to update your theme or plugin, more documentation won’t help us achieve backwards compatibility. For that, we’ll need to make an even greater effort to reach out to as many theme and plugin authors as we can, informing them of the changes being made and how to keep their theme or plugin up to date.
Backwards compatibility is one of the core philosophies behind WordPress. As such, everything we do should be completely backwards compatible.
We’re open to ideas as to how we can make this happen. If you have any suggestions, feel free to leave a comment below.
One idea: using live data
We did have one idea that might help us address the above. Here goes:
The best way to solve this moving forward is to use actual data from real stores. While we tested WooCommerce on WooThemes.com, our site is just one way to use WooCommerce. There are an infinite number of ways to configure your store and products.
Getting store owners willing to test major WooCommerce releases with their own combination of plugins, themes, and customizations is essential for us. It would help us make sure we have covered as much as possible during testing.
Test data created for the purpose of testing is never as good as actual data. And we can never predict how a store owner will use WooCommerce, no matter how many hours we spend testing extensions or themes.
If you’re interested in helping us with your particular use case or store, please leave a note in the comments — we’d love to hear from you.
Our plans moving forward
Overall, I reckon we did a good job. We can, however, do better.
WooCommerce has a ton of incredible features built into it, and it’s very robust. This means that as we grow, we’re going to require an enhanced testing process, as well as regular build tests, unit test coverage, and testing as many use cases as possible.
Our focus is on crafting a continually stable platform for the running of your online store. We’re excited to continue to do this, every day.
Those are our observations from release of the Helpful Hedgehog. As always, if you have any questions or comments — or suggestions for what we can do better — you’re welcome to leave them for us in the comments.