Just over two weeks ago, we launched WooCommerce version 2.4, the Helpful Hedgehog. From our standpoint, we thought the release went well. However, while we thought this was a pretty good release, we think we can do better.
In the interest of openness, we thought we’d share some of our observations from this release. We also wanted to share what we learned as we deployed 2.4, and we’re planning to do better for the next release.
Let’s start with what went well.
What went well: testing 2.4 on WooThemes.com
The aspect of this release that I’m most proud of is WooThemes.com running the final release candidate of 2.4 before releasing it to the public.
We did this to hold ourselves accountable. If our development team doesn’t inspire enough confidence to run this on our own site, then we’re doing something wrong and we shouldn’t release it.
It’s up to us — the development team — to do whatever is necessary to inspire that confidence. That could be writing more unit tests, releasing more betas, running the release candidate on multiple test sites, creating more documentation, and so on.
Running the release candidate on WooThemes.com helped us catch some bugs with variations prior to release. It also put us in your shoes. Testing on our live site makes sure that if there ever is a bug with the cart, checkout, or payment process we’ll feel that pain as much as you.
We think that’s fair. If we’re not ready to use it ourselves, we shouldn’t put the release out.
We really liked this process. As a result, we hope to do this going forward with each of our major releases.
What didn’t go so well: backwards compatibility
While we did pretty well testing our own extensions and themes, we didn’t catch everything. As you might already know, we changed the way variations are handled.
While we didn’t notice any issues with several WordPress.org themes with built-in WooCommerce support, or our other WooThemes like Storefront and its child themes, there were some issues with third party themes sold elsewhere.
We’d like to get better about ensuring backwards compatibility with third party theme developers who are working to make their themes compatible with WooCommerce. This would have prevented some of the issues with variations that were brought up in comments on the release blog post and in support tickets.
While we can put out more documentation about the changes we’re making, why we’re making them, and how to update your theme or plugin, more documentation won’t help us achieve backwards compatibility. For that, we’ll need to make an even greater effort to reach out to as many theme and plugin authors as we can, informing them of the changes being made and how to keep their theme or plugin up to date.
Backwards compatibility is one of the core philosophies behind WordPress. As such, everything we do should be completely backwards compatible.
We’re open to ideas as to how we can make this happen. If you have any suggestions, feel free to leave a comment below.
One idea: using live data
We did have one idea that might help us address the above. Here goes:
The best way to solve this moving forward is to use actual data from real stores. While we tested WooCommerce on WooThemes.com, our site is just one way to use WooCommerce. There are an infinite number of ways to configure your store and products.
Getting store owners willing to test major WooCommerce releases with their own combination of plugins, themes, and customizations is essential for us. It would help us make sure we have covered as much as possible during testing.
Test data created for the purpose of testing is never as good as actual data. And we can never predict how a store owner will use WooCommerce, no matter how many hours we spend testing extensions or themes.
If you’re interested in helping us with your particular use case or store, please leave a note in the comments — we’d love to hear from you.
Our plans moving forward
Overall, I reckon we did a good job. We can, however, do better.
WooCommerce has a ton of incredible features built into it, and it’s very robust. This means that as we grow, we’re going to require an enhanced testing process, as well as regular build tests, unit test coverage, and testing as many use cases as possible.
Our focus is on crafting a continually stable platform for the running of your online store. We’re excited to continue to do this, every day.
Those are our observations from release of the Helpful Hedgehog. As always, if you have any questions or comments — or suggestions for what we can do better — you’re welcome to leave them for us in the comments.
We would be very interested in helping with future releases when it comes to testing backward compatibility. We run an online vendor marketplace, and have some Woo extensions as well as quite a few from other developers. We do have a staging environment where we test everything, but we do keep it updated with live data. Drop me a line if you are looking for another tester, as it could be a great benefit to everyone.
Hey Chris – thanks for volunteering! Email sent. 🙂
After waiting until 2.4.6 was released, (after watching the bugs get beaten away on github) we updated from 2.3.13 to 2.4.6 and everything seemed to go smoothly for us. We would be more than happy to help test and provide our unique setup for your testing process. The only apprehension we would have is… quick access to a developer to help if a release did in fact break something.
We’ve got a fairly busy store, running canvas theme. We’ve got 40+ plugins installed, many from woothemes, many from codecanyon, so we have a nice mix of plugins to test against.
We’ve got this weird bug that is absolutely impossible to reproduce when we ask for help. It involves tax being charged for a fraction of our orders on shipping (on orders clearly out of state). Support said we needed to disable all plugins and keep testing and activating plugins until the problem comes up, problem is…. it seems that we can’t see the messed up tax through the checkout process… it doesn’t happen until the order is placed… and only some of the time. Any chance somebody can help us with that? I can most certainly provide sample orders where this happened, or even hire someone on the side if possible.
Hey Dylan – I sent you an email about testing and also about that tax issue. Let’s dig in. 🙂
http://taxify.co/ has a WooCommerce plugin, soon to be released, that will very likely solve your issue.
Any way around the significant page speed decrease when using geolocation and page caching support? Great feature but slows sites considerably…and every second counts in eCommerce…money lost, money gained.
Geolocation and cache are difficult to make work together. We added our own geolocation features to WooCommerce in 2013, and did a lot of work to make it work quickly.
Our opinion is that page caching should be fixed directly, without the workaround added in WC 2.4 (i.e. page caching prevents geolocation = page caching must adapt to allow geolocation). We are now writing solutions for several of the available caching systems. We have one for ZenCache, one for SuperCache is in progress and we are working with WP Rocket team. We are also going to write how to configure Nginx and Varnish to get the same result, and that should cover most scenarios.
The best part is that the approach doesn’t require reloads or extra lookups, nor URL arguments. Even on an ultra-cheap hosting provider, sites become fast, without losing flexibility. 🙂
We will be deploying our new site on Sept 1st which uses Storefront and it’s extensions as well as a bunch of other Woo Plugins (1 from codecanyon). I also have some custom code in a few woocommerce and theme templates in my child theme which you managed to kill with your update!
As someone who was directly affected by your recent updates (you have mashed my site with Storefront updates too!) I would be keen to help test any future releases. We have a WP Engine Staging environment that I can copy the live data to as and when needed.
Drop me a line if you are still looking for sites.
Hey Guys! Awesome work on the update 😉
Just one comment, could it be possible that you removed the default payment gateway option in the checout settings? I don´t have the radio button to choose default anymore!
This was a great release overall with some nice enhancements. I’ve only run into a few small bugs, as to be expected with a big release and I was encouraged to see how quickly the minor releases were pushed out to fix problems as they were discovered.
I’d like to make a suggestion for a smoother process with extensions. I usually wait to upgrade WooCommerce major versions until I know most of my extensions have been updated or tested. I watch the changelog.txt file for that information as extension updates are released. It’s very manual and tedious to keep track and, as of today, many of the extensions you sell still don’t say they’ve been tested or updated for WooCommerce 2.4.
How about introducing new lines in the extension’s readme.txt like WordPress does to indicate what version of WordPress a plugin needs? For example WooCommerce specific versions of the WordPress “Requires at least” and “Tested up to” might be useful. Those could be displayed in our woocommerce.com account, in your documentation pages, in our site admin on the WooThemes Helper page, and possibly other places. It would provide a quick reference to extension status before upgrading and remind developers to communicate that information in the extension readme.
As always, I would be happy to help with testing and making data available.
Testing using real data is the way to go. As a plugin developer, I am working with several clients to test my plugin before releasing the public version. They get a discount, I get my testing and we both win.
I loved the new configuration wizard, I need to research it and see if I can apply it to some of my projects.
We really like the speed and responsiveness we get with Woo commerce vs magento, looking forward to some more testing.
We recently received the following email from Paypal, which is frankly over my head. Is WooCommerce taking care of this change, or does it not affect WooCommerce?
As we have previously communicated to you, PayPal is upgrading the certificate for http://www.paypal.com to SHA-256. This endpoint is also used by merchants using the Instant Payment Notification (IPN) product.
This upgrade is scheduled for 9/30/2015; however, we may need to change this date on short notice to you to align to the industry security standard.
You’re receiving this notification because you’ve been identified as a merchant who has used IPN endpoints within the past year. If you have not made the necessary changes, we urge you to do so right away to avoid a disruption of your service!
Because these changes are technical in nature, we advise that you consult with your individuals responsible for your PayPal integration. They will be able to identify what, if any, changes are needed. Please share this email and the hyperlinks below with your technical contact for evaluation.
Testing in the Sandbox is one of the best ways to make sure your integration works. Sandbox endpoints have been upgraded to accept secure connections by the SHA-256 Certificates.
Full technical details can be found in our Merchant Security System Upgrade Guide. In addition, our 2015-2016 SSL Certificate Change microsite contains a schedule of our service upgrade plan.
Thanks for your patience as we continue to improve our services.
Yes, I have been bombarded by email from former clients with WooCommerce who have received these warning emails from Paypal. It would be awesome if you guys would publish a post insuring users that their stores will continue to work after these upgrades.
Thank you so much for your diligence and transparency. It is to be greatly admired.
Hey Christy & Joel –
We’ve got some information up on our Docs about this: http://docs.woocommerce.com/document/paypal-update-for-sha-256/
Thanks Aviva that was an easy test!
I am glad to see that you are working towards improving backwards compatibility in the future. I run an ecommerce business at https://herbolab.com/ with a ton of plugins and I only update woocommerce when I am ready to invest the next 1-3 hours finding out what plugin are no longer compatible and how to fix it. Because of that, I only update twice a year, max.
In addition, I am the CEO of https://shopitpress.com/. We have been working on a special front end bundler plugin that has variation support in the pro version https://shopitpress.com/plugins/sip-front-end-bundler-woocommerce/. This support was developed before the release of 2.4 and it was very frustrating to find out that we had to delay the release of the pro version because the variations were no longer working. Luckily the free wordpress version was not affected by this.
I would be interested in contributing with data from my ecommerce store, as well as getting early developer access to new releases if possible. Please let me know how.