It’s been an incredibly tough few days at WooThemes, for ourselves and a lot of our users trying to use our services. We were hacked, and badly. Our lifeblood, the WooThemes database, and all the content on our server was deleted. Not only that, but the backups were deleted and all traces of how they got onto our server too. Leaving us with scraps to work from in re-building the mothership.
We panicked, we yelped, we formulated a game plan and then we grafted like never before (the picture above showing the team going loco on very little sleep & lots of caffeine) – acting as quickly as humanly ninjaly possible. We responded transparently to our loyal community, keeping everyone informed of our recovery progress on a temporary P2 installation.
Where we’re at
Long story short, as we’ll save the juicy details for another blog post, we’re now back up online, with a few small glitches and missing functionality that we’ll be adding back over the next few days. We’ve also moved host from VPS to the market leaders in hosting large WordPress specific sites – WPEngine where we now have a monster dedicated server with backups upon backups of backups, and super tight security. A special thank you to WPEngine’s Jason Cohen, who spent most his Birthday setting up our new server, and his team who have really re-assured us that our website is now finally very safe and secure.
Users should be able to login to their accounts and access their themes. A select few, unfortunately some of our newest users, might be prompted for some personal details to verify their account, and have the facility from their dashboard to select their themes again.
Likewise, some of our recent club subscribers might need to send us some personal details so that we can link your recurring subscriptions back to your club membership. We promise a very quick and efficient fix.
Important note: None of your private information was stolen, and as your card details and personal data were kept safe somewhere else we need to re-link some transactions to user accounts that were deleted during the malicious hack.
Thank you, thank you, thank you
We were absolutely blown away by your support, you could have cursed us for the downtime and how it effected you and your clients, but instead you sent us pictures of beer, bikini girls and pizza to lift the team spirits, you tweeted us and emailed us words of encouragement, and one user even went as far as writing us a song, entitled “Injustice – WooThemes Comeback” . Thank you. It really unified the team and helped us get through this tough time.
Without sounding like an Oscar acceptance speech I need to say a massive thank you to the whole team for really pulling together and demonstrating the awesomeness that WooThemes is capable of. A very large and special thank you to Jeff and Warren who worked tirelessly through many a night fixing the unfixable mothership!
About
Welcome back ! 😛
Yep. Welcome back!
Good to see you back! That first descriptive paragraph is down-right scary!!
Nice to have you guys back !!
Great work guys, you kept everyone informed and kicked ass on getting back online… Great to see you back.
From the team at Number8Media NZ
Welcome back online WooThemes!
After dealing with this, there is nothing too big for the ninjas to handle! Although you all probably deserve a proper nights sleep and a day off first 🙂
Having followed your blog and tweets for the last few days, I’m now in no doubt that WooThemes is the most professional development team out there! You did a fantastic job of keeping the community informed.
Nice to have you back.
Congratulations, It was a fantastic job. I hope never repeat this.
Willkommen zurück 😉
Welcome back, I am glad you made it!
I look forward to the upcoming article on ‘how to properly secure’ a WP server and some of the tricks you may have learned along the way. This is just yet another reminder that it can happen to anyone and I’m hoping there will be some awesome insights to share with the community.
Definitely Shawn. We’ve learnt heaps through our own mistakes and misfortunes. Watch this space!
I also look forward to any insight/tips that you guys can provide (for WC too). I just don’t think that I could take being digitally violated that!
I would also like to say that you shouldn’t be in a rush to go back into production. I’d be quite happy for you to do what ever it takes to get secure.
Good Luck!
Thanks for the support. Product development has gone on temporary hold, but don’t worry we’ve got loads of goodness waiting to be released soon.
No Rush .. So long as I can get to themes and the forum I’m happy.
Positive from a negative
Online hacks are now day to day life for web developers, and how to try to best protect against these, giving the customer the best web experience while keeping their data secure is an impossible task
I look forward to the next blog post from the ninjas with tips regarding securing wp / hosting
Props to the updates
Oh yes, we’ve definitely learnt a lot!
Welcome Back!
Hey guys. Glad to see you are back up and running. Heard good things about WPEngine too.
Great job guys , lovely to have you back 🙂
Great to see you all back at it. Well handled, folks! Well handled!
Dre
So glad to have you back! It really can happen to anyone, and I along with everyone else REALLY appreciate the updates along the way!!
Glad to see you guys are back. Could you guys share more info on your experience with VPS.net. We moved with those guys because we saw you guys were with them … a big name client in the WP community. Now we are scared we might get hacked too 🙂
I am glad to see you guys back though. Woo rocks 🙂
You guys have demonstrated, once again, why I choose to do business with you. You handle things professionally at all times and in the face of adversity, you pull together and prove what can be accomplished when a group of folks work together toward a common goal. Well played, WooTeam!
wOOO!
Glad you guys are back, hope you’ll give us WP users insight into how/what happened, and how we can prevent. We were hacked as well, about 3 weeks ago and two of our clients. Sigh. Keep up the good Woork! 😛
Glad to have you back! Or almost back – my membership is listed as “Expired” when I lock in..
I presume its a general problem?
Same problem here :S any ideas?
Same problem for me.
Same issues here, despite trying clearing cache as explained etc. http://woocommerce.com/woothemes-server-status/
You guys ROCK! (From our whole team at euphoric MEDIA)
I stay tuned into all of your updates on the progress to getting back up and I have to say you guys have kicked some serious being hacked and getting back online ass!
I feel so much more connected to the whole WO0 Themes company and team now too after seeing the pictures of everyone working hard at headquarters and how straight forward and personal the team was with all of us and how awesome the support was during the whole thing.
I had a client’s theme I needed a backup for and the theme got sent to me within 2 hours of requesting it!!
Thank you and I am sold for life! WOO Themes are the best!
Peace In,
Tie Love
Commendations to the WooThemes Ninjas…everyone is thrilled that such a great company is back on the air.
Double commendations for keeping WooFans up-to-date with your progress and status…it was like hearing that a best friend’s site was hacked–which, in a way, is what happened.
Ninjas Rock!
Glad to see you back online, hope things are getting back to normal for you. Don’t think there is anything that can quite prepare you for a bad hack. I’ve only had one really bad experience and I wouldn’t wish it on anyone. Glad you got things back up and running.
Just wanted to say well done. You all very obviously worked flat out to get this back up and running. Very impressed 🙂
We’ve been converting all of our clients over to WPEngine and it’s been a great experience all around! Welcome! And kudos to WPEngine for reaching out.
Woo Hoo, Great to see you guys are back! I was very concerned if hacking was due to the poor software or scripts recently developed. Particularly WooCommerce which we are using for many of our clients. However, despite of the fact Woo Ninjas were busy getting woo back up they responded to my concerns and I thank them for Heroic Support! They also confirmed that WooCommerce is well built and there is nothing to worry about! Thanks guys and again glad to have you guys back!
The blog looks great. Is the affiliate program going to be fixed? My old affiliate link turns out isn’t working now.
Not that you would have thrown in the towel, but I truly appreciate the outstanding effort Woo put forth to get everything right side up. Thanx so much for being open and honest and still providing assistance throughout this ordeal. Congrats on your new server home at WPEngine. From my handful of dealings with them, they’re a decent lot; I think you’re in good hands.
Welcome back!
Impressive. So glad you came through this. Didn’t know what I could do in my little corner of the world. so I sat and hoped for the best.
Just passing along an exploit to your WooFramework I caught elsewhere on the interwebz:
https://gist.github.com/2523147
Looks like you guys just recovered from something major; sorry to have to be the bearer of more bad news.
Eeeek!
Hopefully after the Wooteam gets some sleep they can fix this ASAP.
That has already patched in the latest version of the framework in all our themes.
Do you have any information on the attack that you can share. I’ve seen quite a few forums and other websites being hit this weekend and don’t know if it’s a coordinated attack or not.
I think it was an isolated incident, but we are still investigating the cause/culprit.
Welcome back! You don’t know how much you’re gonna miss someone til they’re gone!
Congratulations WooThemes returned. It seems that information about the affiliate program has not been restored, all previous income is lost and no signs of recovery 🙁
Glad you guys managed to get back online. Looking forward to reading any details you’re willing to share on the server compromise in a future post. Keep on truckin!
Hi
Excited to have you back, but I still can’t access any pages, beside this one…
I cleared and ( force refreshed ) my browser cache, and also visited from another computer ( same internet connection ).
I stil get a 403 Frobidden error: nginx/0.7.65
However, when visiting from my iphone ( Vodacom cennection ) it works!
Do I simply have to wait for my ISP (telkom) to get up to date, or can i do something about it.
Regards
Welcome back guys. I suppose you’re working on the broken affiliate links? Please. Thanks.
Welcome back, Woo – we waited for this time so long
Fantastic comeback Woo! I immediately noticed a speed increase on the new site. Well done.
Good advert for Coke 😉 welcome back guys.
Congrats guys, glad to see you moving over to WP Engine. VPS.net should not be in business; after 30 days with them I had 30 support tickets open. And with over 20 complaints at the BBB, they should not be in business. You’re in good hands with WP Engine.
I’m curious though, do you guys not store off-site backups and why couldn’t you recover with those?
Nick
Happy to see you managed to pull it off. A reminder for everybody at the same time. Ouch!
I think you’re crazy if you don’t check out CloudFlare. They prevent DDOS attacks like this and it actually makes me nervous that you’re trying to do it on your own.
Why would someone do this?? Massive well done to you guys for sorting it out. Sounds like a mammoth effort!
Rock On Ninja Styleee
Hi,
I just received what seems to be a newsletter from Adii with some dodgy URIs:
http://woothemes.createsend3.com/t/y-l-jlnyhd-dhhhjjdjh-p/
Is this legit or phish?
Morten
That’s all well and good but where’s Thursday’s drop? haha, just teasing. Glad you are back online!
Glad to hear you guys made it through. Feels good to come out the other end of something like this I bet.
Great to see you all safe and sound !
Welcome back guys.
I just wanted to say that I check Woothemes out a lot. The speed, transparency, and professionalism that you guys put into this recovery is really inspiring and in my opinion it will generate more memberships and interest for you.
I certainly will be using only Woothemes for my future projects, because I know I can trust you not to let me down.
Thank you.
offsite backups of your database are essential.
Is updating within the theme menu not possible? I see
Framework Update
You have the latest version of WooFramework
→ Your version: 4.4.3
That does not look good…
Glad to hear you’re back 🙂
Love you all
Welcome back.!!! I love your themes 😀
so how secure is your product WOO-COmmerce..
No offsite Backup? Ouch. Even Google does tape backups of everything. -Yes tape. How did you manage to piece it all back together? 🙂
We did have offsite backup I believe, rsynced between locations. They found that too. 🙁
Bravo to all the team !
Take care against new attack and go straight 😉
I hope your woo-commerce theme will improve against attack soon and kept safe ?
He got me on April 20th. Changed my wp-config on 20+ sites and put a photo of a dead baby-saying why do you kill children…
Solution: find him and attached vice grips to all his appendages.
It’s constantly shocking and appalling that this sort of thing happens.
But what’s even more shocking (and AMAZING, I should add) is how fast you got it back together considering the scale of devastation that was wrought!
I’m truly astounded and really glad, both for selfish reasons (I bought a new package just days before Woo went down (no causal relationship, I assure you!)), and in general, because I love Woo and I know it to be a great company composed of really hard working and passionate people.
Having seen what happened to a pretty cutting edge “tech” firm (you… Woo), there’s a lesson to be learned here by all of us.
I’ve been looking at WPEngine myself. One thing everybody can do to protect their investment in time and energy, even if they can’t afford a premium host like WPEngine, is to make absolutely sure everything is backed up offsite everyday (preferably automatically). It’s invaluable insurance! I’m using blogVault.net and really like it (they have a 30 day free trial, too), but there are quite a number of options available insofar as WP back goes. Even WP itself is offering a plan these days…
Anyway, best of luck on the rest of the resurrection, Woo ninjas. My kudos to you all. It’s truly amazing what you have done in such a short period of time. Many lesser firms would have just been… DONE! Fini. Stick a sword in it.
Good job!
🙂
You guys are amazing!
Have a much better week!
fyi protection against ddos:
http://blog.cloudflare.com/cloudflarewebops-for-everyone
alas my site and three others that I know of are still down as other hosts attempt to get it sorted
THANK YOU GUYS! YOU ARE AWESOME!
Congratulation, Woothemes…
what doesn’t kill you makes you stronger…
🙂
anyway…
1) Do you know who is responsible in doing this to your team ?
2) Why do this person does this bad thing to you ?
Hi,
While I am also very impressed with the way you handles things, I also like to urge you to respond to my “expired subscription”. I have emailed the support@w and used the panic button contact form, but still no response.
Even if you can add a few themes to my Dropbox, please private message me and I will let you know which ones.
Hope to be fully alive and kicking soon!
Regards
s
I would like to purchase a theme, but your website is not working correctly.
Nothing happens when I click the Buy It Now button and your ‘test a theme before you by it’ leads me to a blank page. When will you be accepting new accounts?
Which theme are you trying to purchase? Send us a mail through our contact form and we will help you out!
Wow, that sucks being hacked, it was probably a competitor!
But welcome back!
You are awasome. Good job !
I have move all my client sites to WP Engine almost a year ago and I have to confirm that the experience on all levels has been absolutely amazing.
Good to konw about you. You guys are really awesome!
I really need to get myself a plush doll!