I’ve recently taken the dive into PoF, and after some mildly successful campaigns which quickly burned out, I’ve learned my lesson and started testing many more ads per campaign (50 images, and 4-5 versions of ad copy). The problem I had this weekend, is that after an initial test to cull poor performing ads on Thursday and Friday, (about 1,000-1,500 impressions per ad, less than .1% ctr, gets paused) I was left with 20 ads with ctr at .1% or higher. I let those run for another few thousand impressions each on Saturday night, and when I checked my stats, the ctr had plummeted, so I was left with not a single good performing ad. I paused them all Sunday night out of disappointment, but just for further testing, I ran them again last night and the results were the same. So I have no idea if my initial testing was just a fluke, or too small a sample size, or what.
To make a long story short, my question is: when is the best time/day of week to test new ads/campaigns? I always thought it was during weekend but maybe the US holiday weekend threw things off?
This depends a lot on your targeting in terms of how ads will do CTR wise. Prime hours are usually in the evening/nights and on the weekends if you want to aim for the best CTRs, at least in my experience, but you should be testing until you find ads that do decently well at all times. Again though, it depends on your targeting.
2000 impressions is pretty insufficient for testing so you’re going to get very inconsistent results (ex. ads you think would be good aren’t actually good). 50 images with 4-5 txt is way too many ads to test in the beginning so if you did test them all, you basically tested them all insufficiently Obviously, you’re better off testing 10 sufficiently and actually getting results than to test 200+ with insufficient results.
I wrote a blog post about this, I suggest you read it https://blog.ads.pof.com/2011/09/26/h…s-on-a-budget/
I read it and will comply to this to see how it goes..
Hmm, I did read that blog post a while back, but since then I’ve read some other stuff from a few different sources that has lead me to believe that the best bet is to cull non performing ads after 1-3k impressions. I guess I’m going to have to test your method out as well and see what happens. 20,000 impressions per ad you say? I’ll give it a shot. Thanks. Oh, and I almost forgot.. I wasn’t testing all 200 at once, I just put up the first 50 images with one ad copy to see how they did, and the plan would be to take the top 10 images and then test those with the other versions of ad copy, so my testing is kind of serial-ish
Guys, think about this logically instead of applying rules that people made up. It all depends on the specific campaign.
At lower impressions (less significant data), cutting ads at that point means there’s a higher chance some of those ads might be high performers that you’re getting rid of.
At higher impressions (higher significance), you have more data and are more sure of which ads are high versus poor performers, but you’ve also dragged along a bunch of ultimately poor performers, where, in a perfect world, you would have cut a lot earlier.
So, there’s a trade off. But ad performance isn’t black and white. There are great performers, mediocre performers, slightly poor, etc. Therefore, the goal is to cut the worst performers the earliest, get some more data, and then cut the worst performers of the ones left, and so forth.
This is the "Concept of Hurdles" that I explain in my guide. As more data is collected, your ads are filtered through hurdles. At each level, if certain ads don’t make the hurdle, they get cut. If at the end of the hurdles you set up, you are left with not enough ads, then either 1) your hurdles are too strict, or 2) your ads weren’t good enough.
Let me illustrate with an example.
If you have a batch of 40 ads, 35 of them have CTR of 0.5% at 1,000 impressions and 5 of them have CTR of 0.1%. You bet your ass I’m cutting the 5, even though someone once made up a rule of 1 click at 1k imps you keep the ad.
Particularly if the traffic is relatively expensive, and I have a bunch of ads that are performing well, there’s no way I’m going to wait until 40k imps before deciding if I have enough data. The goal is to spend money as efficiently as possible while testing.
Conversely, if I have a bunch of ads that have <0.1% CTR at 1k imps, but a bunch of them are getting impressions (i.e. high conversion rate), then I will keep them running a while longer before cutting the non converters off.
More data is always better. Without thousands of impressions you’re not going to have any real statistical basis for your ads. Sucks, but that’s the sacrifice for paid traffic. I would definitely listen to Tom’s advice, he really knows his shit. And well, Ben is POF so there you go.
Thanks guys, that’s alot of good info and I’ll be sure to put it into practice right away.