[ Sponsored Links ]

Advertise here »

Multivariate Testing and A/B Split Testing - who provides it?

We're seeing growing interest from site owners in doing A/B split testing and multivariate / multivariable testing - for landing pages, for product pages, for buying processes, for e-mails and so on.

But who are the service providers out there specialising in offering such solutions?

We (E-consultancy) are looking at doing some specific content and events (roundtables, training and so on) around the topic of testing and optimisation as we see it as an enormously important topic going forwards.

At this stage we’d be interested to know whether you think the topic is of interest, and also who you know out there that specializes in this area? Have you got any experiences, recommendations or suggestions to share? Feel free to put yourself forwards if you are a supplier / agency!

Following is a list of companies currently on our radar:

Specific multivariate / multivariable / split testing tools and services:

Site conversion optimisation solutions (typically continuous learning approach):

(NB I’ve left out a whole load of online advertising optimisation solutions, including the behavioural targeting crowd here – in theory these can of course be used to serve and optimize content on your own site, not just your ads on other sites.

I’ve also left out a number of optimisation solutions that are largely focused around search engine marketing e.g. landing page optimisation analytics built into a bid management tool. The above multivariate testing tools will often be used for search marketing optimisation of course.)

Any thoughts, suggestions?

Ashley Friedlein
CEO
E-consultancy.com



Add to Del.icio.us Add to Del.icio.us  |  Digg this Digg this  |  Add to redditAdd to reddit  |  Add to Y! MyWeb Add to Y! MyWeb
Comments (15)
Posted 18:20 15 May 2007 by Ashley Friedlein
Filed under: Web Measurement and Analytics, E-commerce, Site Monitoring & Testing, testing, optimisation, conversion, multivariate, A/B, multivariable

Mobile phone iconWant the latest headlines for your commute? Browse from your mobile: www.econsultancy.mobi

Reader Comments

Add a comment »
 
1. I would definitly welcome training and events geared towards this form of testing and optimisation. I have previously come across a useful <a href="http://www.marketingexperiments.com/improving-website-conversion/multivariable-testing.html"> on multi-variable testing</a> which provides comprehensive information and links to both providers and other articles on this subject.

It appears you have already identified most of the software and services provided, but it may provide some more useful articles and findings which you can use when you start pulling together your training/roundtables etc.

In my experience at Littlewoods Shop Direct, for the few times when it was feasible to carry out A/B testing it proved very worthwhile and allowed the business to make key commercial decisions as to what provides the most optimal user experience.

Finally I totally agree that multivariate and A/B testing will be a key method of testing and optimisation in the future, along with other usability testing techniques such as eye tracking and user testing.
Posted 09:42 16 May 2007 by Paul Rouke
2. Thanks for the coverage and also Paul, glad to see you feel this will be useful in the future, we are seeing huge growth in interest currently as more and more people come towards fully exploiting other traffic driving and retention methods.

Just a quick note to add that Maxmyiser also offers Multivariate, split and A/B testing tools separately to our optimisation consulting, we can service all types and sizes of clients.

"I work for this company"
Posted 12:00 17 May 2007 by Alasdair Bailey
3. Please add Amadesa to the mix. We are a business optimization solution with focus on the ecommerce, lead generation and subscription-based markets. Our combination of behavioral targeting, a/b and multivariate testing, analytics and user interface design alterations is unique in the industry, and enables us to support a wide variety of client-specific needs (product & price testing, landing page optimizations, checkout/billing path improvements to name a few).

It's great to see movement forward on round table type discussions. Are you considering webinars, events, tradeshows, etc? I'd be more than happy to help where I can.

Pete Olson
VP Marketing
Amadesa, Inc.
Posted 15:34 17 May 2007 by Pete Olson
4. We don't specialise in A/B Testing, but can't resist a quick plug (our tool offers quite "accessible" A/B Testing from £2.50/month, so can't complain!).

We've also recenly 'eaten our own dog food' and run a series of A/B Tests on the clickdensity (http://www.clickdensity.com) site, which I'll be writing a case study on soon.

I'd personally be really interested in any content/events that you create on this topic.
Posted 16:18 17 May 2007 by Daniel Zambonini
5. We at SiteSpect embrace formalized training, and believe that discussion in the area of testing and optimization is a wonderful idea. Along those lines, an important consideration is inclusion of data from multiple sources – not just web sites, but email campaigns, PPC/banner campaigns, fulfillment systems, etc. Only through this vantage point can Web marketers ensure optimal performance across each segment, and at each point in the customer lifecycle. Seamlessly integrating these data sources is key to understanding the overall impact of multivariate tests, and ultimately to maximizing the ROI of optimization efforts. We would be happy to participate in training and discussion on these topics.
Posted 17:53 18 May 2007 by Rob
6. Memetrics has been focused on applying best practice multi-variate testing and optimization methods for over 8 years now. It is great to see the market embrace an approach that we have seen deliver consistent value.

It is worth considering that the topic is actually much broader than just site side optimization. Inherently, all digital media (and digitally enabled customer processes) can benefit from the same testing and optimization process applied to sites. We have run successful programs across site, e-mail, direct mail, telemarketing, SEM, on-line media etc.

It is also worth noting that while technology is a critical factor for success, equally important are the marketing methods used to drive the capability.

Digital media provides rich opportunities to improve marketing effectiveness and gain actionable customer insights.

We'd be delighted to share experiences and lessons learned along our optimization journey.

Hikaru Phillips
Posted 05:18 21 May 2007 by Hikaru Phillips
7. Hi Ashley,

Thanks for the question. How about a buyer's guide on multivariate testing providers? There's a real dearth of information when it comes to vendor selection.

thanks!

Andrew
Posted 20:02 8 Aug 2007 by Andrew Janis
8. Hi Andrew

Yes, I think that's definitely something we should do - we might broaden it slightly to 'optimisation technologies/solutions' but certainly something is needed.

Ashley
Posted 09:42 9 Aug 2007 by Ashley Friedlein
9. Please include Sokel Inc. which has released Sokel Choicepoints - a cost-effective solution that enables marketers to run simple A/B or more complex multi-variate tests to understand which of many different combinations of creatives performs best. Sokel Choicepoints also supports learning where creatives are served out in the proportion they convert.

Thanks
Sridhar
Posted 18:20 23 Aug 2007 by Sridhar Ramakrishnan
10. Hi Ashley,

The article was quite informative. I liked it !!

Could you tell us how Fractional Factorial testing is actually implemented ? What are the calculations involved ?

If you could point me to some links, that would do too.

Thanks in advance. :-)

Cheers,
Ashlak.
Posted 13:11 7 Sep 2007 by Ashlak
11. Hi Ashlak (BTW is that Ashley in another language? ;) )

In answer to your question "Could you tell us how Fractional Factorial testing is actually implemented"...

No. No idea. I'm not even sure I understand what Fractional Factorial testing is, let alone how it works.

Sorry!

Ashley
Posted 16:24 7 Sep 2007 by Ashley Friedlein
12. Hello Ashley,

Not a problem... Thanks for the reply, anyway !! :-)

Fractional Factorial testing is the process of testing a "Fraction" of all the "factors" ( "factors" are the elements on a page such as Headline, Image, Promo Text, etc... Could be just about anything ) on page to test what's working and what's NOT !!

Its Primary purpose is to Improve Conversion Rates !!

If anyone else has something to ADD or CORRECT, the floor is yours !! :-)



Cheers,
Ashlak


PS - Ashlak is not a name in any language !! I like it for some reason !! :-)
Posted 06:53 11 Sep 2007 by Ashlak
13. I'd defintitly be interested if you did a round table or show case event, as I think there are a lot of people in a similar position - either just starting to get into it or thinking about it. But I'd also be interested in a session on how it impacts your design process. If anyone could paint a solid picture of what is required to run a wesite based on a continual optimisation process I'd definitely be interested in that too. I guess the vendors are part of it, but there's a fair bit else that I'd be interested in as well...
Posted 12:56 2 Oct 2007 by Lucy Spence
14. Hi Ashley

This sounds like a great idea. Certainly we are finding our clients are now showing an interest in this type of activity particularly when we have been teaming this with the development of landing pages as part of a wider PPC intiative.

As a result we have seen some very significant improvements in CTR from landing pages.

We would welcome more exposure in the market of the benefits of this tyep of solution.

Garry
Posted 10:02 4 Oct 2007 by garry davis
15. Hi Ashley,

Last December we launched an article on multivariate testing that featured on the Alexa.com homepage as a Top 3 'Mover & Shaker', as well as hitting the home pages of Digg & Delicious - interest in conversion optimization is huge!

Our recent presentation at A4UExpo in London was also very well received - so go for it!

Warmest Wishes,

Ben

P.S. Shoot me an email if you want any help, feedback or advice with this.
Posted 15:39 26 Nov 2007 by Ben
Enter your comment below and press submit when complete.














Image used to prevent robots submitting forms.




Latest Internet News Stories

 
Subscribe for only €299