[fusion_builder_container hundred_percent=”no” equal_height_columns=”no” menu_anchor=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” background_color=”” background_image=”” background_position=”center center” background_repeat=”no-repeat” fade=”no” background_parallax=”none” parallax_speed=”0.3″ video_mp4=”” video_webm=”” video_ogv=”” video_url=”” video_aspect_ratio=”16:9″ video_loop=”yes” video_mute=”yes” overlay_color=”” video_preview_image=”” border_size=”” border_color=”” border_style=”solid” padding_top=”” padding_bottom=”” padding_left=”” padding_right=””][fusion_builder_row][fusion_builder_column type=”1_1″ layout=”1_1″ background_position=”left top” background_color=”” border_size=”” border_color=”” border_style=”solid” border_position=”all” spacing=”yes” background_image=”” background_repeat=”no-repeat” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” margin_top=”0px” margin_bottom=”0px” class=”” id=”” animation_type=”” animation_speed=”0.3″ animation_direction=”left” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” center_content=”no” last=”no” min_height=”” hover_type=”none” link=””][fusion_text]
Simply put, the ability to perform A/B testing in an automated fashion is one of the most strategic differentiators that DevOps organizations possess over their competitors. A/B testing is also one of the least known concepts among organizations looking at possibly attempting or considering DevOps.
What is A/B Testing?
The basic concept of A/B testing is that it is a way to compare two versions of something to figure out which performs better. A/B Testing is most often associated with websites and mobile apps, the reality is that A/B Testing has been around for almost 100 years. A/B Testing was introduced by a 1920s statistician and biologist named Ronald Fisher who is credited with discovering the key principles behind A/B testing and randomized controlled experiments. Others ran these experiments before Fischer, but he is the one who figured out the basic principles and mathematics that made it a science.
How does A/B Testing Work?
To use A/B Testing you first have to determine what you wish to test. An example would be whether users prefer clicking a button labeled Send or a link text labeled Send. Then using one of the DevOps release strategies to put the experiment into production or maybe an internal user’s environment where you would then evaluate the performance of each. In this example, we want to see which option visitors to a web site click most often. You run the test by selecting two sets of randomized users and then showing them based on random selection either the button or link. From the analytical data, we determine which option users preferred. It is important to remember that other factors could influence the results, so this is why randomizing which users are in which group, you minimize the chances that other factors will skew your results.
What Does A/B Testing Report?
In most organizations, some form of software will be used to automatically handle the calculations and often there will be a statistician who handles interpreting the results. A/B Tests will generally report out two conversion rates, the first for users who saw the control version, and the other for users who saw the test version. Typical conversion rates measure clicks and other user actions.
Mistakes Made in A/B Testing
The main error made when doing A/B Testing is being too eager and not letting the tests run for a sufficient length. Many of the A/B Testing software products actually let organizations watch the test results in real time, which leads to the urge to make decisions too quickly. This over eagerness has been feed by the A/B software vendors who have offered a type of A/B testing called real-time optimization, which allows organizations to use algorithms to make adjustments as the results come in. This causes a problem in that, due to the randomization, it highly possible that if you let the test run its full length, you quite possibly could get a different result.[/fusion_text][fusion_content_boxes layout=”icon-on-top” columns=”1″ title_size=”” title_color=”” body_color=”” backgroundcolor=”” icon=”fa-trophy” iconflip=”” iconrotate=”” iconspin=”no” iconcolor=”#ffffff” icon_circle=”” icon_circle_radius=”” circlecolor=”#8bc34a” circlebordersize=”0″ circlebordercolor=”” outercirclebordersize=”” outercirclebordercolor=”” icon_size=”35″ icon_hover_type=”” hover_accent_color=”” image=”” image_width=”” image_height=”” link_type=”” link_area=”” link_target=”” icon_align=”left” animation_type=”” animation_delay=”” animation_offset=”” animation_direction=”left” animation_speed=”0.3″ margin_top=”” margin_bottom=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””][fusion_content_box title=”Top 5″ backgroundcolor=”#efefef” icon=”fa-trophy” iconflip=”” iconrotate=”” iconspin=”” iconcolor=”” circlecolor=”” circlebordersize=”” circlebordercolor=”” outercirclebordersize=”” outercirclebordercolor=”” image=”” image_width=”35″ image_height=”35″ link=”” linktext=”Read More” link_target=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=””]
A/B Testing
Google Analytics Experiments
Kissmetrics
Adobe Target
VWO
Unbounce[/fusion_content_box][/fusion_content_boxes][/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]