A new user is identified by Google as someone who lands on your site and does not have the Google Analytics cookie in the browser they are using. This could be for a number of reasons:
Returning visitors are identified because they have the GA cookie present, this allows GA to use the Client ID for analysis.
For reporting it is important to remember that visitors can be both new and returning depending on the period chosen. If someone visits on Day 1 and then revisits on Day 2, you would see one new visitor for Day 1 and one returning visitor for Day 2. If we look at a time period that includes both days we would see one new and one returning visitor, but we would only see one “user” for that period.
Sometimes it just seems too hard, you are almost literally hitting your head against a brick wall.
In these cases the best way to convince stakeholders and grab their attention is simply to demonstrate the economic impact achievable by moving down the optimisation path.
A table like this shows the effect on the bottom line for the business.
In simple terms, one-tailed (or one-sided) tests will only tell you if control is equal to the variant or not. If control is not equal to the variant it could mean that control is better than the variant or vice versa. Two-tailed (or two-sided) tests also identify if control is different to the variant, with the addition of whether variant or control is the better performer.
For A/B tests this is particularly important as the lift or otherwise is not known prior to the test starting.