HOW TO TEST AN EMAIL VERIFICATION VENDOR?

The Email Oversight team provides several test accounts each week for marketers who are evaluating multiple email verification platforms. We always encourage potential clients to prepare a comparison level test in order to determine each service’s strengths and weaknesses, but we’ve found that not every email marketer knows the best steps to take to get the most out of the assessment. We decided to create a “best practices”guide to help you test your emails and compare email verification services like a pro.

It’s important to remember the main reason for testing is to see each email verification vendor’s results and how they differ from one another. But you also need to review inaccuracies like False Positives (invalid emails marked as valid) and False Negatives (valid emails marked as invalid).  It’s also essential to understand the output or reasons for certain results, such as Spam Traps or Bot Technology, and realize that not all vendors are identifying or finding these additional threats. Lastly, comparing accuracies and deficiencies will sometimes showcase itself through ‘unknown’ responses, which could indicate a vendor might be lacking with a specific ISP domain. This guide will help simplify your testing experience.

1. Prepping

To get started, try searching for your most recent bounce logs. We’re talking days or weeks old not months or years. Be sure to mix your list to include email addresses from various top-level domains like Yahoo, AOL, Gmail, Hotmail, and Verizon. These emails can account for 50-70% of your total sample list. You may also include international emails in case you’re a global sender.

Be sure to add “fake” emails like mickemouse123456789@yahoo.com, or spamtrap@yahoo.com. Get creative and create some new emails as this will test the vendor’s technology and ability to detect made up email addresses. Next, consider some duplicate emails, simple syntax mistakes (missing @ symbol or .com), disposable emails, and role accounts. Again, this is to help learn about what the service provider can and can’t detect. Lastly, include some emails that have showed engagement, purchases, clicks or opens. This will provide a perspective if a system is over scrubbing or removing valuable email addresses from your database.

2.Volume size

In our system, we allow up to 200 free test credits so if you’re a smaller mailer, 50-200 emails should be sufficient to evaluate your file. However, for Enterprise clients with files larger than 100k, you should definitely consider a larger test size of 10k emails. This allows you a broader range of data and affords you more substance to review.

 3. Processing

Now that your email data is established, be sure that your file is saved in a.CSV format with the proper headers. Go ahead and upload your file and begin processing.  It’s always smart to time the process from start to finish as monitoring speed is important. Check to see what reporting is available for each vendor while you wait for results to be completed. Some vendors report data in real-time as a scan is occurring, while others provide statistics upon completion. In some cases, vendors may not offer any reporting at all. This helps establish credibility with service but also customer friendly service aspects that can assist you during your evaluation. We also would consider running some simple one-off emails in the vendor’s API tools, which will allow you to test the API’s speed and accuracy in real-time. It’s important to note that some verification vendors have different speeds on their API but also different results when using an API vs their batch systems.

4. Analyzing

After the list has completed, you must download the newly verified test list and start to review the results.  It’s crucial to understand each vendor’s terminology and the reason for removal. Each vendor uses different verbiage when defining their results and some vendors may not have as much in the way of detection, so it’s in your best interest to educate yourself on their ‘lingo’. As an example, listed below is a description of the statuses that Email Oversight provides. This is a good time to review your statutes with any known valid, engaged or made up emails where you know the real output and can compare against these results. If you do notice some discrepancies such as false positives or inaccuracies, keep a record of them, and this will allow you the ability to score and grade each vendor’s result.

5. Send A Campaign

If you want to take testing to the next level to find any False Negatives and really see each result’s worth, send an immediate campaign to the verified records and keep track of how many emails actually bounce. This will provide a secondary accuracy score which will help you make your ultimate decision in choosing your new verification provider.

We recommend that you only send to the Verified emails and for B2B mailers, we recommend that you use discretion with the Catch-Alls.

EmailOversight Statuses

• Verified (Code 1): SEND

Email is Valid and will deliver.

• Undeliverable (Code 2): DO NOT SEND

Email is Invalid and undeliverable.

• Catch-All (Code 3): SEND AT YOUR DISCRETION

Email is most likely a B2B address and has a firewall or a spam tool that accepts all emails associated with that domain. It is impossible to determine the validity. If the email address was acquired organically or you have confidence in the validity of the email address, then send it at your discretion.

• Role (Code 4): SEND AT YOUR OWN RISK

Email is associated with common distribution groups. abuse@, sales@, no-reply@, test@ and etc. Several ESP’s like MailChimp will suppress role emails during the import process.

• Malformed (Code 5): DO NOT SEND

The email address does not conform to a valid email format. Missing an @symbol or .com is misspelled.

• Spam Trap (Code 6): DO NOT SEND

Avoid SpamTrap emails at all costs. Those accounts are kept alive to damage the sender reputation.

• Complainer (Code 7): DO NOT SEND

Complainers are common users who like to complain after receiving the email. Complaints can vary from ISP notification to cease and desist letters. In some rare cases, they can also pursue litigation. These are valid addresses but we do not recommend sending.

• Bot (Code 9): DO NOT SEND

Bots are email accounts that are maintained by bot servers for sending spam, clicking every link and other harmful or harmless activities.

• Seed Account: (Code 10): SEND AT YOUR DISCRETION

Seed accounts are known email addresses that are maintained in masses for various reasons. Most commonly they are used for compliance tracking.

• Unknown: (Code 11): DO NOT SEND

Email addresses cannot be verified at the moment. No charge for this status and it’s recommended that you reverify after 72 hours.

• Disposable Email: (Code 13): DO NOT SEND

The temporary email account that is designed to only be used a few times before expiring.