Payment Errors

Authorization and Capture Errors

Some ISPs or mail readers, such as Hotmail or Outlook, will display a message with the Sender header set as “on behalf of” and then the address that is in the Sender header. This is caused by the DomainKeys configuration of your account.

By default, SendGrid signs all e-mail with DomainKeys to improve deliverability with some smaller domains. However, with accounts that do not have whitelabeling, this means that the domain in the From field will not have the proper DNS record to authenticate the e-mail. Because of this, SendGrid inserts a Sender header, and the e-mail can then be properly authenticated.

However, the vast majority of recipient servers now check authenticity using DKIM, which is the evolution of Domain Keys. All emails sent out by SendGrid are signed with DKIM, whether your account is whitelabeled or not. So if you’re having this problem, feel free to disable DomainKeys, and it should not have any impact on your deliverability. You can disable them by going to your Apps, here.

Section 10.1 of the SPF RFC, “Processing Limits”, specifies the following in regards to DNS lookups:

SPF implementations MUST limit the number of mechanisms and modifiers that do DNS lookups to at most 10 per SPF check, including any lookups caused by the use of the “include” mechanism or the “redirect” modifier. If this number is exceeded during a check, a PermError MUST be returned. The “include”, “a”, “mx”, “ptr”, and “exists” mechanisms as well as the “redirect” modifier do count against this limit. The “all”, “ip4″, and “ip6″ mechanisms do not require DNS lookups and therefore do not count against this limit. The “exp” modifier does not count against this limit because the DNS lookup to fetch the explanation string occurs after the SPF record has been evaluated.

Is your SPF record validating? You might want to find out here: SPF Record Testing Tools.

First of all, rest assured that our system is designed with throttling (also known as ‘deferral’; these terms mean the same thing in the email world, and may be used interchangeably) in mind. We continue to attempt to send messages for up to 72 hours until the message is delivered. Normally, it takes between a few minutes to a few hours until the message is delivered.

But what is throttling? Well, it’s when a receiving server refuses a message, but with a specific explanation that tells our server it’s only a temporary refusal. They usually say things like “user is receiving mail at too great a rate right now, please try again later”, or “user’s mailbox is over quota”.
One important thing to note is that these errors are almost always a 400 class of error.

What can cause a deferral? A number of things:

• The receiving server doesn’t recognize your IP, and is afraid you’re sending spam.
  This is why warming up is so important, so they know who you are.
• The receiving server doesn’t have any open ports to receive email.
• The recipient’s mailbox is full.
• Some recipients have marked your mail as spam, but not enough for the server to block you. Instead, they will refuse to receive more until they see what the rest of your current recipients do.

What happens if it’s continuously deferred for 72 hours? Well, that generally means it’s an issue like the account being inactive, having a full mailbox, or some other legitimate problem with the recipient. When we are unable to deliver a message after this time period, we convert these deferrals (also known as Soft Bounces on the Email List) to “Hard Bounces”. When an address is hard bounced either due to this, or because we got a different class of error (such as “this user does not exist”), we add this address to the Hard Bounce email list.

What this means is that even if you send a message through us for that user, we will not even try to deliver to that address, because we know it’s no longer good. Continuing to try to send to a known bad address will harm your reputation with the receiver, so we prevent that. However, if you continue to try to send, and we intercept it, your reputation with us goes down, as noted on your Dashboard Account Overview.

To recap:

Deferrals, aka Throttling, aka Soft Bounces are a normal part of email delivery, one SendGrid knows how to handle.
There are a number of mundane reasons that can cause a deferral.
If a message stays throttled, it will be added to the Hard Bounce Email List.

SPF and Whitelabel

As part of the whitelabel process you must create a subdomain (such as email.example.com) which is used for click and open tracking and is used in the Return-Path email header. SPF uses the value in the Return-Path header for the DNS lookup to determine the permitted senders for the domain.

If you have an SPF record set for your root domain (i.e. example.com), you must add include:sendgrid.net before the all selector of this record. If you do not have an SPF record for your domain you must create the SPF record with the value:

<div>::CODECOLORER_BLOCK_2::</div>

Don’t Exceed Ten DNS Lookups!

Many people (ourselves included at one point) may not realize it, but the Sender Policy Framework (SPF) specification has a limit on the number of DNS lookups (10) required to fully resolve an SPF record. One typically quickly exceeds this limit through the reckless use of the include modifier.

Section 10.1, “Processing Limits” of the SPF RFC specifies the following in regards to DNS lookups:

SPF implementations MUST limit the number of mechanisms and modifiers that do DNS lookups to at most 10 per SPF check, including any lookups caused by the use of the “include” mechanism or the “redirect” modifier.  If this number is exceeded during a check, a PermError MUST be returned.  The “include”, “a”, “mx”, “ptr”, and “exists” mechanisms as well as the “redirect” modifier do count against this limit.  The “all”, “ip4″, and “ip6″ mechanisms do not require DNS lookups and therefore do not count against this limit. The “exp” modifier does not count against this limit because the DNS lookup to fetch the explanation string occurs after the SPF record has been evaluated.

This limit is in place to prevent SPF lookups from being a useful avenue for Denial of Service attacks.

Is your SPF record validating?  You might want to find out here.

The email will look similar to this image below.

IP

This is your dedicated IP which is having the problem.

Usernames

These are the usernames which are associated with that IP.

UserIDs

These is the ID that corresponds to each username.

Status
Displayed here are the aggregate results of the spam filtering applied to all messages sent by the IP during the given activity period.  No spam filter is perfect and, in particular, this information is meant to be only one data point that helps paint a picture, not be a final judgment that the traffic was truly spam or not.  The following table defines the colors in terms of the percent of time that a “spam” verdict is rendered on a message.  Please note that one message to ten recipients counts as ten spam/not spam verdicts, not one.

The percentage range for the yellow designation may seem large but is actually fairly small in terms of the number of IPs that fall into this range relative to the other two.  Unfortunately, since SNDS is available to anyone who can prove they own an IP range, this is a case where we must be careful not to provide too much data that might assist spammers.  One trick however, when viewing data for a number of IPs, is that it can often be enlightening to consider the non-yellow IPs: if they’re green, the yellow results are most likely very close to the 10% end.  Similarly, if the majority of the other IPs are red, the yellows probably represent results near 90%.  The same technique can be applied when looking at one IP’s history.

Please keep in mind that this result doesn’t directly represent deliveries to users’ inboxes or “Junk e-mail” folders.  Settings controlled by each user might rescue some legitimate traffic from being put in the “Junk e-mail” folder, or conversely, might treat other messages more harshly.  It doesn’t take into account messages that might have been caught but weren’t because they were on a user’s safelist, for example.

Trap hits
Displays the number of messages sent to “trap accounts”.  Trap accounts are accounts maintained by Windows Live Hotmail that don’t solicit any mail.  Thus any messages sent to trap accounts are very likely to be spam.  Well-behaved senders will hit very few such accounts because they’re generally sending to people who give them their address and because they collect and process their NDRs.  Spammers have a much harder time avoiding them because, in general, they can’t and don’t do either of those good practices.

RCPT
This is the number of RCPT commands sent by the IP during the time period in question.  RCPT commands are part of the SMTP protocol used to send mail, specifically that which specifies one’s intent to send mail to the provided recipient.  That is, the command “RCPT TO:<example@hotmail.com>” requests Windows Live Hotmail’s servers to respond with whether it will accept mail for example@hotmail.com, information which is invaluable to spammers trying to compile recipient lists for future spamming.  For reference, more than a third of IPs sending mail to Windows Live Hotmail keep the fraction of RCPT commands that don’t result in message recipients under 10% and that is a good benchmark to measure against.

Blocked

IPs which are blocked from sending mail into Hotmail.  Attempts to send mail to Hotmail’s mail servers from these IPs will result in consistent refusal, however use of the Hotmail web user interface may not be affected–it is controlled separately.  The reason or source of the block is provided, along with more specific information about it.  To see about unblocking an IP address, please go to the main Postmaster site and follow the instructions there.

Block reason

The reason why the IP was blocked by Hotmail.

More information can be found here.

Some other things to consider:

• DKIM is an internet standards track standard and obsoletes DomainKeys
• If you turn off DomainKeys we’ll still sign the email with DKIM which is not affected by the on behalf of issue, and is designed for the type of third-party signing we do.
• An SPF record is invalid if it requires more than 10 DNS lookups
• An SPF record is invalid if you have more than one SPF record for a given domain. In this case you will want to merge the additional SPF records into one SPF record.

[1] DKIM Frequently Asked Questions

MailScanner, a popular spam filtering application will often complain of possible fraud when it receives emails that meet the following conditions:

• Click tracking is enabled in your account, and link hrefs are being rewritten so that SendGrid can track the clicks.
• The Link Text of your anchor (<a>) link contains a website address (URL)

Example:

When you have Click Tracking enabled in your SendGrid account, a link such as the following: 

&lt;a href="www.example.com"&gt;<strong>Link_Text</strong>&lt;/a&gt;

is rewritten to something along the lines of:

&lt;a href="uxxxx.sendgrid.org/wf/click?c=ABUNCHOFENCODEDDATAFOLLOWS...&gt;<strong>Link_Text</strong>&lt;/a&gt;

If Link_Text is a URL such as www.example.com, Mailscanner will complain because www.example.com (or more specifically the domain example.com) does not match sendgrid.org specified in the href.

Resolution:

• Disable click tracking in your account
• Or, since most customers will want link tracking turned on
• Make sure your Link_Text does not contain a web address (URL)

See this post for more information.

A low percentage of requests versus delivered emails in your email statistics page (as shown in the image to the right) is typically caused by one of the following deliverability issues:

Dropped Email

The most common problem is sending to email addres(ses) that have landed in one of your SendGrid-managed Email List, you can read more about the Email Lists and how it affects delivery here. The bounce list is most often the culprit and SendGrid automatically drops emails sent to addresses that are on your SendGrid-managed email lists. Check your repeat bounces statistic and consider either removing these emails from your email list or the SendGrid bounce table.

Deferred or Rate-Limited Email

1. You have been assigned a new IP address and are sending new email traffic to the ISPs. In this situation ISPs have not had a chance to characterize your sending reputation for your IP and email traffic and will often temporarily defer your delivery. Typically it takes 2-4 weeks for ISPs to properly characterize your traffic, during which time you may experience some deliverability problems. You should consider warming up your new IP address and gradually move your traffic over to the SendGrid Platform.
2. You have changed your sending behavior and are sending a significantly higher volume of email than you have in the past without a proper warm-up process. ISPs will routinely defer email that shows abrupt changes in sending patterns.
3. Your are sending a large amount of email in a short period of time. SendGrid is built to scale and meet your email demands. We will take email from you as fast as you can send it to us, and will send it to the ISPs as fast as they’ll take it from us. Many ISPs impose IP based rate-limiting on email traffic. Going over this threshold will cause temporary deferrals on your email stream.