Email Event Data with Keen
The SendGrid Event Webhook lets you stream all email events directly to Keen for nearly real-time analysis and long term storage.
It only takes a couple of minutes to start streaming email event data from SendGrid to Keen, and once you do, you'll get tools to create flexible email analytics and stunning email marketing dashboards.
Before continuing with this guide, you should first create SendGrid and Keen accounts if you do not have them already.
- Once you're logged into Keen, add a new project in the Keen application user interface (UI) by clicking New Project in the left sidebar.
- A new page will load where you can name your project. Add a name and click Add Project.
- Click Access in the left sidebar to load a page with your project's information.
- Copy the Project ID and Write Key values from this page.
- Once you have your Project ID and Write Key, you can construct a URL where SendGrid will deliver event data. That URL follows the pattern:
https://api.keen.io/3.0/projects/<YOUR_KEEN_PROJECT_ID>/email/sendgrid/1.0?api_key=<YOUR_KEEN_WRITE_KEY>
- Navigate to the SendGrid application to configure an Event Webhook.
Whether starting with a new webhook or editing an existing one, you will set a Post URL, which is where you'll add your Keen URL.
- To create a new webhook, see the "Add an Event Webhook" section of Getting Started with the Event Webhook.
- To modify one of your existing webhooks, see the "Edit an Event Webhook" section of Twilio SendGrid Event Webhook Overview.
Once your webhook's Post URL is set by following the linked directions above, you will be able to click Test Your Integration in the SendGrid webhook dialog to send a test payload to Keen. The sample email data from SendGrid is an example and not data related to your actual email send.
Once you send a test payload from SendGrid, navigate to the Keen application to see the sample data.
- In the Keen application, navigate to the Keen project you created earlier.
- Click Streams in the left sidebar menu.
- In the box labeled Event Streams, you will see events such as email_bounce and email_click. You can click on any of these events to display the event data in JSON format. See the "Data Models" section of this page for more information about how event data is represented in Keen.
- After you've confirmed the data was sent, enable the Event Webhook in the SendGrid application and your email data will be sent to Keen.
Each email event type that is selected will be recorded as a separate Event Collection in your Keen project. We recommend sending all event types from SendGrid to Keen to access the full functionality of Keen's reporting.
The email events that are recorded in Keen are:
email_bounceemail_clickemail_deferredemail_deliveredemail_droppedemail_group_resubscribeemail_group_unsubscribeemail_openemail_processedemail_spamreportemail_unsubscribe
As SendGrid email data streams to Keen, it will be enhanced using Keen's Data Enrichment capabilities. Keen's data enrichment turn IP addresses into geographic locations and break out timestamps, URLs, and user agent strings into data such as device_type, day_of_the_week, and URL path.
Info
Use the property message_id or singlesend_name contained in each event to track a particular email across all of these possible actions. Similarly, you can follow a particular user's behavior via the email property.
With Keen, you will have access to all of your raw data for as long as you need it. An archive of all of your email data is created, which means you will have access to your historical data and can query for past user behavior.
Just like SendGrid, Keen is 100 percent powered by APIs. This means you can embed rich analytics anywhere you can write code.
The following resources provide more information about working with your data once it's in the Keen platform. See the "Additional Resources" section of this page for more SendGrid and Keen documentation.
- See Keen's "Data Explorer Guide" page to learn more about analyzing your data.
- See Keen's "Customer Facing Analytics" page for more information about embedding your SendGrid analytics into your product.
- See Keen's "Streaming to Amazon S3" page to write a copy of all their data to AWS.
- SendGrid: Getting Started with the Event Webhook
- SendGrid: Event Webhook Overview
- SendGrid: Event Webhook Reference
- SendGrid: Event Webhook API documentation
- Keen: Data Viz Library.
- Keen: How to Build a SendGrid Email Marketing Dashboard
- Keen: 3-part video series on setting up the integration, creating email metrics, and building an email marketing dashboard.
- Keen: API documentation
