Hi! I’m Brad!

I’m an award-winning software developer from Laramie, Wyoming.

Category: PHP

  • Creating an Invisible Application: Adding email as an interface for an application

    This post is designed to serve as a brief technical overview of a recent feature added to ServiceSpark, a community service management platform I develop as a volunteer for the United Way of Albany County.

    ServiceSpark uses email to send email notifications to volunteers about new events and new comments on events that the volunteer is connected to.  The email includes a link back to ServiceSpark.org, and encourages the user to RSVP and comment on the event.  Unfortunately, however, this requires a click, a login, and users rarely follow through with the process.

    The Challenge

    Use email as an interface for ServiceSpark, allowing users to “reply” to an email to leave comments, or allow users to RSVP using their client’s native calendar support.

    The Implementation

    My implementation of this required some sort of way to generate unique reply email addresses for each email that was sent, and a way to make note of the reply address, so that replies, if any, can be processed.

    Dealing with the reply is also problematic.  Many replies include the chain of emails behind the reply, or signatures.  These artifacts need to be stripped from the application, or else the comments will become cluttered.

    Emails also need to be attached to the user’s ServiceSpark identity.  Every message should come in and appear as if the volunteer logged into ServiceSpark and created the comment, or submitted an RSVP manually.

    But, finding a way to receive email at any possible address seemed challenging. For starters, standardizing a way to communicate with an email platform is difficult.  Hijacking qmail, or some other mail queue is just tedious and feels like a kludge.

    The Tools

    Receiving emails turned out to be easy with Mandrill.  Mandrill has an “incoming message API” that allows for your application to receive email via webhooks.

    Briefly, this is how Mandrill works

    1. Set up a custom reply domain.  This is a DNS MX entry that will set Mandrill as handler for a custom domain.  This takes about 5 minutes.
    2. Set up a route from your Mandrill dashboard.  This maps incoming messages to a webhook on your server.  For mine, I set up a wildcard, so that all addresses get sent to the webhook.
    3. Emails received by Mandrill will be pushed as a JSON object to your server in nearly real time.

    Since I’m using CakePHP, which is MVC, I set up a controller that is a singularized endpoint for all incoming webhooks.  A token is used as a shared secret for the application and Mandrill.

    1. Application receives a POST at /webhooks/incoming/<token>
    2. Application fires an event `Webhook.Incoming.<token>`
    3. An event handler is set to listen to `Webhook.Incoming.<token>` and parse the incoming data.

    Once the webserver was receiving messages from Mandrill, I began work to parse the messages.  Dealing with the “junk” in email, like signatures, and threads was a huge requirement.  Posting this information publicly would clutter the application greatly, and annoy users.

    GitHub to the rescue–literally!  GitHub wrote an email reply parsing library, and the library has been ported to PHP.  Including this library and parsing the text from the Mandrill request was trivial.

    Mapping the incoming email address to a specific action is accomplished by a database table that has fields for GUID, user_id, event_type, and event_data.

    So Far

    1. Emails are generated with special GUID@myreplydomain.org Reply-to addresses.
    2. GUIDS and the corresponding event information are saved to the database.
    3. User receives an email.  If they would like to respond, they may do so using their email client.  Replies go to <guid>@myreplydomain.org.
    4. Emails are received by Mandrill, parsed and sent as a JSON object to a webhook on my web server.
    5. The server looks up the guid and processes the email appropriately.

    At this point, email to comment is working beautifully.  The next challenge is sending valid meeting requests and processing the responses into the application.

    Standards are your friend.

    The global standard for calendar data exchange is ICS aka ICAL.  This text format specifies events, and recipients, and metadata to allow loosely-coupled applications to synchronize state (like meeting cancellations).

    There is a protocol to using ICS/ICAL to exchange event information.

    ICS/ICAL crash course

    1. Lines are allowed to be 75 characters long.  If your line needs to wrap, the next line should start with a space.  Ideally, you should construct your file, and then wrap the lines one at a time at the end.
    2. ICS files start with BEGIN:VCALENDAR and end with END:VCALENDAR (calendar boundaries)
    3. Key VCALENDAR fields are
      1. PRODID: a string describing application vendor and application that generated the ICS file.  The format is -//vendor/product//LANGUAGE
      2. METHOD: a string describing the nature of the ICS/ICAL file.  Common values include PUBLISH (for publishing event information), REQUEST (for requesting an RSVP), CANCEL (for cancelling an event)
      3. VERSION: the version of the ICS/ICAL standard used.  This is commonly just 2.0.
    4. Events lie within calendar boundaries.  The event boundaries are BEGIN:VEVENT and END:VEVENT
    5. Key VEVENT fields are
      1. SUMMARY: a title for your event
      2. DESCRIPTION: descriptive text about your event.  Newlines should be replaced with the literal string “\n”
      3. DTSTART: the start of the event, ideally in UTC
      4. DTEND; the end of the event, ideally in UTC
      5. DTSTAMP: the time the ICS/ICAL file was generated
      6. UID: a unique identifier that can be used to reference the event in subsequent event updates or cancellations.  This can be anything (URL, GUID, SHA-256 hash), but you need to record it, or you’re going to bungle the entire protocol.
      7. ATTENDEE: encoded metadata describing the recipient’s relation to the event. Subfields include…
        1. RSVP: true or false, depending on whether you want the recipient to respond (Google and Outlook will not show RSVP buttons without this)
        2. CN: the name of the recipient
        3. MAILTO: the email address of the recipient
      8. LOCATION: a string describing the location of the event
      9. URL: an absolute URL for the event. Subfields include…
        1. VALUE: the actual URL for the event

    Example ICS/ICAL meeting invite file

    ATTENDEE;RSVP=TRUE;CN=Barack Obama:MAILTO:potus@whitehouse.gov
    SUMMARY:Testing Markdown
    DESCRIPTION:#### Overview\nThis should print as markdown in emails.\n\n###
     # Subtext\nCurabitur cursus purus vestibulum\, venenatis nisi eu\, element
     um tortor. Sed at fringilla dolor\, at aliquam tortor. Morbi interdum lore
     m ipsum\, nec varius est porttitor tempus! Praesent sodales dolor sit amet
      feugiat accumsan. Suspendisse pulvinar convallis orci non semper. Nunc fa
     ucibus scelerisque metus\, vulputate eleifend ligula dictum at.
    LOCATION:no addresses specified
    ORGANIZER;CN=Contoso via My Service Site:MAILTO:guid@reply.servicespark.org

    Attaching a file like this to an outgoing message will cause (most) email clients to display RSVP buttons!

    When an ICS/ICAL file is formed properly, Google will show RSVP buttons.
    When an ICS/ICAL file is formed properly, Google will show RSVP buttons.

    When an RSVP choice is selected, an ICS will be sent as a reply to the endpoint.  The email address that sends the ICS response can not be treated as important, and should not be used to identify the recipient.  For example, Google uses one notification endpoint for all of their users, and you will be unable to reliably determine who is RSVP’ing to the event.   Instead, the UID should be used exclusively, or else updates to the event will cause duplicates, and all other manner of chaos.

    Receiving the RSVP Reply

    When Mandrill receives the reply, they will perform a POST to your specified webhook.  The response will be an ICS file.  The ICS file follows the same format as outlined above: it begins and ends with VCALENDAR boundaries, containing at least one VEVENT inside the VCALENDAR.

    A number of parsers exist for robust ICS parsing, but we are not interested in anything beyond the latest response for the UID.  When the ICS was sent, the UID and the incoming email address were saved.  As a result, we can look up the UID based on the email that it came from.  If an email is received, and there isn’t a valid link between that email and UID, then nothing will be done.

    1. Look up the UID based on email address.  This returns the user, and the corresponding action (event RSVP modification, in this case).  It is worth noting that these email events should expire eventually, so these email endpoints will automatically deactivate.
    2. Regex the incoming ICS response for the VEVENT region containing UID.  This is not a multi-line regex.  This will capture and return the entire VEVENT.
    3. Regex the VEVENT region for a valid going/not going/tentative response.  Please note this is a multi-line regex.  This will check each line and then capture and return the DECLINED, ACCEPTED, OR TENTATIVE state of the RSVP.
    4. Once we have determined the new state of the RSVP, update the user’s RSVP.


    The pieces of this project demonstrate the beauty of event-driven programming.  Using modern web development techniques like webhooks allow for decoupled applications to seamlessly interact with each other.  Mandrill’s service integrates so smoothly with my application that the end result is an interface that is invisible, but robust.  The end result is a incredibly rich interface to an application, where the user interacts and derives value from the application without even logging in.


    Mandrill’s email service does not allow for the appropriate attachment headers (specifically `METHOD: REQUEST`) to be included in the message.  As a result, Outlook (desktop and web) will not show RSVP buttons.  Outlook (iOS and Android) perform according to specification and will present RSVP buttons.

  • Recent security changes on bradkovach.com and a new WordPress plugin

    Here is a small enumeration of the ways that I’ve improved security at bradkovach.com

    • Full-time HTTPS is available for all bradkovach.com domains.  My certificates are signed by DigiCert.
    • HTTP Strict Transport Security is enabled.  For compatible browsers (Firefox, Chrome), they should flatly refuse to communicate with bradkovach.com unless HTTPS is available.  This affects all subdomains.
    • `projects.bradkovach.com` will return 406 Not Acceptable for any clients that do not attempt to communicate over HTTPS.
    • `bradkovach.com` will redirect traffic to HTTPS if any requests are made over HTTP.  This is a potential security risk as the request path and query string will be exposed prior to the redirect.  Since the contents of bradkovach.com are public, the trade-off was made in the interest of convenience.
    • I have taken down my public email address and now request that you use my Secure Message form.  I will receive your message via email and use PGP to decrypt it.  Optionally, you can use this facility to securely send me your public key so that we may begin secure correspondence.  By design, no sensitive information is exposed in email headers.
    • My Secure Message form is available as a free plugin so that you can also accept encrypted email from your visitors.
      • I recommend that your site use HTTPS full-time with this plugin.
      • There is no client-side encryption at the moment, which might compromise the security of the message when HTTPS isn’t used.
      • It is licensed GPLv2.0 in accordance with its relation to the GPG project as well as WordPress.
      • Download and Installation details can be found at the GitHub repository.
      • Please feel free to send a pull request if you would like to improve the plugin.
      • I will package the plugin for the WordPress Plugin Repository soon.
  • Free WordPress Theme! Introducing Civique.

    I’ve been working on a WordPress theme lately.  It’s a theme designed for non-profit organizations, but it could work well for almost any organization.

    It is MIT Licensed (compliant with GPLv2).

    It has full support for…

    • Theme Customization
      • Header color
      • Logo (keep it under 75 px tall)
    • Header Images
    • Post Thumbnails
    • Attachment alignment
    • Menus
    • Sidebars

    It includes other goodies, too…

    • Non-profit Summary Shortcode
      A shortcode that uses the ProPublica Nonprofit API to generate a page of information about your 501(c)(3) organization, including summaries of non-profit financial activity.  Insert the shortcode `[civique_summary]` to display the non-profit rundown.
    • Fundraiser Progress Widget
      Add the Civique Fundraiser Widget to your sidebar to easily show progress on one or many fundraisers.  Add as many fundraisers as you need.  They will show a progress bar, and a link to an online donation page, if you include it.

    It is an open-source project hosted by the United Way of Albany County.

  • WordPress Goodie: Theme Customization API code generator

    This isn’t quite as robust as what I usually find over at GenerateWP, but it’s certainly handy, and it makes it easy to correctly link your settings to groups to controls.

    There are some limitations

    • No support for non-`text` WP_Customize_Control elements.  This is an easy fix, and you should reference the official WordPress API Documentation for more help.
    • It is an Excel spreadsheet.  It SHOULD open in OpenOffice/LibreOffice/Google Spreadsheets, but I make no guarantees, nor have I attempted to test these claims.

    Download WordPress Theme Customization API Generator (Excel Spreadsheet)

    MIT Licensed

    Copyright (c) 2014 Brad Kovach

    Permission is hereby granted, free of charge, to any person obtaining a copy
    of this software and associated documentation files (the “Software”), to deal
    in the Software without restriction, including without limitation the rights
    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
    copies of the Software, and to permit persons to whom the Software is
    furnished to do so, subject to the following conditions:

    The above copyright notice and this permission notice shall be included in
    all copies or substantial portions of the Software.


  • Exciting WordPress Developments

    I have been working on some exciting new WordPress things that I plan on releasing in compliance with the GPL.

    First, since there wasn’t a decently simple (free) front-end profile management system, I decided to write one if my own. It is completely customizable with short codes and allows you to validate input with regular expressions before you save the data. All of this is controlled in the post editor. It is nonced using WordPress’ nonce API. It’s pretty elegant in its implementation.

    Next, I plan to release some sort of iteration of my SCSS/CSS and WordPress template framework tools. I have tons of code generation spreadsheets that make grid design and implementation a piece of cake. Provide a couple parameters and the spreadsheet will calculate responsive grids. The grid is based on 6 columns and intelligently resizes all the way down to small screens. I have spreadsheets to make a lot of development work easier. It would be a shame if I didn’t share.

  • Make Twitter Tools respect your Tweets

    Twitter Tools is defacto Twitter-WordPress integration software, but it has one glitch that is easy to fix: tweet posts are formatted so horribly, it’s criminal.

    By default Twitter Tools places the full text of a Tweet in post_content, and an abbreviated, ugly version of the tweet in post_title.  This is fine, because there are (usually, depending on your theme) formats for the proper display of a status.

    Open twitter-tools.php.  After line 515, which reads

    add_post_meta($post_id, 'aktt_twitter_id', $tweet->tw_id, true);

    type a new line that reads

    set_post_format($post_id, 'status');

    Next time Twitter Tools fetches tweets, it will format them as statuses.