InterExchange Auth Service - Part One

By ozzyaaron

Friday, Dec 11, 2015

Another Microservice Post - Not Really!

Microservices, they’re all the rage at the moment - as in that rage you feel is the microservices architecture you built out before you needed to.

This is part one of an unknown series of posts about migrating from a few separate apps managing their own auth and user databases to one central user service. Part One deals mostly with the nitty gritty of getting the service off the ground and how we test this in an automated way.

We knew we needed some sort of identity or authentication service but we fought hard to keep it in our main application due to experience with early break-out and the issues that can cause. That being said after a few days of trying to get various libraries working together we spent about a day doing the majority of the work to enable a central auth service. It was just easier to separate this out to a service both from an operational and design point of view. The team can now easily reason about how our applications will integrate with it and the siloing of the service has made implementing features a piece of cake.

In other words we fought against building a service but we got the traditionally espoused benefits of SOA. Namely iterating on the service is lightning fast, the interface is solid and the service can be improved and updated independently of the rest of our services and apps. We also feel better about the fact that we absolutely required a new service be built.

Why?

I think it’s always important to start with a real questioning phase of a new feature. Again we didn’t start out with the idea that we needed a new service, this was to support a feature we desired. We fought against having a centralised auth feature for a while but after discussions across the C-level and engineering teams it became obvious we really did want users to have a central “User Profile” and “User Authentication” area.

Currently we have many silos of repeated user data. For example if a user applied two years in a row we could quite easily have their details in two places. This isn’t in and of itself bad but take my word for it, in our case it kind of is. There were valid historical reasons for this but a central area for users to manage their information would help to remedy this problem for us moving forward.

Whilst the main long term driver for our auth service was a feature around data management and being able to data dive more betterer in the future, driving out this feature really screamed for a central auth service as a starting point.

The Pieces of Of the Puzzle

So straight up this puzzle isn’t a difficult one but a lot of the pieces did seem to be missing when we opened the box. Just like with our oAuth + Devise post it was difficult to find the step-by-step guide. To be fair our solution is a little out-of-the-box so hopefully this will help some people in the future.

Basically this is the diagram that started everything for us.

Already Signed In User Attempts Login

sequence - user already signed in

Anonymous User Signs in Using Third Party

sequence - 3rd party oauth

Anonymous User Signs in Using Existing InterExchange Credentials

sequence - auth with iex

The main part that keeps this working is that as long as our auth service has a valid oAuth authorisation and a Devise session (the later is more important) then you can login to any of our applications. At this point the application takes over Authorisation as you’d expect.

With this in mind whether you achieve an authentication session via an external oAuth process or the internal Devise login, as long as you’re logged into IEXAuth then you’re golden.

The Difficult Bits

As I mentioned there are a few things that didn’t work out of the box so let’s go through a few of them.

Our Omniauth Strategy

For this we basically cracked open other strategies and checked out what they were doing. Open source huh?

There were two concerns that we had here. Firstly redirection after login should continue to work and secondly we needed to provide data after the oAuth process was complete so we could log the user into the consumer.

class InterExchangeOauth2 < OmniAuth::Strategies::OAuth2
  option :name, :inter_exchange_oauth2

  option :client_options, {
    site: ENV.fetch("AUTH_URL")
  }

  info do
    {
      email: email,
      id: id
    }
  end

  uid { id }

  def id
    data["id"]
  end

  def email
    attributes["email"]
  end

  def attributes
    data["attributes"]
  end

  def data
    raw_info["data"]
  end

  def raw_info
    @raw_info ||= access_token.get('/oauth/user').parsed # this uses the token to retrieve user details
  end

  def authorize_params
    super.tap do |params|
      params['return_to'] = "#{ENV.fetch('ROOT_URL')}#{session['return_to'].to_s.gsub(/^\//,'')}"
    end
  end
end

So this is probably pretty easy to follow after I make the point that most of this is just stuff you find by reading the Omniauth and Doorkeeper documentation. You can also read the code from Omniauth and you’ll see why info is called (and therefore raw_info) in the first place.

The pertinent parts are that we retrieve the users information by using the oAuth token we retrieved. This returns JSON formatted data about the user that we pick apart to become available in the response from the oAuth process.

The authorize_params call adds a redirect URL to the call to our oAuth provider. This is stored in the provider so that after oAuth finishes successfully the user is redirect back to the correct location.

Auth Service

This is actually a pretty basic app at this point. We shifted all of our existing login infrastructure to this service so users could login to IEXAuth via email and password or the Google oAuth provider. Once you’ve authed to IEXAuth then we slipstream you into the app that you came from or you’re presented with a choice of where you’d like to go. The great thing about this is we have Single Sign On by doing this. Once you’re authed to IEXAuth then by hitting any of our apps that are integrated you’ll start the oAuth process against IEXAuth which will be invisible to the user (as long as their IEXAuth session hasn’t expired) and Bob’s your mother’s brother.

When users are created in our consumers they ping the Auth service to create the user and grant it access to the appropriate application/s. Each application has a setup in the Auth service like any normal oAuth provider.

Migrating Users

BCrypt. BCrypt. BCrypt.

If you’re like me then you might go into this with some knowledge of hashing, salts, peppers and other terms about security-or-something. I thought I understood how passwords are encrypted these days but BCrypt is pretty awesome. Sufficed to say we were able to write a simple script to import a CSV of our users’ information. It literally was as simple as importing the data we wanted with the encrypted password and it worked.

I expect importing passwords from our PHP app will be equally simple. Importantly myself and the CTO removed this risk at the outset of the project so technically we won’t have issues importing users from other applications we have as we integrate them.

So long story short this part is incredibly easy and involved us writing a dataclip that allows a script to stream users from our current data sources into the Auth service.

Creating and Updating Users

This is just simple API work and involved adding create and update endpoints for our users on the Auth service. We then added an adaptor to the consumers and have callbacks send the relevant update/s to the Auth service.

class Adaptor
  attr_accessor :user

  def create_or_update_user
    return Response.new(config.test_response) if config.test_mode

    response = RestClient.send(strategy, users_url, user_payload.to_json, content_type: :json, accept: :json)
    return false unless response.code.to_s.starts_with?("2")
    return Response.new(response.body)
  end

  def user_payload
    base_payload.merge({ email: user.email, access: "App" })
  end

  def base_payload
    { api_key: config.api_key }
  end

  def strategy
    return :put if user.uid
    return :post
  end

  def users_url
    # Yep but File.join is actually much better than URI::join
    File.join(config.base_url, "users", user.uid.to_s)
  end
end

This is the meat of the adaptor. It defines its own Response class to parse the returned data, has a configuration that allows the user to setup a test response and put the adaptor into test mode and handles the cases we’ll see from our service.

Testing

Urrgh.

Honestly once we’d chased down a few unpromising solutions and read a bunch of documentation we built a working prototype in less than two days of work. Testing our solution was a slight nightmare at first until we remembered to treat Auth like a black box.

Initially we were attempting to stub our Auth service using Omniauth’s methods but really we just needed a strong test suite for login after which we assume this works in other tests. We simply use Devise’s strategy to login and logout as necessary. Its these sort of headslapping moments that teach me that after 15 years building software I’m still stupid (sometimes).

I’m happy to report that we now have a green suite and there is nothing special involved.

For us specifically we swapped out our existing Google oAuth provider tests for our InterExchange oAuth provider and shifted the Google oAuth tests to the InterExchange Auth service suite.

My advice would be first to make sure that your boundary is well defined so that when you setup your tests you can easily determine where responsibilities end and what data will cross the boundary. In our case we have a strong boundary between our Auth service and any consumer. This means that we test things in our Auth service like “Given I get this, I respond with that” but we can’t really reach outside of our own box without some other integration tests. Equally from the consumers all we can really do is test that given our Auth service responds with x then I’ll do y. My plan is to get our Project Manager/s trained in an integration tool that will fill the gap.

Conclusion

Long story short developing this service and integrating it with our App/s has been a lesson in why planning and reading documentation is important. It was also important to me that we fought against building another service and only succumbed because it was the best way to solve the problem in this case.

We will iterate on this service over the coming year as we leverage it to service business cases in the future. It will be an important part of our infrastructure and I’m happy that we can easily begin to provide SSO through multiple auth providers. It’ll be really great for all of our users and steamline many of our processes in the future.

Lovely.