Getting started on Shibboleth

Shibboleth, as defined by shibboleth.net, is "a standards based, open source software package for web single sign-on across or within organizational boundaries". If you're reading this blog post, it's likely because your organization has heard about the benefits of Federated Identity and is eager to jump in, and you are the lucky person who's been asked to set up an Identity Provider, Service Provider, or both.

While there is lots of in-depth information on specific components and settings for Shibboleth online, we had a harder time finding good material for beginners. What if you've never worked with SAML before? What if you only have a vague notion of what "single sign on" is after implementing "Login with Facebook/Google" on a website or two?

For us, these questions all came down to: What's the simplest possible working setup for Shibboleth that we can make? Once we've created that, the in-depth documentation starts to make a lot more sense. But without that basic setup, it's hard to even know where to begin.

With that spirit in mind, we're hoping that by the end of this blog, you'll have a really simple working Shibboleth setup (both a sample Identity Provider and Service Provider), and you'll have wrapped your brain around some of the most important concepts. We're still learning too, so please point out anything that is just completely wrong and we'll correct it, but keep in mind that the purpose of this post is getting started and getting a decent comprehensive view as quickly as possible.

Anyone looking for a deep understanding of Shibboleth should definitely refer to the product's extensive documentation.

Basics

Definitions

  • SP: Service Provider. For our purposes, and the way it seems to be used in most Shibboleth setups, the Service Provider is simply a website or web service that uses Shibboleth as an authentication mechanism.
  • IdP: Identity Provider. From the end user standpoint, the Identity Provider is often nothing more than what you log into for a group of Service Providers. Behind the scenes, it's more complicated (see "The IdP is a broker" below).
  • LDAP: Lightweight Directory Access Protocol. For the purposes of our simple Shibboleth setup, this is a service that can be used to verify if a particular user exists and has entered the correct password. In more advanced setups, it also acts as a source of information about the user (email, phone number, address, etc.). In most situations, you'll be setting Shibboleth up to work with an existing LDAP server.

Logging in

Here's how a regular login story works:

  • You go to a page on a website where authentication is enabled.
  • You don't have a cookie set for the website that is passing along a session token.
  • You are redirected to the website's login page.
  • You submit your username/password back to the website.
  • The website checks that you're you.
  • Once confirmed, it tells your browser to set a cookie with a session token that will now get passed along with any future requests to prove it's still you (and keep you from having to enter your login credentials with every page request).

Now for the Shibboleth login story:

  • You go to an area of a website where shibboleth authentication is enabled.
  • You don't have a cookie set for the website that is passing along a session token.
  • Apache forwards that request to the Shibboleth shibd daemon (most likely running on the same server as Apache).
  • If you have a cookie set saying you're already logged in via the IdP, the request is forwarded to the IdP. It checks the information in that cookie and verifies it has already logged you in.
  • If the cookie for the IdP is not set, the IdP will redirect you to a login page.
  • You submit your username/password back to the IdP.
  • The IdP processes that request and checks the username / password with an LDAP server.
  • The LDAP server verifies that you're you (by ensuring you've entered the right username/password).
  • The IdP sends an authenticated (proving it's the IdP) request back to the website you're logging into.
  • Apache passes that onto the Shibboleth shibd daemon, which verifies that, yep, it's a valid piece of information.
  • The daemon passes that information, along with any requested attributes, back to Apache.
  • Apache passes any of the requested information onto the application and sets a cookie for the specific website (Service Provider), saying you're authenticated.
  • You browse the site in the same way you would if it was a non-shibboleth authenticated session.

Why so difficult?

It's no secret that Shibboleth can be pretty difficult to set up. Expect to bang your head against the wall more than once. This difficulty falls into two areas:

  1. Difficulty in configuration
  2. Difficulty in having a separate site manage your identity

Configuration could probably be improved a lot. Don't like doing all of your config in XML? Don't like having to configure the same thing in two places? Spoiled by "configuration by convention"? Well, you're not in for a fun ride. And there's no reason why you couldn't create your own, easier to configure compatible IdP in Python, Ruby, NodeJS, etc. Except, tied up in all of the old school setup, is a bunch of working code that deals with the second difficult thing, and you probably don't want to spend your time catching all the edge cases there, which is why you're here.

But why is it so difficult for another site to manage your identity? Blame browser security! If two different sites could read from the same cookie, you could easily have another site authenticate you without having to go back and forth so much. It would be much closer to the "regular login" story. But if a site could read the cookie set by another site, it would mean people could hijack your sessions just by having you visit their website or by sticking some malicious Javascript in an advertisement on a legitimate website. Browsers are strict in this area, for some very good reasons.

Okay, so fine… you're going to dive into getting this Shibboleth monster set up. Here we go!

The IdP is a broker

An important thing to understand about the IdP (Identity Provider) is that it is not what authenticates you. It's a middleman that vouches for your identity once something else authenticates you. The Shibboleth IdP does come with a default login page, but it handles the login request by passing it off to another service to do the actual authentication (often an LDAP service). In fact, unfortunately, setting up your own LDAP service seems to be the simplest way to get the IdP working with a full password-based authentication.

Assumptions

Let's get a few things out of the way so we don't have to address them every time they're brought up below. We're going to use Ubuntu 14.04, Java 7, and Tomcat 7. We know of no reason why you can't set up Shibboleth on another version of Ubuntu or an entirely different Linux flavour. You're just going to have to adapt some of the instructions. Having the correct version (version 7) of both Java and Tomcat may be a more critical requirement. You could potentially run into some problems if you're not running the latest versions. Weird problems. Stuff that's hard to find troubleshooting tips for. Uggh.

Note that shibboleth.net recommends that you do use Oracle's JDK instead of OpenJDK, as there seem to be some strange issues when using it with OpenJDK. Unfortunately, Oracle's JDK is a bit more complicated to install, so we will proceed with OpenJDK, which works for what we're doing in the tutorial and avoids the need for a java installation detour.

We're also assuming you're doing all of the below as the root user (because we like living dangerously). If you're the more responsible type, feel free to go through the steps as a non-root user and stick 'sudo' in front of any of the commands that you're told you need root access for.

Requirements for a minimal Shibboleth setup

Assuming you want to create a working example of a Shibboleth SP working with a Shibboleth IdP, using username / password authentication, you'll need to set up the following:

Server 1

  • Shibboleth IdP (non-package install)
  • OpenLDAP (packages: slapd, ldap-utils)
  • Apache2 (package: apache2)
  • OpenJDK 7 (package: openjdk-7-jre-headless)
  • Tomcat 7 (package: tomcat7)
  • Routable IP

Server 2

  • Shibboleth SP (package: libapache2­-mod­-shib2)
  • Apache2 (package: apache2)
  • Routable IP

What you don't need:

  • Proper DNS entries (as long as you are willing to edit your own /etc/hosts file and edit the /etc/hosts files on both the SP and IdP to mimic real DNS resolution).
  • Actual SSL certificates. Most tutorials seem to say that you need real SSL certificates (from a trusted SSL authority) to set up Shibboleth. This may be true for building an example SP that has to operate with an IdP that you haven't built, or an example IdP that has to interact with SPs you haven'€™t built (for example, if you want to use TestShib). But if you're building both so you can see how they interact, it's a complication you can avoid (you will get browser warnings). Obviously, if you have real certificates available to you for the IdP you're setting up, use them.
  • Because we're setting up LDAP on the same server as the IdP, we can avoid having to do secure communication between the two (one more place where we can avoid the overhead of SSL). Obviously, if you're working with an LDAP server that is on another machine, you'll want to set up SSL communication here as well.

Preparation

Create two servers: one for the service provider and one for the identity provider. Attach public IP addresses to them. If you have a DNS provider at hand and an actual domain name you want to use, feel free to skip this /etc/hosts setup. But if not, here's how to trick all the components, including your laptop, into thinking you're working with yourdomain.com.

Modify your /etc/hosts:

555.555.5.5 sp.yourdomain.com # Change to the public IP of your SP server
555.555.5.6 idp.yourdomain.com # Change to the public IP of your IdP server

Modify the IdP's /etc/hosts:

127.0.0.1 idp.yourdomain.com
10.0.5.5 sp.yourdomain.com # Change to the private IP of your IdP
127.0.0.1 ldap.yourdomain.com

Modify the SP's /etc/hosts:

127.0.0.1 sp.yourdomain.com
10.0.5.6 idp.yourdomain.com # Change to the private IP of your SP

IdP Setup (Server 1)

LDAP Setup

apt-get install slapd
apt-get install ldap-utils
dpkg-reconfigure slapd

The reconfigure script will be interactive. Here are the answers you should provide:

1. Omit OpenLDAP server configuration?: No
2. DNS domain name: ldap.yourdomain.com
3. Organization name: YourCompany
4. Administrator password: supersecretpassword
5. Confirm password: supersecretpassword
6. Database backend to use: HDB
7. Do you want the database to be removed when slapd is purged?: No
8. Move old database?: Yes
9. Allow LDAPv2 protocol? No

Testing that LDAP is working

Assuming you've used ldap.yourdomain.com during the setup above, run:

ldapsearch -LLL -x -D 'cn=admin,dc=ldap,dc=yourdomain,dc=com' -b dc=ldap,dc=yourdomain,dc=com -w supersecretpassword

Explanation:

  • -LLL: Takes out some of the extra logging information

  • -x: Don't worry about using SSL communication

  • -D 'cn=admin,dc=ldap,dc=yourdomain,dc=com': Essentially, this is your "username" for authenticating with LDAP to do the search

  • -w supersecretpassword: Your password

  • -b dc=ldap,dc=yourdomain,dc=com: And this is the base you're searching from

The dc=...,dc=...,dc=... strings deserve some more explanation. You essentially need one dc=... for each section of your domain name. So, if you had entered ldap.ridiculous.subdomain.shenanigans.yourdomain.com to the "DNS domain name" question above, you'd be specifying: "dc=ldap,dc=ridiculous,dc=subdomain,dc=shenanigans,dc=yourdomain,dc=com" instead of "dc=ldap,dc=yourdomain,dc=com" anywhere it's mentioned. The admin user (used in cn=admin) is set up by default for you.

You should get output that looks like:

dn: dc=ldap,dc=yourdomain,dc=com
objectClass: top
objectClass: dcObject
objectClass: organization
o: YourCompany
dc: ldap

dn: cn=admin,dc=ldap,dc=yourdomain,dc=com
objectClass: simpleSecurityObject
objectClass: organizationalRole
cn: admin
description: LDAP administrator
userPassword:: e1NTSE...crazyhashstring...xPSExDS0NEZDRBd0FuVnJ1MEU5Q20=

Creating your first user

Great! Now we're going to create a single user in your LDAP database. You'll actually use this username and password to log into your sample application.

ldapadd -x -D "cn=admin,dc=ldap,dc=yourdomain,dc=com" -w supersecretpassword <<EOF
dn: cn=SOME_USERNAME,dc=ldap,dc=yourdomain,dc=com
objectClass: inetOrgPerson
cn: SOME_USERNAME
uid: SOME_USERNAME
UserPassword: SOME_PASSWORD
sn: SOME_SURNAME
EOF

If you prefer, you can create a temporary .ldif file and pass it into ldapadd via -f your.ldif instead of the <<EOF ... EOF block.

Important note: The above user we created is a minimalistic one. You'll probably want to add other attributes like the user's email address, first name, last name, etc. And Shibboleth can be given access to those attributes and pass them on. But we're not trying to create LDAP wizards here. We want to see what the most basic working Shibboleth setup actually needs, so that we can separate that from all the extra bells and whistles. (Why do you need to specify a surname? Not sure, but ldapadd won't work if you leave that out).

And that's it! You're now done with the basic LDAP setup required to have it authenticate your Shibboleth logins. By the way, if after going through this you're raising your eyebrows at the "Lightweight" in "Lightweight Directory Access Protocol", you're not alone. If anyone has any links that give good explanations as to why LDAP is the way it is, and how to wrap your brain around it, please send them our way and we'll add them!

Shibboleth IdP Installation

Now, on the same server we just set up LDAP on we're going to set up the Shibboleth IdP. Remember, that this doesn't have to be the case. LDAP could be on a completely different server. But having it on the same server is going to allow us to avoid setting up extra security setup.

First, let's install the necessary packages:

apt-get install openjdk-7-jre-headless
apt-get install tomcat7

Hold onto your keyboards. This installation is about to get a bit ugly.

You'll need to grab the latest tarball of the IdP. At the time of this writing, the latest non-beta version at Shibboleth.net is 2.4.2, and you can grab and install it via:

cd /tmp
wget http://shibboleth.net/downloads/identity-provider/2.4.2/shibboleth-identityprovider-2.4.2-bin.tar.gz
tar -zxvf shibboleth-identityprovider-2.4.2-bin.tar.gz
cd shibboleth-identityprovider-2.4.2
JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 ./install.sh

Note: The installer will ask you several questions.

  1. Where should the Shibboleth Identity Provider software be installed?: /opt/shibboleth-idp
  2. What is the fully qualified hostname of the Shibboleth Identity Provider server?: idp.yourdomain.com
  3. A keystore is about to be generated for you. Please enter a password that will be used to protect itpassword_that_we_don't_use

Copy the endorsed directory into your tomcat7 share directory:

cp -R endorsed /usr/share/tomcat7

Shibboleth IdP Configuration

Now you have to edit some configuration files:

1. To make your life easier, turn on as much logging as you can to help when things inevitably go wrong. Open /opt/shibboleth-idp/conf/logging.xml and set the level to ALL for shibboleth, opensaml, and ldap:

<!-- Logs IdP, but not OpenSAML, messages -->
<logger name="edu.internet2.middleware.shibboleth" level="ALL"/>

<!-- Logs OpenSAML, but not IdP, messages -->
<logger name="org.opensaml" level="ALL"/>

<!-- Logs LDAP related messages -->
<logger name="edu.vt.middleware.ldap" level="ALL"/>

2. Open /etc/tomcat7/server.xml and uncomment a couple of sections to do with ports 8009 and 8443:

<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
           maxThreads="150" scheme="https" secure="true"
           clientAuth="false" sslProtocol="TLS" />

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

3. Create /etc/tomcat7/Catalina/localhost/idp.xml with the following content:

<Context docBase="/opt/shibboleth-idp/war/idp.war"
         privileged="true"
         antiResourceLocking="false"
         antiJARLocking="false"
         unpackWAR="false"
         swallowOutput="true" />

4. Disable the RemoteUser LoginHandler and enable the UsernamePassword LoginHandler in /opt/shibboleth-idp/conf/handler.xml:

<!-- Login Handlers -->

...

  <!--
  <ph:LoginHandler xsi:type="ph:RemoteUser">
    <ph:AuthenticationMethod>
      urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified
    </ph:AuthenticationMethod>
  </ph:LoginHandler>
  -->

  <ph:LoginHandler xsi:type="ph:UsernamePassword"
                   jaasConfigurationLocation="file:///opt/shibboleth-idp/conf/login.config">
    <ph:AuthenticationMethod>
      urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport
    </ph:AuthenticationMethod>
  </ph:LoginHandler>

...

5. Configure Shibboleth's LDAP connection for login in /opt/shibboleth-idp/conf/login.config:

...

ShibUserPassAuth {
edu.vt.middleware.ldap.jaas.LdapLoginModule required
  ldapUrl="ldap://ldap.yourdomain.com"
  baseDn="dc=ldap,dc=yourdomain,dc=com"
  bindDN="cn=admin,dc=ldap,dc=yourdomain,dc=com"
  bindCredential="supersecretpassword"
  ssl="false"
  userFilter="uid={0}";
};

...

6. Configure LDAP again (yes, you're really doing almost exactly the same configuration again in a different format) for attribute resolution in /opt/shibboleth-idp/conf/attribute-resolver.xml:

<?xml version="1.0" encoding="UTF-8"?>

...

<resolver:AttributeResolver ... >

  ...

  <!-- Example LDAP Connector -->
  <resolver:DataConnector xsi:type="LDAPDirectory"
                          xmlns="urn:mace:shibboleth:2.0:resolver:dc"
                          id="myLDAP"
                          ldapURL="ldap://ldap.yourdomain.com"
                          baseDN="dc=ldap,dc=yourdomain,dc=com"
                          principal="cn=admin,dc=ldap,dc=yourdomain,dc=com"
                          principalCredential="supersecretpassword"
                          lowercaseAttributeNames="true">
    <dc:FilterTemplate>
      <![CDATA[
        (uid=$requestContext.principalName)
      ]]>
    </dc:FilterTemplate>
  </resolver:DataConnector>

  ...

</resolver:AttributeResolver>

7. And make sure all the permissions are right for Tomcat:

chown tomcat7: /opt/shibboleth-idp/ -R
chown tomcat7: /usr/share/tomcat7/ -R
chown tomcat7: /etc/tomcat7/ -R

8. Restart Tomcat:

service tomcat7 restart

Apache Setup

We're going to set up Apache now to service regular HTTPS requests and forward anything shibboleth related to port 8009.

1. Install the package:

apt-get install apache2

2. Create a self-signed certificate. Or, if you already have a certificate from a certificate authority, feel free to use that.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/idp.yourdomain.com.key -out /etc/ssl/certs/idp.yourdomain.com.crt

You'll be asked some questions about your location, etc. Answer whatever seems appropriate. Since this is only being used by yourself, it doesn't really matter. The main question to pay attention to is:

Common Name (e.g. server FQDN or YOUR name) []: 

idp.yourdomain.com

Whether or not you're using self-signed certificates, /etc/ssl/certs/idp.yourdomain.com.crt and /etc/ssl/private/idp.yourdomain.com.key should now exist.

3. Now we want to create an Apache configuration for the IdP, so that we can access its login page.

/etc/apache2/sites-available/idp.yourdomain.com.conf

<VirtualHost *:443>
  ServerName idp.yourdomain.com
  SSLEngine on
  SSLCertificateFile /etc/ssl/certs/idp.yourdomain.com.crt
  SSLCertificateKeyFile /etc/ssl/private/idp.yourdomain.com.key
  # If you have an intermediate certificate from an SSL provider, you can specify it here
  # SSLCertificateChainFile /etc/ssl/certs/your-ssl-authority-intermediate.crt
  <Proxy ajp://localhost:8009/idp/*>
    Allow from all
  </Proxy>
  ProxyPass /idp/ ajp://localhost:8009/idp/
</VirtualHost>

See the <Proxy> section above? That's what redirects your request to the various Java objects that comprise Shibboleth.

4. Enable the site and required modules and restart apache.

a2ensite idp.yourdomain.com
a2enmod ssl proxy_ajp
service apache2 restart

Verifying IdP setup so far

You'll probably want to check that everything is working so far. You can do that by visiting:

https://idp.yourdomain.com/idp/profile/Status

It should display 'ok', but you may have to wait a few minutes (Shibboleth seems to take a long time making the LDAP connection during startup). For bonus points, you can check:

https://idp.yourdomain.com/idp/Authn/UserPassword

It should display a simple login page.

You're not done yet. You're going to have to come back to add the metadata once you've set up the SP. Then you're going to have to restart tomcat again. Let's finish off here with some Apache configuration and move onto the SP.


SP Setup

Compared to the IdP, the SP is going to seem ridiculously easy. We're going to install Apache again, this time simply because that's what's going to be serving our website. We will then install and configure the Shibboleth Apache module and configure it. The next step will be to get all this metadata stuff sorted out, taking a brief trip back to the IdP to do so. Finally, we're going to set up a page that we want to require the user to be logged in to see, and turn on Shibboleth authentication for that area.

Let's install our packages:

apt-get install apache2
apt-get install libapache2-mod-shib2

Website

You're probably going to be adding this to an existing site, not creating a brand new one. So let's take a brief detour and create the simplest existing website we can think of.

1. First we'€™ll do a bit of security that's technically not needed for Shibboleth to work, but would be critical in any real world scenario. You should at least be doing anything authentication-related over SSL these days. In fact, anything that you need to be logged in to do should go over SSL. To simplify our lives, we'll use self-signed certificates, just like we did with the IdP (and again, feel free to use actual certificates if you have them). And we won't worry about redirecting http requests to https, or providing any http content, for that matter.

Alright, let's generate that self-signed certificate and key:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/sp.yourdomain.com.key -out /etc/ssl/certs/sp.yourdomain.com.crt

As before, you'll be asked some questions about your location, etc. Again, answer whatever seems appropriate. The main question to pay attention to is:

Common Name (e.g. server FQDN or YOUR name) []:

sp.yourdomain.com

Whether or not you're using self signed certificates, /etc/ssl/certs/sp.yourdomain.com.crt and /etc/ssl/private/sp.yourdomain.com.key should now exist.

2. Create /etc/apache2/sites-available/sp.yourdomain.com.conf:

<VirtualHost *:443>
  ServerName sp.yourdomain.com
  ServerAdmin youractual@email.address
  DocumentRoot /var/www/html

  SSLEngine on
  SSLCertificateFile /etc/ssl/certs/sp.yourdomain.com.crt
  SSLCertificateKeyFile /etc/ssl/private/sp.yourdomain.com.key
  # If you have an intermediate certificate from an SSL provider, you can specify it here
  # SSLCertificateChainFile /etc/ssl/certs/your-ssl-authority-intermediate.crt
</VirtualHost>

3. Enable the site and restart apache:

a2enmod ssl
a2ensite sp.yourdomain.com
service apache2 restart

4. You should be able to go to https://sp.yourdomain.com and see the default apache page. Let's add one more section:

mkdir /var/www/html/auth
echo "<html><body>logged in</body></html>" >/var/www/html/auth/index.html
chown -R www-data:www-data /var/www/html/auth

You should now be able to visit https://sp.yourdomain.com/auth and see logged in. But you're not logged in! That's okay, we're going to set that up.

SP Configuration

Before getting started with the config files, generate your application-specific Shibboleth key:

shib-keygen

This will be unique to your application. In this example you're using a single server, but if you had more than one server handling authenticated requests for your application, you would need a copy of this key on each of those servers. If you set up a second application, you would need to generate a new key.

Now we need to configure Shibboleth. This time, thankfully, we only need to touch one XML file!

/etc/shibboleth/shibboleth2.xml:

1. Change

<ApplicationDefaults entityID="https://sp.example.org/shibboleth" REMOTE_USER="eppn persistent-id targeted-id">

to:

<ApplicationDefaults entityID="https://sp.yourdomain.com/shibboleth" REMOTE_USER="eppn persistent-id targeted-id">

2. Next change

<Sessions ...>

  ...

  <SSO entityID="https://idp.example.org/idp/shibboleth"
       discoveryProtocol="SAMLDS" discoveryURL="https://ds.example.org/DS/WAYF">
    SAML2 SAML1
  </SSO>

  ...

</Session>

to:

<Sessions ...>

  ...

  <SSO entityID="https://idp.yourdomain.com/idp/shibboleth"
       discoveryProtocol="SAMLDS">
    SAML2 SAML1
  </SSO>

  ...

</Sessions>

3. Add in a reference to the IdP metadata file that we're about to create (make sure it's below the Sessions section):

<ApplicationDefaults ...>

  ...

  <Sessions ...>
    ...
  </Sessions>

  ...

  <MetadataProvider type="XML" file="idp-metadata.xml"/>

</ApplicationDefaults>

Proxy shibboleth requests to shibd

Just like with the IdP, you have to give Apache an idea of what to do when it receives an SAML (Security Assertion Markup Language) request. In this case, we only have to pass it to the shibd daemon, and we have a nicely packaged Apache module to do that. Reopen /etc/apache2/sites-available/sp.yourdomain.com.conf and add:

<VirtualHost *:443>

...

  <Location /Shibboleth.sso>
    SetHandler shib
  </Location>

...

</VirtualHost>

Grab the IdP Metadata and restart services

The SP needs to know about the IdP. Both the IdP and SP have URLs that you can make a simple GET request on in order to retrieve the metadata. To grab and refresh with the IdP's metadata:

cd /etc/shibboleth
wget --no-check-certificate https://idp.yourdomain.com/idp/shibboleth -O idp-metadata.xml
service shibd restart
service apache2 restart

Add the SP metadata to the IdP

We still haven't let the IdP know about the service provider. That's because, when we were first setting it up, the SP didn't exist in a working state. So we have to go back and do that. And, just like with the SP, we unfortunately can't just put the metadata in a directory and have the IdP automatically pick it up. We have to additionally modify the relying-party.xml configuration to add a pointer to the SP's metadata on the filesystem.

1. Log back into the IdP server and add a reference to metadata for your SP (which you're about to create) in /opt/shibboleth-idp/conf/relying-party.xml under the existing Chaining Metadata Provider:

<!-- ========================================== -->
<!-- Metadata Configuration -->
<!-- ========================================== -->
<!-- MetadataProvider the combining other MetadataProviders -->
<metadata:MetadataProvider id="ShibbolethMetadata"
                           xsi:type="metadata:ChainingMetadataProvider">

...

  <MetadataProvider xsi:type="FilesystemMetadataProvider"
                    xmlns="urn:mace:shibboleth:2.0:metadata"
                    id="URLMD"
                    metadataFile="/opt/shibboleth-idp/metadata/sp.yourdomain.com.xml" />

...

</metadata:MetadataProvider>

2. Run the following to get the metadata from the SP and refresh tomcat:

cd /opt/shibboleth-idp/metadata/
wget --no-check-certificate https://sp.yourdomain.com/Shibboleth.sso/Metadata -O sp.yourdomain.com.xml
service tomcat7 restart

Protecting the /auth directory

Hooray! We are DONE with the IdP! Now we just have to go back to our website config on the SP and actually use Shibboleth to protect our authorized area.

1. Reopen /etc/apache2/sites-available/sp.yourdomain.com.conf and add:

<VirtualHost *:443>

...

  <Location /auth>
    AuthType shibboleth
    ShibRequireSession On
    require valid-user
  </Location>

...

</VirtualHost>

2. Restart apache for the very last time:

service apache2 restart

Demo

  1. Go to https://sp.yourdomain.com/auth.
  2. You'll be redirected to https://idp.yourdomain.com/idp/Authn/UserPassword.
  3. Enter SOME_USERNAME and SOME_PASSWORD (that you set up way back at the start during the LDAP config) and submit.
  4. You'll be redirected back to https://sp.yourdomain.com/auth.
  5. If all went well, you should again see logged in.
  6. Refresh the page. You should still be logged in.
  7. If you're curious, check your cookies. There will be two for idp.yourdomain.com and one for sp.yourdomain.com.
  8. Try deleting just the sp.yourdomain.com cookie once you'€™ve logged in, and refresh. You should briefly see the web browser go to the IdP url URL and then return to https://sp.yourdomain.com/auth. You didn't have to re-enter your username/password because you're still logged into the IdP.

Troubleshooting

Using the default login page as a debugging tool

You can't just log in to Shibboleth's default login page, but going there can tell you a lot about whether you've got the basic shibboleth setup working.

You can even try passing in a username/password that you know won't work, just to see how far it gets along in the process. If it gets all the way to the LDAP logs, you know that at least the basic communication between Shibboleth and LDAP is working.

Check the default login page at: https://your.idp.domain/idp/Authn/UserPassword

You've restarted tomcat and get errors now when accessing any IdP page

After doing a service tomcat7 restart, it can actually take quite a while after the service has given its 'ok' status for it to get to the state where it can serve requests again.

Keeping logging set to the most verbose level (ALL) makes it easier to tell when Shibboleth is still starting up. This will also likely pause, however, at:

destroyed ldap object: edu.vt.middleware.ldap.Ldap@1357968235::config=edu.vt.middleware.ldap.LdapConfig@280469060::env={java.naming.provider.url=ldap://ldap.yourdomain.com, java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory}

We're running these examples on virtual servers. It may not stall out like this if you're running on bare metal. Try and wait at least 2-3 minutes before engaging in any further troubleshooting (if you're pretty sure you've followed the steps up to that point correctly).

After making any change…

You may need to re-grab the metadata on the IdP from the SP when you make a change to the configuration of the SP. You may also need to re-grab the metadata on the SP from the IdP when you make a change to the IdP. We're not sure exactly why this is necessary. Certainly, if you regenerate any keys, you'll need to do it. But certain smaller changes may require metadata refreshes as well.

When in doubt, just do it. It will save you a bunch of head scratching. It's not entirely clear to these authors why Shibboleth is so particular about this.

Checking that essential services are running on the IdP

  • netstat -anp | grep 8009: IdP backend. If this isn't up, all bets are off.
  • netstat -anp | grep 443: Apache HTTPS. You won't be able to get to the login page or the status page without this.
  • netstat -anp | grep 389: LDAP backend. If this is down, you may get the login page, but you won't be able to actually log in. Or, you may get errors starting up when Shibboleth tries to connect to LDAP.

Debugging LDAP communication problems

This will shut down the OpenLDAP daemon and run it as a blocking process:

service slapd stop
slapd -h ldap://localhost -d 481

To go back to normal, hit Ctrl-C and service slapd start

Checking the logs

Tail all of the important logs at once via:

tail -f /var/log/apache2/*.log /opt/shibboleth-idp/logs/*.log /var/log/tomcat7/*.log