I've been fiddling with IFD and ADFS a bit more than what's healthy the past couple of years and I still face the "What the h*** is wrong now"-moments from time to time. Admittedly they're a bit more rare and not quite as severe as they uses to be, thank god.
One thing I've been running into quite frequently the latest times I've been messing with IFD is that when you think you're done and you try to log on for the first time you get a "I can't connect to the ADFS-server"-message in the browser.
On the ADFS server the logs say "the element 'serviceIdentityToken' was fount to have invalid data", on the CRM server you have a "Exception information:
Exception type: EncryptedTokenDecryptionFailedException
Exception message: ID4036: The key needed to decrypt the encrypted security token could not be resolved from the following security key identifier " and the CRM trace tell you something like "Microsoft.IdentityModel.Tokens.EncryptedTokenDecryptionFailedException: Microsoft Dynamics CRM has experienced an error" (and by the way, yes, that's the coherent part of that error).
You think, probably at least since I have, What happened, I just connected to all these servers.
So, what needs to be done? I can't tell you exactly what makes things work but if you update the metadata in the ADFS server manager, restart ADFS-service, restart the IIS you should be fine.
If that doesn't work, start browsing Chris Cognettas blog.
Developer at CRM-Konsulterna