While you sleep at night, you can watch a card authentication cycle go by. It’s almost relaxing. The machine takes out the card, reads the chip, checks the certificate, asks for the PIN, and then waits. In less than four seconds, everything takes place. Then the next one starts all over again. The engineers who are watching the line hardly ever look up. They’ve seen it a million times. Still, the line keeps getting longer.
This is how things really are in authentication labs that check smart cards—places that most people don’t think about until something goes wrong. Several of these labs are working around the clock right now to get through a backlog that has been building for months. There are many causes, and none of them are easy to understand.
Smart card authentication is not a new idea, but the systems that make it work are now much more difficult to use. Before a card can be used in a real setting, it has to pass a set of tests, such as checking the validity of the certificate, verifying the PIN, encrypting the data, simulating a login against an active directory, and domain-level authentication. Each step builds on the one before it. The whole process stops if a card fails at making a signature. There are no short-cuts, so speed is limited no matter how many technicians are added to a shift.
Some people thought the backlog problem would happen, but they didn’t want to say it out loud. As companies added more identity and access functions to smart card systems, partly for security and partly to meet regulations, the number of cards that needed full authentication testing increased faster than labs could handle. A lab that was used to handling a few hundred cards a week was now dealing with thousands. It was never going to be easy with the math.
That this is harder to fix than it seems is that testing authentication is not a simple task that can be done in parallel without any limits. Each card has to be put into a reader by hand and checked to make sure it’s there. It then has to go through mechanism support checks, such as checking to see if it can handle RSA signing and decryption, and finally it has to be tested for user login against a domain. Logging in doesn’t work if the system clock is a little off. If the network goes down for a second, you can’t log in. If the account that is linked to that card has been disabled, you can’t log in. Surprisingly many things can go wrong with a single card test, and each failure adds time instead of taking it away.
There is still no clear answer to the question of whether testing around the clock is a long-term solution. Most labs are working through the backlog by adding extra shifts and running parallel reader banks, but the real problem, which is that there isn’t enough testing infrastructure to handle all the cards that are being issued, hasn’t been fixed. Adding more people to a process that is already crowded can help in small ways. It does not make the constraint go away.
As I watch this, I can’t help but think of what happens on software teams when they have a backlog that seems like it will never go away. The backlog is no longer just a list of things that need to be done; it also shows that there is a problem with capacity, setting priorities, or both. In a way, labs that have to wait in line for card authentication are going through the same stress. The work is real and important. But it’s just as important to know why the pile got this big in the first place as it is to know how fast it can be cleared.
