You all know Murphy, right? The one whose law revolves around anything that can go wrong will go wrong.
And you probably know the Law of Unintended Consequences: all actions have consequences, whether intentional or unintentional.
One of the reasons I have remained in software security is because I have a knack for finding those unintended consequences. This time, I found many of them. In many disparate but related systems. Including myself.
It was supposed to be a great weekend of seeing friends and family. Everything started out fine. The flight was departing kind of later than expected, but we still arrived around the planned 10:30 PM arrival time. What happened over the next roughly 10 hours would have made Murphy proud.
Cue the dramatic music.
I arrived at the car rental facility. All the rental agencies shared the same garage and building. The two Hertz employees were serving what looked like 10 cars worth of customers. A third representative showed up, and then one of the others left. And then a second employee left. The lone employee was trying to calm people down as she served everyone. It’s not her fault.
The QR code for skipping the line seemed sufficient. Put in the information and waited. The “Please wait as we get your car ready” screen flashed a 15-minute timer. Okay, I’m still in line, no biggie. The “We don’t do the web check-in” announcement from the clerk followed shortly after that 15-minute clock hit zero. I wasn’t the only one trying.
I stepped up to the counter, tried to be friendly, and told her I was sorry she had to deal with this. Her comment: “It’s the typical crappy Friday night crowd.” I was told to go to one of two areas and pick a car.
User Interface
Immediately after exiting, I see a sign with one of the two areas above a line of cars. I pick a car, get situated, and start the car. As I start pulling away, I glance down and notice the key is for “Avis.” Whoops. I put the car back in the parking spot and go get my things.
Lesson Number 1: User messaging needs to be crystal clear. When users are in a rush, tired, or otherwise not in a clear state of mind, the directions need to be clear and the layout of the product needs to restrict surface area issues. A better sign or even some method of blocking would have made it clear the line of cars wasn’t for Hertz.
Lesson Number 2: Make sure you prevent authorization bypasses early. Leaving a key for a remote start seems like a great idea. The car doors are open, customers can go in and out, and start the car. No need to spend time managing the keys. They just stay in the car. The issue is that the key ring gives the customer a warning as to which company the car is associated with. By not having to look at it or having it handed to them, it is an easy miss.
On to car number 2.
Feature Creep
I looked around for the other Hertz sign. Found it and went to the car. As I got in, I looked at the key and double-checked the Hertz keychain. Started the car. I looked for the fuel gauge and there was none.
I am not a car guy. I see cars as utility. Get me from point A to point B. I love the Tesla S. I want one.
The reason I bring this up is I was not prepared for an electric car. This trip was a good bit of driving. I didn’t know where there were chargers. I’m sure I could have found them, but this was not the trip to be testing it out.
Lesson Number 3: Feature flags are a great way for testing. But when you are testing new features, don’t do it in a manner that can cause major headaches for your users.
On to car number 3.
Race Conditions
This has to be the end of this part of the story, right? Third car. I got in, got situated, and started heading to the exit. I had my driver’s license ready along with the paperwork. Car rental places really are great at output encoding and making sure the car leaving is authorized.
The attendant scans the barcode on the car and takes my license. The next thing that happened sent me into a laugh. The attendant says, “You can’t take this car; it is reserved.” My reply was simple: “Can you tell me what car I can take that isn’t pure electric?” After a minute he says, “Do you want a black or blue Mustang?” I’ll take the black one. He directs me where to park this car and where the black one is.
Lesson Number 4: Set the system up to prevent users from accessing out-of-stock items or reserved items. Check frequently and put blocks on items that can’t be used. In this case, Hertz should have had this car in a different location or even held the key at the desk. My guess is they don’t do this because Gold members can go straight to their cars. There is almost certainly a solution for this.
Maybe Resetting Input Fields is Good?
Well, the Mustang was fine. Got in, showed my license again, and off we went. That experience took way longer than anticipated. As I drove to the hotel, I could tell I was tired. Fortunately, it wasn’t that far. But it was late and dark. About 5 minutes from the hotel, I had that feeling. The feeling I forgot something or didn’t have something.
License? Check. Phone? Well, I was using the GPS on it. Bag? Check. Wallet? Oh…. I’ll save the language there. See, here’s the thing. I knew I needed my license out, so I kept it out. Usually, it’s in my wallet and my wallet is in my pocket.
Alright, let’s get to the hotel, check in, drop off my bag, and head back to the rental agency.
The clerk at the front desk tried to be helpful. I just lost my wallet. I had my cell phone so could at least check in with that and my license. He was able to get the number for Hertz, but of course, it was closed.
I go into the room, drop my stuff off, and head back to the garage.
Lesson Number 5: I usually hate when user interfaces reset key fields when an error occurs. I am now reconsidering that notion. If I reset my license into my wallet, then I would not have lost it. Or at the very least, I would have had to retrace my steps much sooner and may have found it sooner. Maybe having the user re-verify critical data points isn’t horrible? Yes, it is. Sometimes, the responsibility is on the user. It was my fault for not checking for my wallet.
Nobody had turned the wallet in, so time to go back to the hotel and cancel the cards.
Remove Access to Dead Functions
At this point, I’m beat. But, I know the first thing that I have to do is cancel all the cards that I know were in my wallet. I have to consider it gone. And even if it is returned, I’m not sure I can trust the cards are safe. Better to just get new ones.
Channeling my best Jim Carrey in “Bruce Almighty,” I flop down on the sofa and feverishly cancel all the cards. I always bring my laptop with me. I either want to do work, journal, or just if I need it.
As I finished up the last one, I glanced up. I noticed the second drawer of the dresser was missing. I looked a little to the left and noticed the wardrobe closet had a huge dent in it. I looked at the floor and noticed it was filthy. I stand up and just say it is time to go to bed. Go to use the bathroom and open the lid to the toilet. A softball-sized hairball.
That’s it. I’m out. Go back to the sofa to get my bag and notice the cushions are shredded. Get my bag, go to the front desk, and there are no other rooms. It’s something like 2:30 to 3:00 AM. I’m exhausted. I’m fortunate my kids weren’t with me (they wanted to go but couldn’t). And all I can do is go back to the room, crash on the sofa, and leave in a few hours.
Neither the night clerk nor the morning clerk could do anything for me. The morning clerk suggested calling the general manager on Monday. Which I did. The General Manager informed me that the room I was in was a “Do Not Sell” room. She also informed me that since I booked the room using Hotels.com, they couldn’t refund me the money. They could offer me some free nights. Yeah, no thanks. I’m not staying at that hotel chain.
Lesson Number 6: In the world of software, there is the concept of dead code. Or a feature that is either set to be removed or shouldn’t be enabled yet. It’s imperative to make sure that if this code is deployed to production, do not run it. And when it is, test it thoroughly. There’s also a joke that can be made here about making sure the code is “Clean Code,” but I won’t go there.
Well, at least the rest of the trip went well. I saw people I planned on seeing and some I didn’t. Quite a few that I hadn’t seen in years. Oh, and it turns out, a Hertz employee found my wallet. I recovered it when I went to return the car and it had everything in it. I gave the employee who found it $20 since all the cash was still in there. He earned it. I don’t regret canceling the cards.
Of course, the story doesn’t end there. I didn’t just try to get the hotel to make it right.
Error Handling and Disaster Recovery
After not getting anywhere with the General Manager of the hotel, I figured I would try Hotels.com. I called and the representative submitted a ticket and explained the situation, including that I was rented a “Do Not Sell” room. They said I would hear within a few days. I got a response the next day and was told the hotel wouldn’t cancel. I replied restating that I wasn’t looking to cancel, I wanted a refund and sent the pictures of the room. A few hours later, the same reply came back.
One last step, the credit card company. I thought for sure they would help me. I got the service representative on the phone and explained the situation. The question asked was, “Did you stay in the room?” I said, “Yes, it was 3 AM, the hotel was booked, and they offered no other option besides leaving. Should I have slept in the car?” The reply was, “By staying in the room, you accepted the quality of the room and therefore we cannot dispute the charges.”
Wait, what? I asked her, “What you are saying is that in order for me to have gotten my money back, I had to leave and sleep in my car?” The answer was a “yes.” This baffles my mind. A friend pointed out that I should have asked, “And if my kids were with me?” Again, I said I have pictures. Apparently, Visa’s (I can’t speak for anyone else) stance is that your safety and having a roof over your head is not that important if you want your money back.
Lesson Number 7: Mistakes happen. Things go wrong. I wasn’t mad that the room was a disaster. I found it somewhat annoying but amusing as well. It’s a great story to share. What makes it worse is that three different businesses had poor error handling. The front desk clerks could do nothing. The General Manager couldn’t make it right in a reasonable way. Hotels.com seemed to not even try. I’m not sure why they don’t have something in their contract with their providers. Visa didn’t even seem to care at all. Each of them had an opportunity to make it right. I was put in a room that was marked, according to the General Manager, as “Do Not Sell” and I was. Systems cannot operate that way.
So, in the end, I have to give credit to Hertz for their employees seem to be put in tough situations but are given the ability to make things right. Sonesta Select set its desk clerks and general manager up for failure. Hotels.com seems to be a pass-through and willing to take money from customers without care. And Visa, well, I expected at least a willingness to go to bat.
Murphy sat back and chuckled. That’s all he could do. He does remind me—you had a good time, you made the flights, and you got home safe. Not everything that could have gone wrong did.