I made a much worse mistake once. We wanted to use rsync to snapshot one of our file servers every evening because the tape backups we did took a long time and could only run them weekly. My boss asked me to develop an rsync script.
So I sat at my desk and after reading the man page a bit, I wrote a script and kicked it off as a test. It seemed to be humming along, so I turned my attention to other tasks.
While I was doing that, one of our users came to my desk and said “hey, the file I am working on just disappeared.” I said OK, I’ll be over to look in a minute. A few minutes later, another person came over and said the same thing, and my reply was the same. OK, I said, I’ll come check it out in a minute. When the third person came by, I started getting the nagging suspicion that something I did was the cause. When I looked at the log of my running script, I had a realization that filled me with utter horror. I had gotten the arguments to rsync backwards. Instead pf the source being the file server disk, I used the blank disk, and the target was the file server. Worse, I had used the —delete flag. As a result, rsync was valiantly trying to sync the disks by deleting all the files on the server.
An because the last backup had been a tape backup nearly a week ago, I lost the company nearly a week’s worth of work. After the company president sent our hundred or so employees home for the day “so that IT could fix the problem” she called me into her office. I was sure she was going to fire me. All she said was “you’re not going to make that mistake again?” Yes. That mistake and many others like it. I learned a lot that day.
It's a really counter-productive policy to fire people over stuff like this. It breeds a culture where employees hide their mistakes, which can cause a lot more harm than the mistakes themselves. Like repeated negligence, sure, but one-time fuckups, never.
So I sat at my desk and after reading the man page a bit, I wrote a script and kicked it off as a test. It seemed to be humming along, so I turned my attention to other tasks.
While I was doing that, one of our users came to my desk and said “hey, the file I am working on just disappeared.” I said OK, I’ll be over to look in a minute. A few minutes later, another person came over and said the same thing, and my reply was the same. OK, I said, I’ll come check it out in a minute. When the third person came by, I started getting the nagging suspicion that something I did was the cause. When I looked at the log of my running script, I had a realization that filled me with utter horror. I had gotten the arguments to rsync backwards. Instead pf the source being the file server disk, I used the blank disk, and the target was the file server. Worse, I had used the —delete flag. As a result, rsync was valiantly trying to sync the disks by deleting all the files on the server.
An because the last backup had been a tape backup nearly a week ago, I lost the company nearly a week’s worth of work. After the company president sent our hundred or so employees home for the day “so that IT could fix the problem” she called me into her office. I was sure she was going to fire me. All she said was “you’re not going to make that mistake again?” Yes. That mistake and many others like it. I learned a lot that day.