Reflections on Failure, Part Two

This post first appeared on January 17, 2022 and is republished with permission from the author.

Disclaimer: The ideas below are my own and may not reflect those of OffSec.

G. Failure in martial arts

In Part One, we discussed the unpleasantness and inevitably of failure in InfoSec, and we left ourselves with the question “If failure is going to happen anyway, how can we make it more bearable”? To begin answering this question, we will look at several other domains that I have some familiarity with. Practitioners in these domains tend to consider the emotional response to failure as more of a “solved problem” than most people in information security seem to, at least based on my exposure to all these different communities. One of the reasons for this is that all three domains are usually recreational for many of their participants, and therefore the stakes involved are lower than in the professional endeavor of security. However, that doesn’t mean we cannot learn from them and try to implement their lessons in our space. We will observe how failure tends to work out in martial arts, strategy games, and in climbing.

One of the amazing and baffling properties of Brazilian Jiu-Jitsu (BJJ) compared to many other martial arts is that when one faces a better opponent (for sufficient degrees of “better”), the opponent will win almost every match. This is especially true when we control for external variables like size, age, and sex, but BJJ is among the few martial arts where these variables are often much less important than skill – I say this as someone who has practiced many different arts for many years. The combatant who has better technique will generally defeat the one with worse technique.

This raises the question of how to practice, given that a student can statistically predict whether they will win or lose against a specific opponent. How should a new student spend their time? One of my teachers suggested a rule they called “45 & 5”:

• a. spend 45% of your time grappling with students just slightly better than you
• b. spend 45% of your time grappling with students just slightly worse than you
• c. spend 5% of your time grappling with students significantly better than you
• d. spend 5% of your time grappling with students significantly worse than you

Students who follow this protocol will find that they lose approximately half their matches, and win approximately half their matches. Most of their matches will allow them to make small adjustments: (a) will give students understanding about what they need to improve and (b) will act as a space for practicing skills leading to improvement. The remaining few matches will provide context: (c) will grant perspective on how much more there is yet to learn, while (d) will show the student how far they have already come. The important thing to note here is that a student following this advice will fail almost half of the time regardless of the student’s level.

H. Looking at strategy games

Games such as chess and go employ rating systems like Elo and Glicko to estimate the relative skill of different players. Online play has allowed match making systems to develop. These systems implement algorithms that pair players together based on skill. When a player wins a game, they are allocated a certain amount of points. Likewise, they lose points whenever they lose a game. In a large enough pool of players, participants are intentionally matched up against others that are close to them in skill level. This provides both players with a fair game, and gives each the opportunity to improve, even when they lose.

Despite the fact that players lose half their games, players that improve will slowly face and defeat better opponents and thus climb the ratings. Whereas the controlled failure of the BJJ 45 & 5 protocol is self-administered, well-implemented matchmaking systems ensure that each player loses approximately half of the time. Thus we have a curious situation: players engaging in such games completely recreationally do so knowing that they will be manipulated to fail, and yet the “next game” button gets clicked on thousands of times a day.

I. Looking at climbing

Climbing is unique among the domains we’ve discussed so far in that it is generally not competitive against others (despite its recent debut in the 2020 Tokyo Olympics ). Nevertheless, climbing has some wonderful properties that make progress easy to track. Different “routes” are allocated grades by the climbing community, or in the case of a climbing gym, by the “setters” creating the routes. The complexity of climbing makes grading subjective, but there is at least some correlation between what is considered a “hard” problem in one location compared to what is considered “hard” in another.

My experience with climbing is limited mostly to bouldering. Bouldering is typically practiced at relatively low heights, which means there is no need for harnesses or ropes. Routes in bouldering are often referred to as “problems”. Since the height of each problem is low, only a few movements are required to reach the top. For most climbers and most problems, each problem will have one hardest move: the move that the climber tends to fail on. Getting better at bouldering is not entirely identical to locating and improving at hard moves, but it is a pretty good approximation for our purposes. In particular, if the boulderer only attempts problems with easy moves that they know they can handle, they will stagnate. This is because they are not subjecting themselves to any pressure that develops better technique, stronger fingers, greater flexibility or more efficient breathing – all of which are crucial to continuous improvement.

As with BJJ and strategy games, the practitioner must expose themselves to situations in which they will reliably fail, over and over again. How then do martial artists, game players, and climbers deal with the inevitability of failure? Here I propose two mechanisms that I’ve observed in each of these domains: community and locality.

J. The importance of community

It would be naive to think that one will avoid all feelings of frustration and inadequacy during the 50% of time losing in BJJ. But because loses are balanced out by wins, and because every other student is experiencing the same failures and successes, a healthy school develops a camaraderie that empowers the whole student body to improve together over time.

Similarly, many online games (my experience is mostly with real-time strategy (RTS) ) have excellent communities where players can grow and learn together. Practice games are set up, coaching is offered, and replays of games are analyzed. In my view, the best of these communities explicitly address the negative emotions that accompany failure and help players develop strong mental attitudes.

Finally, every climbing gym I’ve entered has been a bastion of helpfulness and compassion. In my experience, climbers all over the world tend to be ecstatic to help each other out, provide insight on how to solve problems and offer moral support when a session is going poorly.

People in information security tend to have a strong sense of community, and indeed “community” is one of OffSec’s core values. But learning security can often feel lonely. If I could ask one thing of the reader, it would be to please never hesitate to reach out to others in the space for practical and emotional support – there are plenty of infosec community members who are more than happy to help out.

K. Local failure and global progress

In addition to strong communities, another property that martial arts, strategy games, climbing and infosec share is what I call locality. Losing a fight or a game, falling off the bouldering wall or failing to get a shell are all instances of local failure. But as we have seen, local failure is necessary to make global progress. If one reframes instances of failure as “just” a local phenomenon, they can reinterpret the action as a successful one, even if it isn’t a direct success.

Many BJJ students and RTS players do this explicitly. They decide that for the next match, they will not focus so much on winning but rather on executing a particular technique or on avoiding a certain error mode. When they lose (and they will for half the time) they can reinterpret their failure as a success if they were able to make progress toward their chosen goal. Likewise, boulderers might fail at a particular problem even while becoming better at a certain movement or while becoming more efficient.

Here is something you can try to reinforce this concept of local failure and global progress in your infosec journey (I apologize for the offensive focus here, but I imagine students can adapt this exercise for defensive areas as well). Set a timer for some arbitrary amount of time, say for three hours. Your goal is to attack a chosen machine and compromise it within the allotted time.

If you are able to compromise the target, then you have succeeded. Pick a more difficult machine or reduce the time period and try again. At some point, some combination of target and time period will inevitably cause you to fail. When you do, write down what you have learned during the process, and particularly what your failed attempts might tell you about the machine. As long as you can write down a single fact, principle, observation or idea you have made progress not only on this machine, but on all future machines and scenarios that you encounter. Your failure, local that it is, contributes to your global progress and makes you a better security professional.

L. Purple empathy

I won’t dive into the appreciations and concerns I have with the term “Purple team” here, but I’ll end off with an observation that I believe can help both attackers and defenders. If failure is conserved in security, you can always interpret your failure as some success. The next time that you fail to accomplish a security goal, note that you would have succeeded had your role been reversed. Likewise, notice that each success that you experience is just as fairly interpreted as a failure from a different perspective. What can you learn from that failure? I’ll leave you with one last question: what kind of empathy can we develop as a community if we acknowledge that our combined wins and loses are always equal?

---

About the Author

Jeremy “Harbinger” Miller is an Information Security professional interested in how security skills are taught, learned, and applied by individuals and organizations. At Offensive Security, Jeremy serves as Product Manager of Content Development.