Expert Talk: Code Refactoring • Adam Tornhill & Christian Clausen

This interview was recorded for GOTO Unscripted at GOTO Aarhus.
gotopia.tech

Read the full transcription of this interview here

Adam Tornhill - Author of "Software Design X-Rays" and Founder & CTO at CodeScene
Christian Clausen - Author of "Five Lines of Code"

DESCRIPTION
When do you refactoring your code? Learn from Christian Clausen, author of "Five Lines of Code" and Adam Tornhill, author of "Your Code As A Crime Scene" and "Software Design X-Rays” how to analyze your code to understand its need for refactoring. Furthermore, gain an understanding of the social side of refactoring and its implications.

RECOMMENDED BOOKS
Adam Tornhill • Software Design X-Rays
Adam Tornhill • Your Code as a Crime Scene
Christian Clausen • Five Lines of Code
Adam Tornhill • Lisp for the Web
Adam Tornhill • Patterns in C
Martin Fowler • Refactoring
Maude Lemaire • Refactoring at Scale

Twitter
Instagram
LinkedIn
Facebook

Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket: gotopia.tech

SUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

Transcript

Intro

Christian Clausen: Hello, welcome to the GOTO office. My name is Christian Clausen. I'm the author of the book "Five Lines of Code" about refactoring. And I'm here with Adam Tornhill.

Adam Tornhill: Hi, everyone. I'm Adam Tornhill. My background, I have a degree in engineering and psychology. And what I do these days is pretty much trying to combine these two perspectives. I've written a couple of books about it. "Your Code As A Crime Scene" and "Software Design X-Rays."

Christian Clausen: Yes. They are great books also. I wanted to talk to you about this social side of tech I heard you talk about. Also because your take on refactoring and code quality and stuff is entirely new to me, at least I've never seen anyone do forensic analysis, which is interesting. So maybe you could start by saying something about that.

Adam Tornhill: Sure. I think it's a perspective that we need to talk much more about. Because one thing quickly learned is that there is so much more to code complexity than just the code itself. We have this whole social dimension, like the current team, are they familiar with the code? And to me, one of the great tragedies of software design is that organization that builds the code, they're actually invisible in the code itself, right? We cannot pick up a piece of code and say, is this a major coordination bottleneck for five feature teams? Or is it a module that's written by just one developer? So we have a massive key personnel dependency. That dimension is simply missing.

Code forensics

Christian Clausen: Definitely. And there is so much about the social part of the code. It's humans working with the code, and it's them who need to refactor it, and there are managers, and there are all of these different things. Actually, about the manager, it was a very interesting thing when I was seeing your talk earlier, that you have these yellow, red, and green categories for code. And then you can use this to show to managers to say, "Look, we have all these red parts, and that's bad. We want all these green parts." What I would expect to happen in that case from the managers I've met in my career is a race to yellow, right? They don't want to be the worst. They don't want to be green, either. That seems wasteful, right? Or at least that's my experience.

Adam Tornhill: Oh, yes. That's interesting. You could well be right. I think the challenge is that if you have a red code and want to move to green, it's a long, long journey, right? It's a lot of effort that goes into that. I've worked with a customer just a couple of months ago, they actually spent a year trying to prove the health of their code. I think they managed to get it up by 25%. So I don't have the absolute numbers, but what was so interesting was that they also measure the amount of unplanned work that they had. They use that as kind of a goal, can we reduce the amount of unplanned work? They measure of like an 80% reduction in unplanned work by doing all these refactorings. So I find that really, really fascinating that code quality is something that impacts every business. And yet, it's something that we underestimate the importance of it, in my opinion.

How to deal with management & refactoring

Christian Clausen: Yes. So there was both that they underestimate what the value is and it's been really hard for, yeah, so I go and coach teams to do it. It’s really hard to argue to the managers and to the management teams in general that we need to invest more in these things. I had the question, how do you actually deal with people? How do you tell them that you need to implement more time for refactoring? And I just said, "Well, we reserved 20% of the time, every Monday in particular." That's my experience based on the DevOps handbook where it says 20% should be used on just status quo, not even improvement, that's just what you need to stay still. Do you have any experience with that?

Adam Tornhill: I do. I do. And I think I've seen it work. I've seen a lot of organizations tempted to do things like having a technical debt budget. And I think it very much depends on what you do with that time. During your talk, you pretty much use that Monday for allowing people an opportunity to develop their skills, and to do self-improvement, and that's something that benefits everyone, including the codebase. But I've also seen organizations where staff like, yeah, paying down technical debt was mandated from above, without necessarily giving the skills and the toolset that the developers need. What tends to happen then can be disastrous. So I've seen organizations that run some static analysis tool, then end up with a list of 8.000 major issues. The first thing that happens is you put a quantitative goal on it and say that, okay, over three months, we got to reduce this from 8,000 issues to 5,000. Have you ever seen something like that happen too?

Christian Clausen: I've seen it with warnings. When you turn up warnings the first time people are like, "Should we look at those?"

Adam Tornhill: Exactly. But that's my experience too. The first thing that tends to happen is that I mean, improvement possibilities, they just get thrown out, no one even looks at them, right? Then you have warnings, we don't have time for that, so you focus on the major issues, the really critical stuff, and even then you might have falseness of it. I think without taking this proactive approach that we're doomed. I think there was no way we were going to solve that.

Christian Clausen: When we're talking about the social side of the refactoring and stuff, also having just a number show up, that's 5,000 problems or something, right? Or 99 plus, or whatever. Whenever it gets too big, humans are like, "Oh, I'm not going to deal with this, it's too many. Why even start, because it's not going to make an impact?" And that sort of sets us further back, right? Because then you just turn off the warnings, or you annotate your code to just not do those types of warnings, or we don't use that or throw out tools. I've seen people just stop using tools because they gave too many warnings and errors.

Adam Tornhill: Exactly. It's where we're common. It's very much, I mean, living with that amount of issues it's very much like a broken window, right? So I always kind of compare it to, going from 5,000 issues to 3,000 or something like that, it's very much like choosing to jump from the third instead of the fifth floor, right?

Christian Clausen: Yes.

Adam Tornhill: Most likely, none of them is a good idea. The reason I say this is because taking a purely quantitative view like the most important thing, and that's the business impact, right? Maybe you don't have to fix all 5,000 issues. Perhaps we just have to fix five of them. But there's no way of knowing, right? Because if you just look at the code, you get, in my opinion, a very limited perspective, because the code itself cannot tell you anything about the interest rate or the business impact.

Christian Clausen: Yes. For me, I've had situations where one of the warnings actually turned out to be hiding a much worse issue, but because we weren't looking into warnings at the time, we would just have that go unnoticed and we had this huge bug. And when we started looking at the warnings, we found errors afterward which were just amazing.

I've started treating all warnings as if it's the worst error I can have because they might be hiding something really important. I take warning signs really, really seriously. I'm really scared of that.

Adam Tornhill: Yeah. I mean, it's virtually free to treat a warning as serious if you do it early on, right? And for all the new code you write, definitely. And then gradually what do you do with existing code, you gradually try to enable a warning style or?

Christian Clausen: Yes. I would say, well, I actually make an error budget, and then I have the trend go down. As you said it's more important that we're actually doing something to get it down and stop the bleeding so that it doesn't grow at least. Then over time, you shave off a little. And also, I'm very big on the human side, so I make sure to celebrate with a lot of cake and drinks or whatever your fancy is. Celebrate whenever something has gone right even slightly, and also celebrate when something's gone wrong, right? So, long as you learned something, I think it's really important.

Adam Tornhill: Yes, definitely I agree with that.

Christian Clausen: You actually have a code base because you have a product company.

Adam Tornhill: I do. I do indeed have a code base.

Product refactoring in CodeScene

Christian Clausen: How do you do product refactoring in CodeScene?

Adam Tornhill: So, CodeScene is fascinating because we have been using CodeScene on CodeScene from day one. It's multiple reasons. First of all, I'm a big fan of eating our own dog food. I think that's very important. And being software developers ourselves and making tools for other software people, we are pretty much in the target audience. So it kind of helps. I think this is where we're much underestimated. I used to be a consultant back in the day. And one thing I always found missing was that my onboarding when I got to a new client was very much focused on the tech and very little on the actual problem domain. And my experience is that the more we understand the problem domain, the better we can optimize what we do.

So I think that's really important. That's why everyone in CodeScene uses CodeScene. The second reason is that we're all a pretty small company. So, our competitors, might have like 10 times the engineering department that we do, so it's very clear that we cannot compete by doing the same thing as they do, right? So, for us as a small company, we simply cannot afford much technical depth. So we need to keep a really, really healthy codebase.

Christian Clausen: So, it's all green, no red?

Adam Tornhill: No red. A few yellow dots. But I think as long as we're aware of where they are, and as long as they're stable and well-understood, then we can live with that, right? Once we get to that area, then we make sure to not make it worse. I think that's my number zero rule for managing technical depth. No matter where you start, don't make it worse. If you do that, then you're already doing better than, I would say, 95% of our companies.

Christian Clausen: How do you budget for refactoring and improvement in general?

Adam Tornhill: We don't. We don't.

Christian Clausen: People just do them, automatically?

Adam Tornhill: People do it automatically. And it works pretty well. I mean, we're a small team, and we have this culture, where we really value healthy code. But we actually started to think about that a while ago that...I like what you said that you know, you celebrate the wins. Even if it just meant that something broke and we learned something, I think that's really important. So, one thing we start to work on a couple of months ago was that, can we identify all the successful refactorings in a code base and put them up there under codes and the dashboard. Because I think there's so much we can learn from each other. So I'm a big fan of refactoring. I'm a big fan of the refactoring books that you mentioned in your talk. I also think that there's value in having refactoring examples from our own code because it's code written by someone on our team, right? So it's very likely, I would say, it's in the same domain, but it's also in the same style that we write code ourselves. So that has been an interesting experiment as well.

Functional programming and refactoring

Christian Clausen: Yes. And you mentioned to me beforehand that it's a very functional codebase, like Clojure, right?

Adam Tornhill: Yes, it's Clojure.

Christian Clausen: Yes. So how do you find people who can do functional programming?

Adam Tornhill: It's interesting. When I started CodeScene in 2015, one investor I spoke to asked me that, "Hey, we're using this really odd, strange language that no one has ever heard about. If you will use Java, you can hire 10 people tomorrow." The thing is, I don't necessarily want 10 people, right? So it turns out that Clojure, not only does it allow each person to kind of do more than what we could have done. And with the larger team, we also have all the coordination aspects. So as long as you can avoid it, it's a win. And I found that Clojure is actually like a hiring magnet, right? The people who know Clojure, really, really want to work with Clojure, because the drawback of learning Clojure or functional languages, in general, is that you never want to go back to what you used before, right? It's a one-way street. So we never had that problem.

Christian Clausen: Ok.

Adam Tornhill: And I think it's also kind of a good hiring filter because if you know Clojure, then you'll most likely learn it in your spare time, and means you're really, really interested in this stuff.

Christian Clausen: Yes. Also, Clojure is getting a great grip. A lot of very good names are recommending it at conferences left and right as the new thing. Although it's dynamically it's acted in life famously it's not a famous dynamic two-type language. Coming from a type theory background, it seems very difficult. It seems you need a lot of discipline. And if there's something that I'm not trusting people with is having discipline. Especially also because I exactly try to build systems where humans' default behavior is the right thing that I want actually also on their worst days. I would find people would tend to skip on stuff like using dynamic types and just I didn't quite get the tests for this and stuff. Do I test for CodeScene?

Adam Tornhill: Yes. We pretty much have tests on multiple levels. And I mean, it's vital no matter what programming language you use. I'm a big fan of things like test-driven development. We have the integration tests, and then we have a bunch of end-to-end tests that run on the whole thing that kind of complement each other. But I think for dynamic-type language, I could never imagine doing that without having tests. I don't know how we should hold it together.

Christian Clausen: Yes. No, I would expect that in dynamic-type testing that's way, way important because you need something to keep the code in place, right, when it started drifting, which is where I would usually use types or even proofs if I'm being academic.

Adam Tornhill: Yes.

Christian Clausen: Yes. It seems very difficult to get some sort of IntelliSense, and all these different refactorings where I want to move stuff and just check that it still works without having to change the tests, which is what I usually find also. You must have also met the problem of, you know, a lot of people end up even if they write tests, they write tests that don't support refactoring. They actually hinder it because they're too tightly coupled to the structure.

Adam Tornhill: Yes, I've seen that so many times. In fact, some of the worst technical depth that I've ever seen has not met the test code. I think we, as developers, we make some kind of mental distinction between application code, which we know is important, and keep it clean, nice and tidy. Then we have the test code, where, frankly, we're happy if we get time to write any of it at all. And I think it's really dangerous. And I've had so many arguments with people telling me that it's just test code. But it's never just test code. Because if it a test lacks in quality, it will hold back your efforts. And the example you gave is really good. Tests are supported to simplify and even make refactoring possible, right? And if the tests are extremely tightly tied to the implementation, then they're actually hindering you. And there's very little use in having a test that breaks the moment you share some internal detail in your code.

Outro

Christian Clausen: I think we could talk about this for hours I can feel because my advice would always be if your tests are too complicated, you should refactor the code so that it's easier to test. That makes the test... Usually, I always attack the code and never attack the test. But I think we're about out of time, more or less. It's been a pleasure talking to you, Adam. Thank you very much for your time.

Adam Tornhill: Really nice to meet you. And thanks a lot for that wonderful talk today.