TheInvisibleSoftware

0:00

S… Speaker 2 (TheInvisibleSoftware)

Yeah, I graduated in 2013 and basically since then I've worked in automation. I got this job opportunity and I was going to the recruiter and I got this invite from this company and this recruiter said I was going to do this. And I looked at the website and I thought I had no idea what they did. Like I communicated, I don't understand any of these words. And what they did was they, well I went to the interview and they said what we do.

0:34

S… Speaker 1 (TheInvisibleSoftware)

And they said, sign this NDA and you can go into our clean room. And I did that. And I walked into the clean room and the machines were manufactured. Vision inspection systems and quality control systems for silicon wafers that are used for Intel, NVIDIA, AMD, you name it. And I was hooked. That was it. I thought, this is amazing. Canceled my other job offers, accepted their job offer, and then I worked there for a year. Maybe we'll get on to why I didn't work there longer later.

1:02

S… Speaker 1 (TheInvisibleSoftware)

After that, I worked at a company in Barton, and we do similar things. It's vision inspection, it's automation, but it's not of the silicon chip industry. It's of various things, mostly pharmaceuticals, historically food, and then some other things. All right, so the problem with giving a talk about this is with AI, I don't know what the industry is going to look like in six months, and I certainly don't know what it's going to look like in two years. I don't think anyone does, really.

1:39

S… Speaker 1 (TheInvisibleSoftware)

So I'm going to do only what I can do, which is talk about what it is now. I'll look a bit into the future, but that's all I can do. Also, I'm going to show you some stuff that I'm not sure if I should or not. I'm not breaking any NDAs, but I haven't had to mention some of this stuff. So if you don't take photos, that'd be appreciated. So what do we think of when we think of writing software and software development? I think most people think almost everything we do now is in the browser. For a lot of people...

2:13

S… Speaker 1 (TheInvisibleSoftware)

that are less technical, that's the only way they access a computer, really. There's a lot of development in there, lots of job opportunities, lots of JavaScript jobs, so front-end, lots of e-commerce, things like that. Really common, really common in the industry. We've got apps on the phone. A lot of them are still just websites. You're just wrapping it and then putting it in a framework. A lot of the interactions that you can do now on the phone. Maybe you can be a bit cleverer than that, and you can say, oh, the operating system. And yeah, of course it is.

2:51

S… Speaker 1 (TheInvisibleSoftware)

And these are some of the most complex pieces of software that mankind has made. Millions of lines, tens of millions of lines over decades that have to support hardware that's decades old and be maintainable decades into a future, which is something that a very small subset of software actually has to do. It's very impressive. You've already done think of stuff like this. So this, I was here a couple of weeks ago. So this is a machine that inspects contact lenses. So that's how your contact lens is made. And if you've got a very special contact lens...

3:24

S… Speaker 1 (TheInvisibleSoftware)

then the orientation of this is important. And so what we do is we inspect the orientation of this. We send commands to the machine to rotate it, and then we inspect it at the end. So we measure what angle it has to be rotated. We tell it to rotate it, and then we check that it has to rotate it. This is the backside of the machine. So you've got these are the robot arms that pick up the individual. It's called a flower. Pick up the individual flowers. They load them. You can just see it there. And then...

3:59

S… Speaker 2 (TheInvisibleSoftware)

It's inspected, rotated, and then they put it on these trays when they're done. So what do we do? We've done machines that check that the vitamins of chocolate is on your hobnob, that an eclair has cream in it. We've done packaging and print. And this is everywhere. So a Coke bottle, when it's made, it will be inspected when it's too small, before it's blown out. It'll be inspected there. It'll be blown out. It'll be inspected. The artwork will be printed here.

4:36

S… Speaker 1 (TheInvisibleSoftware)

The fill level will be printed, the fill level will be infected, and the cap will be infected. All of that's done, and all of that's done by machine, and all of that's done by software. The other thing that you get is you get inspections for dimensional accuracy. So the products that you have, they'll be made of different components, and all of those components will have certain tolerances. And something's got to check those tolerances.

5:00

S… Speaker 1 (TheInvisibleSoftware)

There's no one standing there with calipers measuring screws and things like that. It's done by Vision Systems. Vision Systems did this automatically with specialized cameras. And they do all of that and then they've got to accept or reject the products all automatically. So here, where's my product? After you've had an operation or something like that, you need a dressing of this size.

5:42

S… Speaker 1 (TheInvisibleSoftware)

When you think about it, it's just like a plaster. You've got, think of the layers, you've got your rear material, you've got your front material, you've got the waterproof outer, you've got the absorbent material, and then you've got the adhesive. You've got to put all of this together. And that's what this machine does. So you've got your in-feed material over here, and see that's the absorbent material. And then you've got your other materials coming from the top, up here, coming from the bottom, and coming from later on in the machine. So this is a tuck-in-the-hat side.

6:16

S… Speaker 2 (TheInvisibleSoftware)

The printer up at the top, that red light is in here from the camera inspection. So we're doing contamination inspection, so we're searching the in-seed material pad to make sure there's nothing black in it, to make sure there's nothing black in it. I haven't seen the ceiling. There's a ceiling around the outside. Because this wall, this is in our factory, which is in the video, but it will be in the clean room. And one of the things that we're going to expect for is if this product is sealed, then it will interfere with the seal when it's been sealed.

6:52

S… Speaker 1 (TheInvisibleSoftware)

And then it won't make a proper deal, you'll get back to it. The thing is, if any of these products make it small, there's a big cost. Some of these dressings are pounds. We make one, specifically, it's £1,000 per dressing. If you find that you've got a batch that's all bad because of some problem with your manufacturing, that's an awful lot of money to waste. Here's a machine that we made. So this is pretty simple. This is making...

7:26

S… Speaker 1 (TheInvisibleSoftware)

So you load in the material here. It feeds it through. We inspect the 1D barcode. And we do loads of this. This is what's called UDI. It's really our bread and butter. It's a bit boring, but it makes us a lot of money. So the information that's printed on your drugs, on your pharmaceuticals, your GTIN, your lot number, and your expiry date, that's all mandated by law. And that has to be there. And it has to be correct. It can't be wrong. If it's wrong, you've got to recall the whole thing at the point that you find it.

7:55

S… Speaker 1 (TheInvisibleSoftware)

So we check, we've got a camera on here that's checking that the product that the operator has loaded is the correct product, because you don't want to print the information for product B on the carton for product A. That's a failure mode. So we check that we've got the right carton. We then feed it through. This is the print head. And then in this box is the camera. So we print the mandatory information, and then we immediately check it. And then just over here, so you've got the distance from here.

8:25

S… Speaker 1 (TheInvisibleSoftware)

to here, to get the image from the camera, get it into your software, perform the vision inspection, decide if it's good or bad, send that result to the PLC, which actually controls the reject arm, and then fire the reject arm. Sometimes you've got half a second, if you're lucky. If you're unlucky, you might have 100%. So you might be doing 10 of these every second.

8:48

S… Speaker 1 (TheInvisibleSoftware)

And then it starts to get really hard because your camera is going to take up to 40 milliseconds just to transfer that image off. So you've then got 60 milliseconds. Seashell garbage collection can take up to 30 milliseconds, which is completely unknown to you. So you've got 30 milliseconds to do everything. And you always have to be under that. You can't be over it. There's an interesting case about this machine, which is just an interesting anecdote. But this went to a customer. And they said they gave us specifications for how big the machine should be.

9:17

S… Speaker 1 (TheInvisibleSoftware)

and we made it to their specifications. And then when we delivered it, they said, you've made it wrong, it won't fit in the elevator. So they'd got it in their warehouse, but they couldn't get it to their production floor. Well, what do you do? You've spent hundreds of thousands of pounds on this machine, and you can't get into your clean room, into your production floor. Well, you cut a hole in the building, is what you do. And this is even more than it seems, because this is a clean room in here, so they've had to construct a room inside here, then cut a hole in the building, then you crane the machine in.

9:46

S… Speaker 1 (TheInvisibleSoftware)

And then you've got to destroy the room inside and seal up and all of that. There's another machine that we made. This is making, or it's inspecting, hydrogen fuel cell material. So hydrogen fuel cells...

10:00

S… Speaker 1 (TheInvisibleSoftware)

are made up of a wound up coil of what looks like a bin bag material. And it's actually a clear layer, a black layer, and another clear layer. Something to do with anode and cathode. I don't really understand. And then they coil it really tight. And I've got a video, if I can speak through it.

10:30

S… Speaker 2 (TheInvisibleSoftware)

The way this works is it's got this camera called line-scan camera. So when you think of a camera, think of a sensor of a camera, normally it's square. This is just one pixel wide. It's 4,000 pixels long, but it's only one pixel wide. How can you get an image of that? Well, if your product is moving, and you continually fire this, your product is faster than one pixel, you can build up an image. And that's how they work. So coming up to one of these cameras now,

10:55

S… Speaker 2 (TheInvisibleSoftware)

These are lights, both left and right, and there's a camera, I think it's just off-screen, up at the top, there. And that's looking for scratches or creases. That's about to do... What we're doing is we're shining a light. There you go. It's looking for white spots on the black background. We've then got another camera that inspects the other side. So we're checking one side for scratches and creases, checking for holes all the way through, and then we're checking the other side.

11:33

S… Speaker 1 (TheInvisibleSoftware)

When we find them, we then print. We then print on it. We print a big yellow mark. And all that's for is it means that the processes later down the line, they only need to look for that big yellow mark. They don't need to identify the small scratches and holes and things like that. And then what they do is they cut it. So only a certain, they need a certain length of material to be of a certain quality for it to be useful to them. So then later machines will chop it into basically the biggest usable length that they can get between all the defects. So how is what we do?

12:07

S… Speaker 1 (TheInvisibleSoftware)

different from also the other software. We've also got limited hardware. This is a PC that we'll typically work on because space is at a premium in the factories. So you need the smallest PC you can. Sometimes we don't. Sometimes we've got to...

12:30

S… Speaker 1 (TheInvisibleSoftware)

We need bigger material. The machine for the hydrogen fuel source, that's running three 4U rack machines to process those images, because each one of those cameras is producing images that are 450 megabytes, and it's producing those something like 20 times a second. So that's an enormous amount of data that we actually had to write our own garbage collector to deal with, because the C-Shout garbage collector doesn't like that amount of data. So we've got very limited hardware. We've got a limited time.

13:01

S… Speaker 1 (TheInvisibleSoftware)

to give the result often you've got the time from the printer or the camera more accurately to the rejects gate or if you've got sometimes they're really far apart but you've got a really fast you've got many products close together i.e you've got to make a decision before you get to the next product because otherwise you start backing up so we've got a really limited time like i said sometimes 100 milliseconds or less we can't go to the cloud it's too slow but this time you send you know this is a nanosecond at the speed of light you can't change that

13:30

S… Speaker 1 (TheInvisibleSoftware)

Cloud's data centers are a long way away. It's too slow. We have to do it on-premise. We have to do it there and then. We have to guarantee our results. If you're running web dev, you might look at your stats, you might look at the 95th percentile, or maybe even the 99th percentile. You might want to know that 95% of the customers who visit your website are seeing your website within one second. And there's good reasons for that. You can't control everyone's internet connection. Some people will be able to spot your Wi-Fi, etc.

14:03

S… Speaker 1 (TheInvisibleSoftware)

But we don't have that luxury. Every single product, we have to give a result for. We can't miss it. We can never, under any circumstances, in any scenario, with any failure of any system, let a bad product go good. It's just not acceptable. And while we are allowed what's called false waste, i.e. rejecting a good product, that has to be in a magnitude of under a percent. Normally 0.2% or 0.5%, something like that.

14:36

S… Speaker 1 (TheInvisibleSoftware)

If something goes wrong, we've got to stop everything. Netflix has the very famous chaos monkey that takes down their systems during production at any random time to test the fault tolerance of their system. We do the opposite. If anything goes wrong, we want to stop everything. So everything is interconnected. And if one part has a failure or one part stops responding...

15:00

S… Speaker 1 (TheInvisibleSoftware)

everything has to stop. So we have to constantly be aware of all the different parts of our system, if they're running and if they're not, and if anything's not responding or not doing what we expect it to, we stop everything. Can't scale. If you deploy it on that PC, that's all you've got. You can't put another one in there. There's pros and cons to this. The pros are we don't have to think about scaling. When we're designing a system, we know the hardware we're designing it for and we know the load we're designing it for.

15:30

S… Speaker 1 (TheInvisibleSoftware)

There's also downsides to this in that if you get some of that wrong or the reality is not your expectations, you can't get out of jail by paying AWS more money. And we can't ship updates, which is sometimes a bit of a problem because these systems are qualified. They are checked. So when we produce a system, we test it in our factory. We run the same tests at their factory. When they deploy the machine, they'll run...

16:03

S… Speaker 1 (TheInvisibleSoftware)

say a batch, and they'll say, okay, we're going to get a human to check every single product. And then if that passes, then they'll say, for the next week, we're going to test 10% of the product. If we want to change into that, that's really hard. How can we guarantee that we haven't invalidated all of their testing? So we can't really update the system. It can be done, but it's often quite difficult, sometimes quite expensive.

16:34

S… Speaker 1 (TheInvisibleSoftware)

We also don't always have access to the hardware. We had a printer that was £300,000 to buy. We can't buy one of those just for testing. So we had to write an implementation talking to the hardware of the printer to its specification and then just deploy it in China. And it worked. Don't know how, but yeah, it worked. A lot of testing, a lot of development, a lot of reading the spec. We wrote emulators. A lot of effort went into it.

17:06

S… Speaker 1 (TheInvisibleSoftware)

We're also, to an extent, a more waterfall system than other places. Other places can try things. You can push an update to a website. You can see if customers like it or if they don't. We can't do any of that. Often when we're starting these projects, the machines aren't built. They're not even designed. The big machine that was producing the dressings that I showed you earlier, that's an 18-month to two-year build. So we can't test on the hardware until right at the end. And at the end point...

17:34

S… Speaker 1 (TheInvisibleSoftware)

The company has just invested millions of pounds in producing this machine. The last thing they want is for you to sit around for weeks or months testing it. They want it out the door and sold. So we often have quite a tight URS, which is a user requirement specification. It's where the customer says what they want, which in some ways you can think, oh, that's great, right? The customer tells you exactly what they want. You just develop to that. Of course, it's never that easy because the customer never really knows what they want. And that's fine. That's accepted.

18:02

S… Speaker 1 (TheInvisibleSoftware)

There's always things that they wanted but didn't put in the spec. And sometimes the things they put in the spec that they end up not wanting. And I think this is applicable to all industries. And I'm not going to say that there's one right and one wrong here. You've got to balance your speed, cost, and your quality based on the...

18:24

S… Speaker 1 (TheInvisibleSoftware)

based on the business that you're in. And for me, this is the difference between a programmer and a software engineer, at least one of them, is to take into account not just what you're doing, but more details about the sector that you're doing it in and what is beneficial to the business. So for some examples, something quick delivery at low cost. I had one of my friends, he worked on a website, and one of the things he was asked to do was an FA Cup promotional site.

18:54

S… Speaker 1 (TheInvisibleSoftware)

So it was up for like six weeks, something like that, and then down. You want low cost and you want it quick. You don't care if it's buggy. You can fix it. It's only going to be used once and then you're going to throw the code away. There's 100% use cases where that's the right thing to do. Sometimes you want quick delivery at any cost. That's basically AI. That's a great example of that. Sometimes you've got high quality at low cost. And the only thing I could come up with with this is open source projects like SSH that are basically maintained by two people, but top of the whole internet.

19:27

S… Speaker 1 (TheInvisibleSoftware)

And then you've got stuff that wants high quality at any cost, like the mission to the moon, aviation, nuclear. Now, in pharmaceutical industries, we're much closer to this than anything else. We're not at the same level as aviation and nuclear, but we're getting there. And so it's important to realise what sector you're in. And while these are all extremes, every company is going to fall somewhere within this balance. And it's about writing your code.

19:58

S… Speaker 2 (TheInvisibleSoftware)

to the expectations of the company.

20:01

S… Speaker 2 (TheInvisibleSoftware)

So if we look at some extremes to illustrate the point, Artemis 2 that went to or went round the moon most recently.

20:09

S… Speaker 1 (TheInvisibleSoftware)

So it's got two computers, which each have two flight control modules, which each have two processors. And these are parallelizing. They're all working in lockstep. So every single operation that every processor does, all the other processors are doing it at the same time. And then they're comparing all the results. And if any have got the wrong result, because of space radiation and all other manner of things that can go wrong, they go silent and let the others take over. The cost in doing that is outrageous. But it's worth it. Life critical software.

20:39

S… Speaker 1 (TheInvisibleSoftware)

They also have backup flight software. I found this interesting. So they've got separate hardware running a separate operating system developed by a separate team to do the same thing. And that's in case there's a problem with the hardware or there's a problem with the operating system or there's a bug in the code of the main software, they can fall back to this. I can't think of any other company that would do that. I can't think of any other scenario where that would be a viable cost-benefit analysis. But it's what they do. And then they put all the hardware together.

21:10

S… Speaker 1 (TheInvisibleSoftware)

And then they do what Netflix does. They take their whole system and simulate faults, but they do it with the real hardware, which is, again, tremendously expensive. You've got this incredibly expensive hardware just sitting there, and you're going to monopolize it by testing with all of these faults, fault conditions. And this is an interesting one. Would that code ever be written by AI? And I think no. We've seen how a lot of the AI, well, AI code is typically buggier than human written code. It took down, famously, it took down AWS in,

21:42

S… Speaker 1 (TheInvisibleSoftware)

in Asia a couple of weeks ago because of AI's shipped code. It's much faster, but it's buggy. Now, maybe NASA will use AI here and there. I'm not saying they won't. Of course they will. They'll adapt and they'll use things. But I can't see, I said I'm not going to put it in the future, but I can't see how you'd ever replace humans in this scenario. On the opposite side of the coin, I talked to a...

22:12

S… Speaker 1 (TheInvisibleSoftware)

I saw a talk from a woman who worked at an advertising company. And the way that they work is they make a change and then they put it straight to production. They have tests that take 90 minutes, but they don't want to wait for the 90 minutes before they push it to production. So they put it straight to production. And if then the tests fail and they realize there's a bug in it, they just revert. Because it's low risk and it's high speed. So for them, the 90 minutes that they get of having the change in is more beneficial to the one in 100 times they have to roll back.

22:46

S… Speaker 1 (TheInvisibleSoftware)

Will that be written by AI? Probably, right? Because you're increasing the speed that you can write it, you're increasing your change of development, but you're increasing your risk. But they're okay with that. Because they're already trading speed for the risk. So, what's right, and this is something I didn't think about and didn't realise when I was a graduate, is what's right. So, for a high quality system, you get more time for features, in theory. You get less pressure, again, in theory.

23:23

S… Speaker 1 (TheInvisibleSoftware)

You've got more stability with technology. You're not necessarily working with the bleeding edge all the time. And you get more time for testing. The cons is that for each single release, there's more pressure. You can't just update it if it goes wrong. There's less opportunity for new tech. Sometimes it's nice to try something new and do something new. And there's more testing. Testing is a blessing and a curse. For the high speed, you get to use more different features. You get to try the latest technology. You get to experiment with things. If there's a new language or whatever, you can do it.

23:58

S… Speaker 1 (TheInvisibleSoftware)

And you don't have to test as much. But there's more pressure to get things out. And you get to put less polish on peaches. So on the left side, it's the quality that drives commercial success. And this is what it comes back to. As a programmer, it is your job to develop. It's your job to bring in revenue for the company. It's your job to develop revenue for the company. Some companies, the revenue is going to be driven by quality. And that will drive their commercial success. Sometimes speed drives the commercial success.

24:29

S… Speaker 1 (TheInvisibleSoftware)

Neither one of these is wrong. It's not like writing good code is infinitely better than writing code quickly. It's not. It's about matching it. So why do we focus on quality? Well, as I said, we need to focus on maintainability because the code is now 11 years old. It's going to run for another 11 years. These machines that are in factories will be there for decades. I find that generally not everyone...

25:00

S… Speaker 2 (TheInvisibleSoftware)

Not everyone, but generally, engineers like to be proud of what they put out. Engineers will prefer to work on something and be proud of what they produce at the end of the day, rather than rush something and chuck something out at all. In general, some people prefer to move fast and break things, and that's fine. And hardware testing. Hardware testing is really hard for us. It's really hard, unless you're like NASA and you're going to fully emulate the whole thing. We've got physical hardware. How do we emulate that? We've got a machine. It's hard.

25:32

S… Speaker 2 (TheInvisibleSoftware)

Making sure that the code we put out is right and putting more checks into that has benefits. So how do we do it? Well, we focus on written code quality. How do we do that? Well, this book has the answer, published in 1984. And the answer is this. Programs must be written for people to read and only incidentally for machines to execute. If you remember nothing else about this book, remember this. I would rather have code that has a book in it that I can read than code that has notebooks that I can't.

26:06

S… Speaker 2 (TheInvisibleSoftware)

because code that has a bug in it but can be read, can be fixed, and it can be updated. If code is so obtuse that it can't be read and people can't use it, then even if it's perfect, it's not going to be perfect all the time. You're going to need a new use case. Something's going to change. Customer's requirements are going to change. The industry's going to change. You're going to need to change it at some point. Being able to change it is the most important thing. And I think that's the big difference between the code you write at university and the code you write at work. When you're at university,

26:36

S… Speaker 2 (TheInvisibleSoftware)

You're making small projects, you're writing them yourself, and you're reading your own code. You maybe spend 50% of the time reading a code and 50% of the time writing a code. At work, you'll spend 95% of your time reading a code and 5% of your time writing a code. It's monumentally important in almost all use cases that the code is readable. So here's an example of that. This is just last week. We had a problem where a hand scanner couldn't be recognised, and it took one of our best engineers four days to debug it. That's the thing. It's a single line.

27:10

S… Speaker 2 (TheInvisibleSoftware)

It took four days to come up with that change. Four days of reading code for, what, 40 seconds of copying and pasting the above line onto there and changing what's required. That's where code has to be readable. Because writing that is easy. The AI can do that. What the AI can't do is the four days of reading it and figuring out where the problem is. So how do we do it? Well, we do static analysis, which is checking the code automatically.

27:41

S… Speaker 2 (TheInvisibleSoftware)

Seeing if there's anything wrong with it. We're really tough on this. I haven't seen anywhere else this hard on it as we are. If you have a white space at the end of your line, that'll build error. That'll fail our build. If you have an extra new line, that'll fail our build. If you have braces in the wrong place, that fails the build. Because we're prioritizing written, the writing of the code. You can put it all in one line, the compiler would do it just fine. But we don't do that. Why? Because someone's got to read it. So it's important. We do PRs.

28:11

S… Speaker 1 (TheInvisibleSoftware)

So everyone's work is checked by everyone else. We unit test. That's what we do. We've got quite a high level of testing. And then we test on the machine when we're done. And then, let me think. So now I can maybe demonstrate. So these are our PRs. This is live. This is PRs as of right now. So when someone does a piece of work, it goes in, and then two other people have to review it. That's a choice by us to have two people review it.

28:51

S… Speaker 1 (TheInvisibleSoftware)

Does this one have a comment? Yeah, so here we go. This has failed. This has failed the build. Why has this failed the build? See? What if we failed it? A local variable is never used. If you declare a local variable and you don't use it, that fails the build. Why should that variable be there? Why are you writing code that's not used? Fails the build. That's one way that we enforce quality. We've got warnings. Yeah. A non-knowable field must contain a non-new value when exiting constructor. This will fail the build.

29:37

S… Speaker 1 (TheInvisibleSoftware)

And other people have to look through the code and give questions, give comments. So this is a great example. Everyone makes mistakes. Someone's put the same line in twice and then someone...

30:00

S… Speaker 1 (TheInvisibleSoftware)

else and that's the thing that when you're writing it it's really easy to do when someone else comes on and reads it it's easy to spot it also gives it also gives an opportunity to check the very base assumptions that you make about it that are your are your base understandings are you doing the right thing basically because sometimes you can write the greatest code but if it's the wrong thing for the business if it's not what other people think you should be doing then it's the wrong code and sometimes that's the biggest time thing

30:47

S… Speaker 2 (TheInvisibleSoftware)

So, AI. I'll read this. So, I asked one of my developers who uses AI the most how he does it and what he does. And he said, well, I'll show you first. So, how do we do it? So, we had a requirement to add, to change the way permissions work. I won't go into the details, but...

31:32

S… Speaker 1 (TheInvisibleSoftware)

Effectively, we wanted to write a local permission system. And so the way he did it was he produced this. So this is handwritten. So this is a handwritten description of the problem and what we want. And he's got pretty good at this now, so he knows what to put in it. So we've got a general overview. We've got requirements of what we want it to do.

31:58

S… Speaker 1 (TheInvisibleSoftware)

We've got this. This is really good for the AI. This is a hint as to where to look. Bearing in mind, we've got like 400,000 lines of code. We can't put 400,000 lines of code into the AI and it figure out where things are. We've got to tell it. We've got to help it. And there's different ways you can do that. You can write kind of maps for the AI that it can read to kind of navigate through. And we do that. But here, he's just given it these files. Basically said, look at these files. And then he's given it some tasks.

32:30

S… Speaker 1 (TheInvisibleSoftware)

of what you might want to do. And then what he does, check my notes. So you feed that into the AI with a prompt that often uses the brainstorming superpower, which is a superpower within Claude. And he gives it a prompt that is roughly, help me expand this plan, ask me questions and make suggestions about it until I say the plan is complete. And then you end up with something like this. So this is now AI written based on this input.

33:02

S… Speaker 1 (TheInvisibleSoftware)

So we write this, we give it to the AI with a prompt, and we get this. I thought only while doing it that this could easily be a piece of coursework. Just throwing it out there. And then we get this, which is AI written. And so there's more detail in this. So it's giving an overview. It's talking about how it's going to do it. It's referencing specific types in classes. It's coming up with objectives and requirements.

33:39

S… Speaker 1 (TheInvisibleSoftware)

what it's got to maintain. So we've already got existing functionality in there saying it's got to maintain that, not rewrite it. And then it's coming up with a plan of what it's going to do. And then it's going to talk about how it's going to test it. So it's going to write all these, it's going to test that all this functionality is still there after it's done it. And then once it's produced this, we can modify it. Because it might be that then we read this and we think actually some of that's wrong.

34:10

S… Speaker 1 (TheInvisibleSoftware)

And it's not that it's bad or it's done anything wrong. It's that basically what we're asking it to do is we're giving it a bit of information. And we're asking it to extrapolate that and guess what we want. This is more detailed than the other one. And we haven't told it that information. It's inferred all of that through the code. Some of that's going to be wrong. And so at every stage of this, we have to tweak it. We have to read it manually. Don't just dump it in and then let it go. That's a disaster. We've got to read it manually. And you've got to be an engineer to do it. This isn't vibe coding.

34:38

S… Speaker 1 (TheInvisibleSoftware)

If it says something in here that's wrong, if it says something in here like, oh, I'm going to delete the existing local authentication provider and replace it with this one, well, we think, no, that's not what we want. So we've got to come in here and change that. So then, yeah, you manually edit the file to make changes. And sometimes those changes are only...

35:00

S… Speaker 1 (TheInvisibleSoftware)

apparent after you try and run it in the next step. So you might try and run it in the next step, it does something completely wrong, and you realize, oh, now I understand what it's thinking about. So then we feed what we've just seen, the AI version, with the prompt that basically says, can you generate step-by-step tasks for me for a human or an AI to use based on this document? So we feed it this document, and then what we get is this, which is probably the best specification I've ever seen in my life.

35:33

S… Speaker 1 (TheInvisibleSoftware)

It's 600 lines long, and it's fantastic. I think you could give this to anyone, any competent developer, and they could implement this feature. It contains all of the information you need to know. And bearing in mind, all we wrote was this. This was human written, and then at the end of the process, albeit with manual reviews along the way, this is what we get. We get 600 lines of how to implement the feature in detail. It includes dependencies.

36:04

S… Speaker 1 (TheInvisibleSoftware)

It includes a list right at the end of the steps that the AI has to go through. And then it ticks them off as it goes through them. We'll see that at the end because it hasn't finished this task. So it's broken around tasks. It's worked out the dependencies. It's actually incredibly intelligent of it. And then for each piece, it's got its goal, what it's going to do, how it's going to check it. And then we give this to the AI.

36:42

S… Speaker 1 (TheInvisibleSoftware)

And we say, please implement step one. Please implement R1. Please implement I2. And it goes and does it. And one of the ways that we found to do this is we use an expensive, low, highly performant model, i.e. Opus 4.6, to do the planning. And then once you've got a plan that's this detailed, you can give that to something quicker and cheaper. You can give that to Haiku or we use Gemini Flash.

37:15

S… Speaker 1 (TheInvisibleSoftware)

It was Gemini 3 Flash all the time. You can give this to Gemini 3 Flash and you don't need Opus. You don't need to pay for all these tokens anymore because the specification is so good. Then if we go right down to the bottom. So this is its progress. So it's done these things and it's still got these things to do. Effectively, it's implemented the change and now it's going to test it. And that's done step by step. And each step is run with, like I said, a more basic model.

38:02

S… Speaker 1 (TheInvisibleSoftware)

And then we look at the output, we look at the diff, and we see, is it good, is it not? Has it done what we expected it to do? Or has it done something different? Has it done it the way that we want it to do? Because sometimes it does something, and we've had many debates internally about this, it does something and it outputs some code. And we think, well, it works, but no one can read it. And we've had debates about, well, is that important? And...

38:31

S… Speaker 1 (TheInvisibleSoftware)

Going back to my first slide, at the moment, the answer is yes, because humans still need to read it. Maybe we get to a future where no human is reading the code, and so maybe it doesn't matter. But at the moment, humans still need to read it. And AIs need to read it. And readable code is not just readable for humans, it's readable for an AI as well. If you have a 2,000-line class, and you feed this into an AI, A, you're going to consume more tokens, and B,

38:56

S… Speaker 1 (TheInvisibleSoftware)

The more unrelevant things you have to the task that you want it to perform, the less accurate the AI is going to be. So if you can split your code into more manageable sections, which is exactly what you do to make it more readable for a human anyway, then you make the AI more efficient and you make its results better. So, yeah, we feed in each one of these. We check the results. We make any tweaks if necessary manually or by asking the AI to do it. Or we can say...

39:26

S… Speaker 1 (TheInvisibleSoftware)

You know, yes, you implemented it like this, but can you implement it like that instead? Hopefully those things would have been caught in this plan because you'd have read through each of these sections before feeding it to the AI, and if it's going to do something grossly wrong, you'd have caught it at that stage. And then the result, you get... There we go. This is the change. This is the change that it's done. So it's implemented these files. It's figured out stuff that we didn't ask it to do.

39:54

S… Speaker 2 (TheInvisibleSoftware)

So it's figured out that this is an existing authentication provider that we have.

40:00

S… Speaker 1 (TheInvisibleSoftware)

It's going to have to deal with that. It's done that. It's added new classes. And this is the bulk of what it's done. And then we've got to actually run this and test this. Just to make sure that it actually works. It's going to write unit tests. But again, it's testing what its understanding of it is and what it's implemented. It doesn't necessarily know how we want it to function at the end. So I think, what else has he said here? Yeah.

40:36

S… Speaker 1 (TheInvisibleSoftware)

So we use the more complex model to do the planning. We can use a simpler model to do the thinking. We use the Superpowers plugin for brainstorming, and that helps you with prompting for ideas, and you kind of go backwards and forwards with it, and you build these big documents. Yeah, you give it a lot of information in the prompt. The more vague you are, the more it has to infer, because essentially that's what you're doing. Code is a...

41:09

S… Speaker 1 (TheInvisibleSoftware)

minimal list of what you have to do to get this functionality, right? So when we're explaining something in human terms, we're missing out a lot of that context, and it's inferring it, and it may get too far. Yeah. So this is how he does it. He has Claude, Gemini, and Copilot running all the time. And the downside of having them write tests is that they are writing against the method, not the intent of the method like we would. Because it doesn't necessarily know why it's doing what it's doing. That's where we are.

41:46

S… Speaker 2 (TheInvisibleSoftware)

That's where we are now. That's all I've got for you. If there's any questions...

TheInvisibleSoftware

خلاصه

از هوش مصنوعی در مورد این رونوشت بپرسید