My new role: AI Babysitter

How you goin’ AI?

Over the past couple of years AI has truly been hitting its stride and we are now using it in ways that were a pipe dream just a short time ago. LLMs are able to parse large repositories of code and add meaningful additions which are compliant with the models and methods that we have already been using. It can also remove the drudgery out of a lot of common replicated tasks. If we look at software development, we are generally fighting to maximize the following fives axes budget, time, quality, functionality and performance. In terms of budget the costs of utilizing AI are minimal and the cost savings of increased developer performance greatly outweighed this. From this it logically follows that there is a time saving as well. In terms of functionality and performance, except for some types of tweaking, these can be done manually or AI with the same level of proficiency however these improvements can come at a cost. If one is unaware this cost is quality. 

The Quality Concern 

The effect on quality can have a number of different types of impact. One is the impact on the code base can be significant often AI will do what is what is most common and not what is most needed. It will often hallucinate code that does not need to be there, and it will often hallucinate logic conditions which are incorrect. For instance, I spent a considerable amount of time looking for a pair of curly brackets that AI had just decided to insert into one of my lines of code. This is the type of change that a human would rarely make. 

Quality can also be affected by AI not realizing that there are potential areas where a piece of code can be reused, which can lead to a bloated code base. Additionally, as AI assist us in generating more and more lines of code, each line can represent a security risk, especially if it is not fully understood.

Finally, and perhaps most significantly AI can be used as a crutch on which a developer can create code without really understanding it. As such when it comes time to review the code best practices are not followed because they are not understood. 

Mitigation

So how do we mitigate this?  Well, the first thing we can do requires a slight change in mindset.  Perhaps don't think of yourself as a software developer but a part time software developer part time AI babysitter. Like a child AI never thinks in terms of what people really want, it only thinks in terms of what it has explicitly been told to do.  here is no nuance. There is no filling in the blanks. As such quality assurance will be key. Tests and code reviews will need to be more stringent. During these reviews it is important to understand why AI may have created code the way it did. It may have done something in a way that is an improvement on what you could have done, or it might just be garbage. At any rate you need to fully understand what is going on and not just rubber stamp whatever it may have served up.

Happy babysitting