Coding was hard and complex but fun 10 years ago. Now, it’s just fun thanks to recent developments.
Five months ago, when I first saw DevinAI, I was shocked, but I also knew it would be the next big thing after LLMs like ChatGPT, Claude 3.5, and Gemini.

As I predicted, we saw Devika, a free version of Devin. They all had impressive features, but the accuracy wasn’t great — until I saw Genie’s chart, which is truly amazing! Let’s dive in and explore it!

The Rise of Genie: Background and Vision

 

Ref

Genie is created by Cosine and is presented as the world’s best AI software engineer. But according to what? According to SWE-Bench, a test designed to assess the coding abilities of LLMs.

Ref

As you can see above, even GPT achieved only 12.47%, while Cosine scored 30.08%, which is almost 2.5 times better than GPT. This is really impressive. Now, let’s look at the specific use case they presented.

Github-Repo

 

Reference

As you know, GitHub is a platform where developers can share their code. In open-source projects, users who rely on these libraries or tools can report issues they encounter.

Step 1- Select The Trigger

In the initial step, you need to select the trigger, which serves as the starting point for your autonomous software engineer.

In this real-life example, they have selected the Github issue, by giving link of it.

Now Genie has found the Github issue.

Ref

It is time to identify the issue and find the related files. As you can see it finds the files and plans what to do.

 

 Ref

 

Step 2 — Execution

Now next step is to execute the code.

 

Ref

 

Step 3 — Testing

Okay, it’s time to test the code it writes.

 

Ref

 

Step 4 — Debugging

In the next step, if it finds an error, it will debug it — but how?

 

Ref

It will go back to Step 1 to plan the scenario, then went to Step 2 to execute the code, and finally move to Step 3 to run the tests.

Now it has solved the issue.

Ref

As you can see, it solved an unknown issue involving 2 files and 2 commits, and tested everything by running 17 tests — all in just 84 seconds.

Final Thoughts

As you can see, it works impressively well. However, I would expect more impactful examples, like those Devin provides — such as solving Upwork gigs using this tool.

If you want to keep up with these changes, consider being a paid subscriber to my substack, here.

Here, we offer special GPTs to our followers, data projects done with LLMs like ChatGPT, a weekly AI newsletter, and more!

Reference

Here are the free resources.

Here is the Prompt Techniques cheat sheet.

Here is the ChatGPT cheat sheet.

Here is my NumPy cheat sheet.

Here is the source code of the “How to be a Billionaire” data project.

Here is the source code of the “Classification Task with 6 Different Algorithms using Python” data project.

Here is the source code of the “Decision Tree in Energy Efficiency Analysis” data project.

Here is the source code of the “DataDrivenInvestor 2022 Articles Analysis” data project.

 

“Machine learning is the last invention that humanity will ever need to make.” Nick Bostrom