Introduction

Generative AI has dramatically shifted the software development landscape. As AI tools and techniques become increasingly integrated into the development process, they offer unprecedented opportunities to enhance productivity, creativity, and efficiency.

In this article, we will discuss the use of AI in the software development process and best practice.

TLDR：

1. Coding AI can be split into three types depending on the situation.
2. There are already solid options for “copilot in your IDE” available on the market.
3. “Agent inside your workflow” can hit a decent accuracy level in specific scenarios.
4. “Agent outside your workflow” helps developers broaden their skill set.

3 Types of Coding Agents

For developers, there are generally three types of daily tools:

• IDE: The integrated development environment used by developers for coding. It is where developers spend most of their time each day.
• Workflow: Development workflow tools like GitHub, where developers collaborate with their teams.
• Browser: When developers encounter problems they can’t solve on their own, they turn to the internet for help.

Based on different usage scenarios, we categorize AI coding tools into three types: copilot inside your IDE, agents inside your workflow and agents outside your workflow.

Copilot Inside Your IDE

In June 2019, GitHub introduced Github Copilot bata. Since its launch, Copilot has become synonymous with AI-powered coding assistants. It helps IDE users by predicting their coding actions, making the development process more efficient.、

Scenario

When you are working in an IDE and have typed a few keystrokes, Copilot predicts what you are likely to input next. This is the core experience of Copilot.

Beyond this, today’s Copilot has grown to include even more features:

1. Modify multiple lines or files at once.
2. Make suggestions based on your recent changes and linter errors.
3. Have an ongoing chat conversation to ask questions about your code and iterate on coding ideas and concepts.

What Copilot completes for you might not be entirely correct, and you may still need to make adjustments or add some detials, but it truly helps save a lot of time. It’s like having someone write out your ideas for you as soon as you think of them.

If you’re not familiar with coding or software principles, Copilot might not be that helpful. It’s really meant for professional developers, you still need to know what you’re doing. But for those who are experienced, Copilot tools are mature enough to offer a lot of assistance.

Product

Category	Github Copilot	Cursor	Windsurf
Price	Free trial $10/month Pro $19/month Business $39/month Ultimate	Free trial $20/month Pro $40/month Business	Free trial $15/month Pro $60/month Pro Ultimate
Provider	Github	Anysphere	Codeium
IDE	Supports popular IDEs	Cursor	Supports popular IDEs
Model	GPT-4o	GPT-4 Claude 3.5 Sonnet GPT-4o	cursor-small GPT-4 Claude 3.5 Sonnet Llama 3.1 70B

Github Copilot

Github Copilot is an AI-powered IDE plugin designed to assist developers in writing code more efficiently. It supports VS code,Visual Studio, JetBrains and so on. Github Copilot is currently the most widely used Copilot product, with a total of 1.3 million developers and 50,000 enterprise subscribers.

Cursor

Composer is an AI code editor. It helps developers explore code, write new features, and modify existing code.

Cursor’s most well-known feature is its “tab, tab, tab” which means you continuously accept code written by Cursor. It’s a very enjoyable experience.

Compared to previous Copilot products, Cursor offers a standalone editor for a better experience, but it also comes with a higher trial cost.

Case Study

• Use Cursor for Prediction of Next Lines

The primary feature of copilot as Cursor is to predict the next lines of code when developers are typing.

Cursor predicts next block of code

• Use Cursor for Prediction of Next Action

One of the important feature of cursor is to predict user’s next action. For example, when the user add a field to an object that is not defined in the object type, cursor will predict the user’s next move is to modify type definition to add the type.

Cursor predicts the next move is to add field definition after line 9

Cursor predicts the content to be added when tab is pressed

• Fix Multiple Files in One-shot Based on Error Message

It is common that a big change may breack a lot of related files such as test cases. Submit the test result to Cursor and ask Cursor to fix the problem. It will give editing suggestions for multiple files. Users can accept/deny the suggestions one by one.

Future

Copilot products have been developed for over 5 years, and the improvement in product experience is inseparable from advancements in technical capabilities. The chart below shows the changes in AI performance in the code generation scenarios of Copilot evaluations.

However, we can see that the accuracy of model capabilities in the current scenarios has reached a relatively high score. We can consider that Copilot has achieved a stable performance in the current scenarios, making it suitable for adoption by any company.

Agent Inside Your Workflow

In a developer’s daily work, they get an issue (like a feature request, bug fix, or change of configuration), work on it, and then submit a Pull Request(PR), which moves the workflow into the testing and release phase. AI can handle a lot of tasks throughout this process.

But before we discuss how AI can integrate into our workflow, we need to address one question: how should AI and humans divide tasks in future software development?

AI-human labor division

What skills and knowledge does a developer need when completing an issue?Then we will discuss if AI can deal with it.

We divide skills/knowledge into three types:

• Business context: Knowledge or logic related to your business, such as the logic for adding items to the shopping cart if you’re building an e-commerce website.
• Engineering context: such as your tech stack and your development environment.
• Technical problem: For example, how specific algorithms are written and how interfaces are implemented.

Since business context includes a lot of private and scattered information that is difficult to learn, it is the most challenging area for AI to address.

Engineering context is somewhat easier than business context, but older projects often suffer from missing documentation and require feedback from specific environments. While AI can theoretically address these issues, it would need extensive product engineering adaptations and human input to be truly effective.

In contrast, pure technical issues, which are the furthest removed from specific business problems, are the easiest to address.

Therefore, it’s unrealistic to expect AI to completely replace humans, as the business context that humans understand is irreplaceable in AI’s current evolutionary path. However, AI can be quite effective in addressing technical issues that are further removed from business concerns.

Scenario

The scenarios here are dispersed throughout the workflow, as illustrated in the diagram below, but not every scenario currently has a corresponding product.

Compared to scenarios within an IDE, these agents cover scenarios that require more stable code versions and can incorporate feedback from the environment. These products function more like a coworker than a copilot.

These products typically exist as GitHub or GitLab apps. For instance, after you submit a PR, the AI can assist by reviewing the code, adding tests, or offering suggestions based on feedback from the environment(e.g. CI/CD pipeline).

As AI agent products are maturaing, more repetitive tasks will be replaced by AI, enabling developers to spend less time on routine tasks and devote more attention to creative work.

However, such an idealized future won’t materialize within a year. Software engineering is a highly non-standardized industry. The chart below shows scores from the joint release of the OpenAI and SWE Bench rankings. Compared to Copilot in your IDE, current products still do not meet commercial standards and often need to sacrifice success rates or limit the scope of application to enhance performance.

Product

These products serve different purposes and are not in direct competition, allowing the team to use multiple products simultaneously.

Product	Test Gru	Devin	Code Rabbit	Dosu
Scenario	Unit Test	All kinds of PR	Code Review	Documents
Price	Free trial $19.9/month	$500/month	Free trial $12/month lite $24/month Pro	Free or enterprise
Provider	gru.ai	cognition.ai	coderabbit.ai	dosu.dev
Model	Mult model	Own model	Mult model	Mult model

Test Gru

Test Gru is an AI developer that helps you add unit test from Gru. He will automatically add unit tests to PRs, with fully automated generation and testing, ensuring your code remains maintainable.

The code generated by Test Gru has achieved a merge rate of 80%, reaching enterprise-level standards.

Devin

Devin is an AI agent that creates a pull request. Developers can assign tasks to Devin, who will provide the runtime environment for the AI to work. He will also maintain the environment, so developers can review AI’s work. Currently, Devin is more suitable for small projects such as single repo websites.

Devin is currently the most well-funded company in this field.

Code Rabbit

Code Rabbit is an AI-powered code reviewer that delivers context-aware feedback on pull requests within minutes, reducing the time and effort needed for manual code reviews. It provides a fresh perspective and catches issues that are often missed, enhancing the overall review quality.

Case Study

Let’s use Test Gru as an example. Test Gru is an coding agent that helps you add unit test automatically.

When deverloper submit a PR on github, Test Gru will automatically detect code that requires unit tests and add tests for it.

After Gru completes writing the test code, it will run the tests. Once it confirms there are no issues with the test code, it will submit a PR with the unit test code to the current PR.

AI acts as a collaborator in helping developers write unit tests, enabling them to save time for writing more business code while ensuring the quality of the project.

According to the pilot project cases, the number of successfully merged PRs(test related) by Test Gru has already surpassed most human developers.

This case demonstrates how AI integrates into software development teams—not by replacing developers, but by freeing them from tedious tasks, thereby enhancing software quality and boosting productivity.

Agent outside your workflow

Secenario

When working outside of your workflow, there are often two scenarios:

1. Tackling a standalone technical issue, like writing an algorithm.
2. Engaging in activities unrelated to your current work projects, such as setting up a personal podcast or processing data.

This demand has resulted in many products in this field utilizing natural language interaction to replace tasks that once required searching websites, writing code, and executing it in an environment.

Unlike the previous two types of agents, this type can be used by non-professional developers as well. For example, data engineers can handle data processing and analysis, while beginner programmers can swiftly create a website.

Product

Product	V0 V0	Assistant Gru	Perplexity	Bolt.new
Scenario	Front end	Technical issue	Any questions	Small independent project
Price	Free Trial $20/month Premium $30/month	Free Trial $19.9/month	Free Trial $20/month	Free Trial $20/month
Provider	vercel.com	gru.ai	coderabbit.ai	dosu.dev
Model	Mult model	Own model	Mult model	Mult model

V0

An AI coding assistant created by Vercel, specialized in web development. It can create React components and full-stack Next.js applications. Developers can view the front-end results directly on the website.

Assistant Gru

Assistant Gru is an advanced AI developer designed to assist you in solving technical issues such as coding/testing/debugging, building algorithms, solutions. Compared to V0, Assistant Gru places more emphasis on backend issues and is capable of running and validating code.

Perplexity

Perplexity is an advanced AI-powered search engine designed to enhance the way users discover and interact with information.

Case Study

Let’s use V0 as an example. You can select a temple to build an application. In this case, we select Next.js + shadcn/ui.

Once I give my requirements, V0 starts working by creating a plan and writing the code.

You can see the change of the pages directly.

In this article, we explore the future of software development and the best practices for AI coding. Overall, while the field of AI coding is still developing, its vast potential is clear. Teams that adopt these capabilities early are likely to stand out in future competitions.