CodexGraph Team Contribute
Qubits | Official account QbitAI

Can't finish the code generation and completion tasks?!

Try using Large Language Models (LLMs) to help.

However, when dealing with large, multi-file codebases, LLMs can be difficult. In this case, it is often necessary to need a Retrieval-Augmented Code Generation (RACG) system to retrieve the relevant code information.

Existing solutions, such as semantic similarity-based retrieval and manually designed tools and APIs, are effective for specific tasks, but have limitations in generalization and generalizability.

To alleviate the commonality of existing RACG methods, researchers from the National University of Singapore, Alibaba, and Xi'an Jiaotong University proposed CodexGraph, a system that uses a code graph database as a medium to connect language models and code bases.

The research team evaluated CodexGraph in three representative academic benchmarks, and also implemented CodexGraph in five real-world code application scenarios based on ModelScope-Agent.

Experiments and real-world practices have shown the effectiveness and versatility of CodexGraph.

Research Methods:

CodexGraph innovatively connects language models with code bases through the code graph database interface interface to overcome the shortcomings of existing methods.

The workflow of the system mainly includes the following steps:

First, build a code graph database.

Using static analysis techniques, code symbols (such as modules, classes, functions, etc.) in the code base and their relationships are mapped to nodes and edges in the graph database according to the predefined schema.

Second, code structure-aware queries.

The flexibility of the graph query language is combined with the structural characteristics of the graph database to realize complex queries and deep retrieval of code graphs.

Third, the "Write then translateˮ strategy."

The Primary LM agent first analyzes the code problem and generates a natural language query.

A dedicated translation LM agent then translates these queries into a formatted graph query language.

Fourth, iterative pipelines.

Through an iterative approach, the LM agent gradually optimizes the query and retrieval results based on user problems and obtained information.

experiment

In the experimental part, CodexGraph was evaluated on three representative codebase-level benchmarks: CrossCodeEval, EvoCodeBench, and SWE-bench.

These tests cover areas such as cross-file code completion, code generation, and automated GitHub problem solving.

Experimental results show that CodexGraph shows excellent and stable performance in all benchmarks, which verifies the effectiveness of the code graph database as an interface between the language model and the code base.

The analysis of the original paper further points out that there are differences in the effectiveness of single query and multiple query strategies in CodexGraph when dealing with inference tasks of different difficulty.

In CrossCodeEval, the multiple query strategy is more effective; In SWE-bench, the single query strategy performs better.

In addition, removing the Cypher LM agent significantly increases the inference burden of the primary LM agent, resulting in degraded performance.

This suggests that the Cypher LM agent plays a key role in reducing the inference stress of the primary LM agent.

Examples of real-world use cases

Based on the ModelScope-Agent framework, the practical application value of CodexGraph has been reflected in multiple scenarios, such as:

• 代码聊天助⼿（Code Chat）

• 代码调试⼯具（Code Debugger）

• 单元测试⽣成器（Code Unit tester）

• 代码⽣成器（Code Generator）

• 代码注释⼯具（Code Commenter）

• ……

These applications demonstrate how CodexGraph can help developers solve real-world problems in a production environment, improving the efficiency and quality of code development.

summary

Using graph databases as a medium, CodexGraph provides a new way for language models to interact with code bases.

It not only improves the language model's ability to understand complex codebases, but also enhances the overall accuracy of code retrieval and navigation.

Its emergence brings new solutions to the field of automated software engineering, especially when faced with RACG challenges, demonstrating a wide range of potential and practical value in diverse code tasks and real-world applications.

As the technology continues to evolve, CodexGraph is expected to support more programming languages in the future and further optimize its workflow and index building efficiency.

This article is from Xinzhi self-media and does not represent the views and positions of Business Xinzhi.If there is any suspicion of infringement, please contact the administrator of the Business News Platform.Contact: system@shangyexinzhi.com