Improving the academic workflow: Introducing two AI agents for better figures and peer review

Extracted article text

Introducing two AI agents to streamline academic research. These include: PaperVizAgent, a visualizer agent for drawing academic figures, and ScholarPeer, a reviewer agent that automatically and rigorously evaluates academic papers.

Academic research is evolving at an unprecedented pace driven by the rapid advancements in AI. The academic research workflow is notoriously rigorous, involving far more than just conceptualizing an idea and writing a paper. One hurdle many researchers face is how to effectively visualize their research. While AI can draft text, creating the complex methodology diagrams and precise statistical plots required for top-tier conferences and journals is significantly more difficult. Furthermore, the scientific community relies on the peer review process to maintain the integrity of published research. However, the exponential growth of paper submissions has severely strained this system, leading to reviewer fatigue and inconsistent evaluations.

PaperVizAgent: generating publication-ready figures

PaperVizAgent is an autonomous framework designed to generate publication-ready academic illustrations from academic text. To initiate the process, a researcher provides two inputs: source context and communicative intent.

The PaperVizAgent framework orchestrates a collaborative team of five specialized AI agents including: a retriever, a planner, a stylist, a visualizer, and a critic. The retriever and planner gather references and organize the content. The stylist synthesizes aesthetic guidelines. The visualizer renders an image or generates executable python code for statistical plots. The critic evaluates the output against the original text and triggers a loop of iterative refinement when inconsistencies are found.

Through iterative refinement, this multi-agent system ensures the final illustration is both visually appealing and technically accurate.

In comprehensive experiments, PaperVizAgent consistently outperformed leading baselines, including direct prompting, few-shot prompting, and Paper2Any. The system achieved an overall score of 60.2, significantly surpassing all evaluated baselines and exceeding the established human baseline of 50.0.

ScholarPeer: emulating senior reviewers

ScholarPeer is a context-aware, search-enabled multi-agent framework designed to automate and elevate the peer review process by following the workflow of a senior researcher.

Unlike standard language models that treat reviewing as a simple text-generation task, ScholarPeer relies on a dual-stream process of context acquisition and active verification. It dynamically constructs a domain narrative using a sub-domain historian agent that grounds the review in live, web-scale literature. A baseline scout acts as an adversarial auditor, while a multi-aspect Q&A engine rigorously verifies the paper’s technical claims.

The final review report includes a detailed summary, strengths, weaknesses, and questions for the authors, much like a standard expert peer review.

ScholarPeer’s performance demonstrates the potential of integrating active web search with multi-agent orchestration for academic evaluation. In side-by-side evaluations, ScholarPeer achieved significant win-rates against state-of-the-art automated reviewing approaches.

Why it matters

PaperVizAgent and ScholarPeer are part of a broader effort exploring AI-assisted research more generally. By tackling two distinct but equally demanding phases of the publication lifecycle, these tools serve as collaborators that elevate the quality of scientific discourse and can accelerate the dissemination of knowledge.

The authors envision a future where researchers have access to a rich, interconnected ecosystem of AI assistants seamlessly integrated into every facet of the scientific workflow.