"How do we assure the quality and integrity of scientific research in a world where AI agents are both generating and vetting scientific outputs? Who should be the arbiter of scientific knowledge? For at least the last 40 years, peer review has been the gold standard for ensuring the quality of the scientific literature, and for engendering trust in science. Replacing human reviewers with automated ones seems hugely problematic, for at least two reasons: (1) current AI systems have known and inherently unavoidable biases baked into how they are built, and (2) the point of science is to push the envelope of knowledge forward, and it is precisely at the edge of our knowledge where these models are unreliable. This follows from the statistical framework that underpins the training of AI systems.

There has been increasing use of AI systems to judge the performance of other AI systems, most notably the paradigm of “LLM as a judge” — including extended to the fascinating concept of a “Language Model Council” that may benefit from the same advantages as human group decision-making and oversight. This provides apparently easy automation of the complex and resource-intensive process of human evaluation. How well this can be automated and what these systems struggle with, however, remains to be properly assessed. The organisers of AI4Science describe the conference as an opportunity to explore such questions; I hope that they take seriously the need to rigorously investigate the capabilities and the limitations of this approach before we hand over the scientific process to machines."

"Agents4Science 2025 is an interesting experiment. The whole conference restricted to AI written papers, and reviewed by AIs.

Many authors are now routinely using AI to write or rewrite their papers, including finding missed references. Conversely conferences and publishers are now exploring how AI can be used to referee papers - and establish which work is genuine and which is AI-hallucinated. I myself have found AIs have hallucinated several papers my colleagues and I might have (and possible should have) written. In one case, this was in a grant application and it turned out the applicant had asked the AI for further relevant papers from our group.

AI researchers are still trying to get a grip on this, and the Association for the Advancement of Artificial Intelligence (AAAI) this year introduced AI reviewing as a supplement to human reviewing (with authors seeing both anonymous human and AI reviews, as well as an AI-generated summary). AAAI-26 saw another massive increase in submissions, and this review system tested in practice. But recognizing AI-authored papers, distinguishing AI-hallucinated 'research' from real work, and assuring the ongoing quality of publication venues remain daunting challenges.

Agents4Science 2025 will provide an opportunity to see papers that are openly AI-written and openly AI-reviewed, and analyse this data to inform the community’s efforts to ensuring research integrity and optimized processing in our new AI-driven age. This doesn’t mean just identifying AI-generated papers, but exploring the scope for active human-AI teaming in solving important research problems, and deploying AI help systems, advisors and chatbots. I’ll look forward to seeing the data.

The acceptance rate of ~16% (24 out of 300+) is comparable to many journals and lower than most conferences. This looks like being an interesting and useful dataset for analysis to help us understand the use of AI in the research world."

"Agent4Science 2025 is an open conference that welcomes AI systems as recognised authors and reviewers. Its goal is to promote transparency in how AI is used and to encourage open sharing of methods and outcomes. At first glance, the concept sounds controversial—some might compare it to the Enhanced Games, which permits performance-enhancing drugs for athletes, but applied to academia.

Despite potential controversy, the conference represents an important step toward a more open dialogue about AI in science. Automated Theorem Proving, a field that has used AI to prove mathematical statements since the 1950s, shows that AI has long played a legitimate role in research. Yet, the current hesitation to acknowledge AI use mirrors the secrecy of the old alchemy era—when withholding knowledge stifled progress.

My main concern and the only real drawback of this conference lies in letting AI review academic papers. Since AI often reinforces familiar patterns, it may fail to recognise genuine innovation or undervalue unconventional writing styles. This could disproportionately affect many minority groups. I believe AI should mainly assist with technical aspects, such as formatting and citation compliance.

The good news is that this conference will let us assess the performance of AI reviewing systems. I hope for strong participation so we can achieve a meaningful outcome and conclusion."

"The emergence of conferences such as Agents4Science is a clear indicator of the scientific community facing head-on the promise and responsibility of AI.

Not only how AI can be used as a tool, but as how it could reimagine and reshape the scientific process: challenging us to rethink authorship, accountability, and creativity in research.”

"My 35 years of experience as an AI researcher taught me that AI does not qualify for academic authorship.

Academic papers are a unique form of publications due to expectations for innovation and discovery.

Authorship is a sacred section in academic publications.

We must pause and ask: what has changed to demand authorship for an AI?

Academic authorship has four corners: contribution, integrity, accountability, and consent; AI can’t get held accountable and does not have the will or agency for consent; current AI systems can’t guarantee integrity without human oversight; simply put, authorship of academic papers is a human responsibility and is inappropriate for an AI.

AI has been making scientific discoveries since its inception.

Thanks to large language models, significant advances have been made that allows the AI to partially or fully automate the scientific method in defined contexts, opening the possibility for AI to automatically generate academic papers.

Authorship is a different pool game! As an advocate for AI and as an AI psychologist who designs and diagnoses AI cognition and behaviour, there is a sacred line I do not cross; the line that distinguishes humans from machines; academic authorship is only meaningful for humans, not AI."

"Even by the standards of a highly innovative university as Stanford, this conference where 'AI serves as both primary authors and reviewers' is poorly motivated. The responsible practice of AI is to refrain from attributing human characteristics to AI despite its capacity to generate intelligent and human-like output. The same applies here, conducting research and then presenting the findings at a research conference are deeply human activities of peer-reviewed research, knowledge gain, discussion, collaboration, and networking. By assigning and attributing these to AI agents, we are belittling and devaluing the purpose it serves. This type of “AI serves as both primary authors and reviewers” activity is much better suited for a simulation, experiment or demonstration  as part of a human-led conference rather than a conference by itself.

Despite recent achievements in academic research, the foundational limitations of AI (such as the lack of experience and lack of compositionality of the real world) heavily overshadow any research innovation that AI produces without human intervention. Just as most of these recent achievements in academic research were human-led or human-in-the-loop activities, it is critical that global research communities are unified in sustaining this message to the rest of the world. AI is too flawed to generate research output by itself and even more flawed to conduct peer review on such output, without any human intervention."

"The idea of a research conference where both the authors and the reviewers are artificial intelligence systems is, at best, an amusing curiosity and, at worst, an unfunny parody of what science is meant to be. If the authors and reviewers are AI, then perhaps the conference attendees should be AI too, because no human should mistake this for scholarship.

Science is not a factory that converts data into conclusions. It is a collective human enterprise grounded in interpretation, judgment, and critique. Treating research as a mechanistic pipeline where hypotheses, experiments, and papers can be autonomously generated and evaluated by machines reduces science to empiricism on steroids. It presumes that the process of inquiry is irrelevant so long as the outputs appear statistically valid. But genuine scholarship is less about p-values than it is about conversation, controversy, and embodied knowing.

Equating AI agents with human scientists is a profound category error. Large language models do not think, discover, or know in any meaningful sense. They produce plausible sequences of words based on patterns in past data. Granting them authorship or reviewer status anthropomorphises what are essentially stochastic text-prediction machines. It confuses the illusion of reason with reason itself.

There is, of course, a legitimate discussion to be had about how AI tools can assist scientists in analysing data, visualising results, or improving reproducibility. But a conference built fully on AI-generated research reviewed by AI reviewers embodies a dangerous kind of technocratic self-parody. It reflects an ideology of techno-utilitarianism, in which efficiency and automation are celebrated even when they strip away the very human elements that make science legitimate.

So, to me, 'Agents4Science' is less a glimpse of the future than a satire of the present. A prime example of Poe’s law, where parody and extremism become indistinguishable. It reminds us that while AI can extend our capabilities, it cannot replace the intellectual labour through which knowledge becomes meaningful. Without humans, there is no science, just energy-intensive computation."

"The upcoming Agents4Science conference raises interesting questions about the authorship and ownership of AI generated works, and the role played by copyright law in the protection and incentivization of human cultural endeavour.

Our current law, in both the US and Australia, does not recognise human authorship of AI-generated works, even when those works are the result of complex, highly specialised sets of human prompts, Should our law be adapted to extend to AI-generated works where those works (peer-reviewed scientific research papers are a great example) show a significant degree of human input and control?

Organisers of Agents4Science - mostly academics based at Stanford University (and still human, at least for now!) - have put out a call for papers featuring 'AI-generated computational works that advance scientific discovery'. The conference is wholly online, and free to attend. Submitted papers need to be 'primarily authored by AI systems', which are 'expected to lead the hypothesis generation, experimentation, and writing processes'. The AI must be listed as the sole first author of each paper. Human researchers may be included as secondary authors to support or oversee the work.

Let’s be clear: excluding human authors in this fashion excludes the very essence of copyright, as currently understood in Australia and the US. This is now settled law, but it was not always so. A spate of cases in the early 2000s focused on authorship and ownership of an earlier generation of machine-generated works, including telephone directories (Desktop Marketing Systems Pty Ltd v Telstra Corporation Ltd [2002] FCAFC 112) and TV guides (IceTV v Nine Network Australia (2009) 239 CLR 458). Until the IceTV case reached the High Court in 2009, it was unclear in Australia whether or not human overseers could claim copyright in such works. The High Court unanimously and emphatically ruled that they could not.

In coming to this conclusion, the High Court emphasised two fundamental points about copyright: In order to attract protection:
1.    the work must originate with an author (who must be human); and
2.    the work must be the result of ‘independent intellectual effort’ and ‘sufficient effort of a literary/artistic nature’ to create the original expression which constitutes the work.

In 2010, in Primary Health Care Limited v Commissioner of Taxation [2010] FCA 419, the Federal Court of Appeal put this more eloquently. The Court held that to attract copyright, a human-generated work must show 'a continuous narrative showing independent intellectual effort expended in expression'. This works very well as a litmus test across every field of human cultural endeavour. Had AI never been invented, copyright law based on this principle would probably have muddled along quite nicely.

AI upsets the applecart, completely, because copyright as currently defined in Australia and the US, cannot currently subsist in any of the products that it generates. This is a big problem, because AI is arguably the single most powerful new tool for cultural creativity since the printing press. In its long history (dating back to the Statute of Anne in 1710) copyright law has adapted to many new technologies without sacrificing its ‘inner core’ of human originality and authorship. Forty years ago there were adaptations in response to the computer age; there is now a strong argument to suggest that copyright should adapt again, to vest copyright in certain classes of AI-generated works that result from complex and original human prompts, and which exhibit clear signs of significant overall human control.

Fortunately, we don’t have to look far to see what this adapted copyright law might look like: the UK has already introduced it. Section 9(3) of the Copyright, Designs and Patents Act 1988 (UK) states that in the case of a work that that is computer-generated, the author, for copyright purposes, 'shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken' (emphasis added).

This means that, in the UK, if you are a human working with AI, and you put in original prompts, choose the scope of your work, and make edits to various drafts along the way, you will be taken to be the author of the final AI output for copyright purposes – subject of course to the terms and conditions of the LLM you happen to be using; for example ChatGPT does generally allow users to own the outputs that result from their prompts.

What would this mean for the authors of the papers to be presented at Agents4Science? It would make a world of difference. It would mean that the investment of time, effort, and original thought which goes in to creating each AI generated scientific paper would be rewarded with ownership, and all of the protections that this entails. Absent this, and these carefully curated works will become faceless, cogs in the machine, and the sparks of human ingenuity and originality inherent in the creation of each such work will go unrecognised.

Currently, contributors to Agents4Science will not mind this: the conference is an experiment, and there is a value always in being involved in something new. But this will pass. Scientists will soon find, as musicians and artists and writers have already found, that without proper recognition of the distinctly human vision inherent in the architecture of their works, they will be left high and dry. The AI tide will move on without them, leaving 'all the voyage of their life bound in shallows and in miseries'."