← All posts

Has This Paper Been Retracted? Crossref Can Give One Retraction Three Dates, and OpenAlex Leaves It Out of a Plain List

Published November 2026 · 24 min read

Written 20 November 2026. The facts, and every live database value quoted below, are as of that date. Live values can change.

If you cite scientific papers, or build software that does, you need to know whether a paper has been retracted. A retraction means the journal has withdrawn the paper and says it should no longer be relied on. Two free, open databases will tell you without a sign-in or a fee. (OpenAlex responses on 20 November 2026 said that keyless requests share a small daily budget per IP address, so a long reference list may hit a limit.) This post works one real paper through each step and shows every query. It also shows where the answer can mislead you. For one paper, Crossref holds three dates that span more than three and a half years, and the publisher's own entry is dated to the month of the paper's print issue, about four and a half months after it first appeared. A second database, OpenAlex, flags the paper correctly when you look it up on its own, but drops it from a plain list query.

The two databases, and what they say they do

Crossref is the registry where most journals register their papers' DOIs. A DOI (Digital Object Identifier) is a permanent ID such as 10.1016/j.future.2018.03.020. Crossref publishes the record behind each DOI through a free REST API, which is a web address that returns data rather than a web page.

Retraction Watch is a database of retractions that its staff collect and check by hand. Crossref's documentation says: "In September 2023, we acquired the Retraction Watch database". It also says the database "is updated every working day by Retraction Watch" (Crossref documentation, "Retraction Watch", last updated 19 January 2025).

Since January 2025, Crossref has put Retraction Watch's entries into the same API as the publishers' own entries. Crossref's best case for this, in its own words, comes from the blog post that announced it (Martyn Rittman, 29 January 2025):

So Crossref says plainly that one retraction can appear more than once, and that the reason for each entry is its source. That is the design the method below relies on.

OpenAlex is a free index of scholarly works. Its documentation defines the field is_retracted as "True if the work is known to be retracted: the Retraction Watch database lists its DOI as a retracted paper (and not as reinstated), or Crossref or PubMed records a retraction of it, or its title starts "Retracted"." Its best case on speed: "We read Retraction Watch every day, as Crossref publishes it, so a retraction added there usually shows here the next day." (OpenAlex work object documentation, read 20 November 2026.)

Step 1: look up the paper in Crossref, and read updated-by

The worked paper is "RETRACTED: A hybrid whale optimization algorithm based on local search strategy for the permutation flow shop scheduling problem", published in the journal Future Generation Computer Systems (Elsevier), issued August 2018. Paste this address into a browser:

https://api.crossref.org/works/10.1016/j.future.2018.03.020

The record is a long block of JSON, a plain-text data format. One field matters here, and it is easy to pick the wrong one.

If you are checking a paper you want to cite, read updated-by. Crossref's documentation page and its announcement post both name update-to and neither names updated-by. A Crossref community forum post of 29 May 2026 does show updated-by on a retracted paper (forum post). Crossref's blog post uses a SAGE retraction notice, 10.1177/17588359231172420, as its example. The paper that notice retracts is 10.1177/1758835920922055, and its record shows the difference: it has no update-to key, and updated-by is present.

That SAGE paper also shows what agreement looks like. Its updated-by value, as returned on 20 November 2026, was:

[{"DOI":"10.1177/17588359231172420","type":"retraction","label":"Retraction","source":"retraction-watch","updated":{"date-parts":[[2023,4,22]],...},"record-id":"44124"}, {"DOI":"10.1177/17588359231172420","type":"retraction","label":"Retraction","source":"publisher","updated":{"date-parts":[[2023,4,22]],...}}]

There are two entries, one from each source. Both say "retraction", both point to the same notice, and both are dated 22 April 2023.

The whale-optimisation paper is different. Its updated-by holds three entries:

# type source updated date points to
1 retraction retraction-watch 2021-10-01 notice 10.1016/j.future.2021.11.015, record-id 34581
2 retraction publisher 2018-08-01 the paper's own DOI
3 erratum publisher 2022-03-01 notice 10.1016/j.future.2021.11.015

Each row disagrees with the others in its own way:

How far apart the dates are, counted by hand so you can redo it with any date calculator:

A reader who takes the first retraction entry they see, and its date, could report that this paper was retracted in the month of its print issue.

For scale, on 20 November 2026 Crossref's is-referenced-by-count for this paper was 248 and OpenAlex's cited_by_count was 289.

This pattern was already reported, and the reported example is still unfixed

This is not a new discovery for Elsevier. A thread on Crossref's community forum, opened by the user "gcabanac" on 14 June 2026, compared the updated date in an Elsevier record with one in a Springer Nature record. The Elsevier record (10.1016/j.heliyon.2024.e28357) carried the paper's publication date. The Springer Nature record (10.1038/ni829) carried, at the time, the retraction date, 9 February 2026. On 25 June a Crossref staff member replied that the publisher appeared to have deposited an incorrect date. On 9 July the same staff member reported that Elsevier had confirmed the observation, that it was putting an inaccurate date in the metadata, and that it would fix this, though there was no timeline yet (forum thread). Credit for spotting the Elsevier problem belongs to that thread.

On 20 November 2026, 134 days after that reply, the thread's own example still had the problem. (July 9 to July 31 is 22 days; add 31 for August, 30 for September, 31 for October and 20 for November: 22 + 31 + 30 + 31 + 20 = 134.) The Crossref record for 10.1016/j.heliyon.2024.e28357 had last been re-indexed on 18 November 2026, and its entries were:

From 2024-04-15 to 2026-01-22 is 647 days: 365 to 2025-04-15, then 275 to 2026-01-15 (one month-step each from 15 April 2025 to 15 January 2026: April 30, May 31, June 30, July 31, August 31, September 30, October 31, November 30, December 31), then 7 more.

The same pattern appears in Springer's in-place retractions

Some publishers retract a paper "in situ", which means in place: the retraction is stamped on the paper's own DOI rather than issued under a new notice DOI. Crossref's guidance for publishers says "In-situ updates or corrections are not recommended because they tend to obscure the scholarly record." (Crossref, "Registering updates", last updated 27 May 2024). That page does not say which date a publisher should put in an update.

Springer's in-place retractions under the DOI prefix 10.1007 show the publication-date pattern too, but they also carry their own correction. Each Springer record has an assertion block, which is a set of labelled statements from the publisher. It includes the date first online and a sentence stating the retraction date. In all four records checked on 20 November 2026, the publisher's updated date equalled the first-online date, and the assertion gave a later retraction date:

DOI Publisher updated first online (assertion) Retraction date (assertion, exact words) Retraction Watch entry in Crossref Gap
10.1007/s11277-021-09072-0 2021-09-11 "11 September 2021" "This article was retracted on 6 December 2022" yes, 2022-12-06, record-id 42746 451 days
10.1007/s12652-021-02990-8 2021-03-14 "14 March 2021" "This article was retracted on 30 October 2022." none 595 days
10.1007/s10338-025-00601-z 2025-03-26 "26 March 2025" "This article was retracted on 3 May 2026" none 403 days
10.1007/s00500-023-08541-x 2023-06-10 "10 June 2023" "This article was retracted on 2 February 2026" none 968 days

The gaps, counted step by step:

That is a range of 403 to 968 days, about 13 to 32 months. In two cases the publisher's own sentence could be cross-checked against Retraction Watch's file, and both matched. For 10.1007/s11277-021-09072-0 both give 6 December 2022. For 10.1007/s12652-021-02990-8, Retraction Watch's row (Record ID 41257) has a RetractionDate of "10/30/2022", which is the assertion's 30 October 2022. In all four, the updated field matched the first-online date.

Two more details from these records. In all four, the assertion's change_type value is "Clarification", while its change_details says the article was retracted. And three of the four have no Retraction Watch entry in Crossref. For 10.1007/s12652-021-02990-8, a reader who reads only Crossref's publisher entry would date the retraction 595 days early. Retraction Watch holds the correct date for it (row 41257, with a RetractionDOI of "Unavailable"), but on 20 November 2026 Crossref's API showed one publisher entry only, so the Retraction Watch date had not reached the API record. For the other two without an entry in Crossref, 10.1007/s10338-025-00601-z and 10.1007/s00500-023-08541-x, a search of the Retraction Watch file found no row.

The Springer Nature record in the forum thread, 10.1038/ni829, was reported in the thread as carrying the retraction date, and that would have been a counter-example. Opened on 20 November 2026 (indexed 16 October 2026), it no longer is one. Its updated-by now holds two publisher retraction entries: one pointing to 10.1038/ni829 itself, dated 2002-08-05, which is its publication date, and one pointing to a separate notice, 10.1038/s41590-026-02446-1, dated 2026-02-09. So today it also shows an entry on the paper's own DOI dated to the publication date, the same pattern. The four Springer 10.1007 records are a small number, and they are not a rate.

How common is disagreement? A small sample

To see whether these cases are unusual, a random sample of 20 Crossref records carrying a retraction was drawn on 20 November 2026 with this query:

https://api.crossref.org/works?filter=update-type:retraction&sample=20&select=DOI,update-to,publisher

The same day, https://api.crossref.org/v1/works?filter=update-type:retraction&rows=0 reported 81,721 records in total. The sample option returns a different random draw each time, so the 20 DOIs are listed in the Sources so anyone can re-check them. For each one, the retracted paper's own updated-by was also read where that could add entries, except for three noted below.

The count, step by step:

The four disagreements are the whale-optimisation paper (dates and type differ), a Results in Physics paper (dates and type differ), an Aggression and Violent Behavior paper (dates differ by 886 days), and one Springer paper not retracted in place (dates differ by 5 days, 2014-03-27 against 2014-04-01).

The Results in Physics paper, 10.1016/j.rinp.2022.105738, follows the Elsevier pattern exactly. It has a publisher retraction dated 2022-08-01 pointing to itself, a Retraction Watch retraction dated 2024-06-14 (record-id 59528), and a publisher erratum dated 2024-07-01 for the same notice. Retraction Watch's row for this record carries a caveat in its Notes: "date of retraction unknown, html overwrite ... separate retraction notice published after html overwrite". So 2024-06-14 is itself not a firmly known date. The gap from the first to the second date is 683 days: 365 to 2023-08-01, then 305 to 2024-06-01 (31 + 30 + 31 + 30 + 31 + 31 + 29 + 31 + 30 + 31), then 13.

For the Aggression and Violent Behavior paper, 10.1016/j.avb.2021.101559, the publisher's date is 2021-01-09 and Retraction Watch's is 2023-06-14. (2021-01-09 to 2023-01-09 is 730 days; on to 2023-06-09 adds 31 + 28 + 31 + 30 + 31 = 151, for 881; on to 2023-06-14 adds 5, for 886.) Its title begins "WITHDRAWN". Retraction Watch's row (Record ID 45736) says in its Notes: "date of retraction unknown, html page overwrite, date estimated from notifications". So by Retraction Watch's own account its 2023-06-14 is an estimate, and which of the two dates is the true withdrawal date cannot be told from the records read.

What this sample supports, and what it does not:

A larger study before the merge found disagreement too. Schneider, Lee, Zheng and Salami compared Crossref, Retraction Watch, Scopus and Web of Science in a 2023 conference paper. Their abstract says: "For 10,512 publications (21%), there is disagreement: at least one source deems them retracted while another lacks retraction indexing" (abstract on IDEALS; the full text was not read for this post).

Step 2: the reason lives in the Retraction Watch file

The Crossref API carries no reason for a retraction. Crossref's announcement says: "The record-id refers to the entry in the csv file with further details, such as the reason for retraction." A CSV file is a spreadsheet saved as plain text. Crossref's documentation says: "The Retraction Watch database is available in csv format from a git repository." The repository is at https://gitlab.com/crossref/retraction-watch-data. On 20 November 2026 its latest update was titled "Retraction Watch data for 2026-11-19", with daily updates before it. Its columns include Record ID, RetractionDate, OriginalPaperDate, RetractionNature and Reason.

For the worked paper, the row to find is Record ID 34581. Open retraction_watch.csv, search the Record ID column for 34581, and read the Reason, RetractionDate and Notes columns. The whole file was fetched for this post (73,936 rows), and that row says:

These are Retraction Watch's own words. They say that its October 2021 date does not come from the notice. "PP" is not spelled out in the note; it presumably stands for PubPeer, a site where readers comment on papers, though the note does not say. They also give the notice a date of 9 December 2021, a fourth date besides the three in Crossref. The notice itself was not read, so 9 December 2021 is Retraction Watch's statement and not a fact checked here.

Three Springer notices and the Heliyon notice (10.1016/j.heliyon.2026.e44556) were also read, but this post does not report the reasons they give. The Elsevier notice for the worked paper was not read. Apart from the date-related entry quoted above, no reason from the Retraction Watch file is reported either. So this post does not say why any of the Elsevier or Springer papers above was retracted, and it does not compare stated reasons across sources.

The SAGE control case is the one notice whose reasons are quoted. The text was taken from a copy in Europe PMC, a public archive of biomedical papers. It says the editor retracted the paper "Due to unresolved concerns about image integrity that call into question the validity of the findings", and that "Authors disagreed with the retraction." This shows what a notice adds that no database field carries: the reason, and whether the authors agreed.

Step 3: check a second index, and do not use a plain list

On its own, an OpenAlex lookup for one paper looks like this:

https://api.openalex.org/works/doi:10.1007/s12652-021-02990-8

On 20 November 2026, is_retracted was true for all ten retracted papers in the list in the Sources, and also for the two papers shown above, 10.1007/s12652-021-02990-8 and the worked paper 10.1016/j.future.2018.03.020 (OpenAlex ID W2790763152). Seven of the nine papers that Crossref holds only from the publisher were among the ten. So OpenAlex agreed with Crossref that the retraction exists in every lookup. But its record gives no retraction date, no reason, and no source. Nine of the ten titles begin "RETRACTED" or "Retracted", and OpenAlex's definition counts a title that starts "Retracted", so for those nine the flag could have come from the title. The tenth, 10.1016/j.avb.2021.101559, is titled "WITHDRAWN: ..." in both OpenAlex and Crossref, and OpenAlex still reads is_retracted: true. For that one the flag cannot have come from the title rule, so it can come from the data.

The trap is in list queries. OpenAlex marks some works as "pulled". Its documentation defines is_pulled as covering every retracted work, except retract-and-replace notices and Cochrane reviews withdrawn as out of date, plus withdrawn or removed works. It says pulled works "are left out of works lists, searches and counts unless you add include_pulled=true; a filter on is_pulled or is_retracted brings them back too." Here is what that does in practice. This query asks for two retracted papers by DOI:

https://api.openalex.org/works?filter=doi:10.1007/s00401-008-0407-x|10.3389/fmicb.2015.01001&select=doi,is_retracted

It returned a count of 0. The same query with &include_pulled=true added returned 2, both with is_retracted: true. A filter query for 14 DOIs without the flag also returned 0. The same happened for the worked paper: https://api.openalex.org/works?filter=doi:10.1016/j.future.2018.03.020 returned a count of 0, though the single lookup flags it. If you check your reference list against OpenAlex with a filter query, the retracted papers (apart from the exceptions in the definition above) are the ones that will not come back, and silence looks like a clean result.

OpenAlex's flag has been wrong before, in the other direction. Hauschke and Nazarovets found in early 2024 that "the mere presence of any information about an update causes OpenAlex to categorise the publication as retracted", and their abstract warns: "The issue affects data provided by OpenAlex in the period between 22 Dec 2023 and 19 Mar 2024. Anyone using data from this period should urgently check it and replace it if necessary." They report that "Approximately, 2300 incorrect records were identified and corrected." (arXiv 2403.13339, version 1, 20 March 2024.) OpenAlex acknowledged the error on its users' mailing list on 19 March 2024: works that had only received corrections or expressions of concern had been labelled retracted, about 2,300 of them. The paper they used as their example, 10.2807/1560-7917.ES.2020.25.3.2000045, which they say "was never retracted", read is_retracted: false in OpenAlex on 20 November 2026.

The method, in short

  1. Look up https://api.crossref.org/works/<DOI> and read updated-by, not update-to.
  2. For each entry, note its type, its source and its updated date. Expect more than one entry.
  3. Distrust a publisher updated date that matches the paper's publication date, or an entry that points to the paper's own DOI. For Springer, read the assertion sentence "This article was retracted on…". For Elsevier, compare it with the Retraction Watch entry and the notice.
  4. Treat an Elsevier notice typed "erratum" as possibly a retraction. Check its title and the Retraction Watch entry.
  5. Take the record-id to the Retraction Watch CSV for the reason, and read the publisher's notice in its own words.
  6. In OpenAlex, look up one paper at a time, or add include_pulled=true to any list or filter query.
  7. Read "no entry" as "no entry in this source". It does not mean "not retracted".

What could not be established

Sources

All read 20 November 2026 unless another date is given.