Findings and conclusions of the paperWilson and colleagues seek to provide an ‘improved’ estimate of the relative risks of smoking and e-cigarette use through biomarkers of toxicant exposure.[1]  Based on 17 comparisons of biomarkers in e-cigarette (ENDS) users and cigarette smokers drawing on five studies,[2][3][4][5][6] the authors make a striking claim in their conclusion:This analysis, suggests that the use of modern ENDS devices (vaping) could be a third as harmful to health as smoking in a high-income country setting. But this estimate is based on a limited number of biomarker studies and is best be considered a likely upper level of ENDS risk given potential biases in our method (i.e., the biomarkers used being correlated with more unaccounted for toxicants in smoking compared to with using ENDS). (emphasis added)It is true that the study does not incorporate biomarkers for hundreds of hazardous or potentially hazardous products of combustion that are found in cigarette smoke but are unlikely to be present in ENDS aerosol because of the absence of combustion. However, this is far from the most serious flaw in the analysis. Below we present four further concerns that we believe render the paper unreliable. A critique of the methods and findingsThe conclusion is not supported by the data or the analysis presented.  There are four clear problems beyond the narrow coverage of likely hazardous agents in cigarette smoke.Background exposure is ignoredMany “exclusive" ENDS users were in fact smokersValid adjustment indicates no incremental risk for acroleinArbitrary inclusion criteria meant important studies were excluded from the analysis1. Background exposure is ignoredFirst, the authors ignore background exposures (the ambient exposure experienced by non-users arising from the environment, food etc.), with the exception of an attempt to correct for acrolein exposures (discussed below). This is a serious error because most of the biomarkers measured are present in nonsmokers at significant levels.[7][8][9][10] Consider this illustration: if smokers have a level of a given chemical of 60 and ENDS users of 20, the authors would assert that exclusive ENDS use poses a third of risks of smoking. However, if non-smokers also have the level of 20 for the given biomarker, ENDS use poses no incremental risk at all. In fact, in the Jay[6] and Hatsukami[4] studies (comprising 11 of the 17 comparisons), abstinence and NRT arms show near-identical outcomes to those of their ENDS arms, yet this important finding is not reflected in the analysis.2. Many “exclusive" ENDS users were in fact smokersSecond,  many subjects designated as “exclusive ENDS users” were in fact smokers. Table 3 shows high levels of carbon monoxide markers in exclusive ENDS users for three of the five studies, but there are no significant CO emissions from ENDS under normal operating conditions tolerated by users.[11]  Son et al. compared CO emissions from e-cigarettes and cigarettes and concluded:[12]All of the tested e-cigarettes under our experimental conditions generated 40 to 3618 times less CO than conventional cigarettesThat means ENDS CO emissions are respectively 2.5% to 0.03% of the smoking-related exposure.  Yet Wilson et al. include comparisons for CO emissions from ENDS at 27.1%, 38.9%, 43.0% and 53.7% of cigarettes in Table 3 and calculate a cardiovascular risk in Table 4 on this basis.  It should also be noted that Son et al. recognised the upper end of the range (40 times less CO than cigarettes) was likely caused by the device overheating - something that would produce CO measured in machine tests, but would also make human vaping and therefore human exposure impossible. Two of these studies explicitly note that some in the exclusive e-cigarette group were smokers. Hatsukami et al.[4]  note that only “33% of participants achieved CO-verified 7-day smoking abstinence” in the “exclusive e-cigarette" group. Oliveri et al.[3] state that “a small proportion of [ENDS users] (17% tank, 25% cartridge) exhibited levels of COHb that exceeded 5% saturation. As [ENDS] are non-combustible products and therefore do not generate carbon monoxide, the observed levels of >5% saturation suggests that a select group of AEVP were not exclusive users and may have been smoking cigarettes.”The presence of high carbon monoxide markers should have been used by the authors as a reality check on whether people they classified as exclusive ENDS users were in fact also smoking.  Instead, they just took the high exposures and built them into the risk calculation creating a misleading figure for ENDS cardiovascular risk. Once it is clear that study participants designated as exclusive ENDS users have also been smoking, it should be assumed that all their biomarker data are contaminated with smoking exposures and the data are unreliable for comparative purposes. Carbon monoxide exposures should have been used as a reality check on the data, not incorporated into the results.3. Valid adjustment indicates no incremental risk for acroleinThird, where correction has been attempted for background exposure, a realistic correction shows no incremental risk from vaping. In Table 4 the authors state that they have adjusted for one chemical, acrolein.  3-HPMA is a metabolite of acrolein and one of the biomarkers included in the study.  This biomarker is found at a non-trivial level in non-tobacco users, as recorded in Alwis et al. 2015.[7]  In Alwis at al, non-smokers had 20.1% of the 3-HPMA level of smokers. Wilson report that in the Jay et al. study, smokers had 1.87 mg and exclusive ENDS users 0.2 mg over 24 hours, suggesting ENDS users had 10.7% of exposure of smokers. However, if 20.1% of the level attributed to smokers (0.37 mg) was deducted from both groups, the exclusive ENDS users would have zero incremental ENDS-associated exposure compared to smokers having 1.5 mg over 24 hours. On this measure, vaping poses 0% of the risk of smoking, not 10.7%.  The other studies used to estimate acrolein exposure are compromised through smoking by subjects designated as exclusive ENDS users as described directly above.4. Arbitrary inclusion criteria meant important studies were excludedFourth, the authors impose an arbitrary cut-off date to include only studies that were published and had data collected after 1 January 2017.  This excluded 11 studies (references 37-47 in the paper), most of which were informative and explicitly designed to assess biomarkers of exposure.  This was ostensibly done only to include the most modern devices, but there is no real reason to exclude high quality older studies based on an arbitrary date. Had that been the reason, the results could have stratified by pre- and post 2017. It is likely that the main difference would arise from improved nicotine delivery, but that could have been addressed by indexing the biomarker exposures to nicotine exposure. The five studies included used different technologies - vape pens, pods, salts, tank systems and several had the problem of ongoing smoking.  So device heterogeneity is a challenge with the post January 2017 studies chosen, though the Wilson et al paper has far more serious flaws than this.Summary of erroneous findingsThe table below provides brief comments on the 17 comparisons between cigarettes and e-cigarettes that the authors present. All 17 comparisons are erroneous for one or more of the reasons given below. StudyBiomarkerExposure: ENDS/cigarettesSignificant problemNga et al. 2020eCO38.90%CO is not emitted from e-cigarettes (NASEM, 2017)[11] Hatsukami et al. 2020CEMA34.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Hatsukami et al. 20203-HPMA53.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Hatsukami et al. 2020HMPMA53.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Hatsukami et al. 2020PheT79.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Hatsukami et al. 2020eCO43.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Hatsukami et al. 20203-HPMA53.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Hatsukami et al. 2020NNAL47.00%Measurements relate to smokers randomized to exclusive ENDS use, but most were noncompliant and still smoked.Jay et al. 20203-HPMA10.70%Results are identical to those of the absolute cessation arm in that study.Jay et al. 20203-HPMA10.70%Results are identical to those of the absolute cessation arm in that study.Jay et al. 2020COHb27.10%Results are identical to those of the absolute cessation arm in that study. Jay et al. 2020NNN38.60%Results are slightly below those of the absolute cessation arm in that study. The authors attribute this to endogenous formation from nicotine/nornicotine, as similar results were reported for NRT which is unlikely pose a measurable cancer risk (NASEM, 2017).Oliveri et al. 20203-HPMA53.20%Oliveri et al note that high CO and NNK levels indicate substantial misreporting (unreported smoking)Oliveri et al. 2020NNAL12.40%Oliveri et al note that high CO and NNK levels indicate substantial misreporting (unreported smoking)Oliveri et al. 20203-HPMA53.20%Oliveri et al note that high CO and NNK levels indicate substantial misreporting (unreported smoking)Oliveri et al. 2020COHb53.70%Oliveri et al note that high CO and NNK levels indicate substantial misreporting (unreported smoking); CO is not emitted by e-cigarettes (NASEM, 2017)[11] Boykan et al. 2019NNAL17.90%This compares the percentages of subjects above the NNAL cutoff level for non-smokers. However, because NNAL’s precursor NNK can indeed be found at low levels in some e-cigarettes but is on average 40 times lower compared to tobacco smoke,[13] relying on a binary measure based on a cutoff level far below the mean exposure level for smokers, would certainly not provide a good measure for quantitative comparison.ConclusionThe analysis provided by Wilson et al. is flawed, its conclusion is unreliable and retraction should be considered. The results would be misleading if taken seriously by policymakers or incorporated into subsequent analysis as though the paper was a reliable source. For example, the Wilson et al. analysis was reused by some of its authors in a recent public health cost-benefit analysis of ENDS in New Zealand (Summers et al. 2022).[14] The estimates provided in Summers et al are unreliable as a consequence of the underlying assumptions about relative risk from Wilson et al. This paper would be best reformulated as sensitivity testing the credibility of New Zealand policy against extreme assumptions about relative risk.  In spite of the exaggerated estimates of relative risk used in its calculation, Summers et al. did conclude that: This study found evidence using updated biomarker studies that ENDS liberalization could result in QALY gains across the New Zealand population lifespan that are also cost-saving to the health system.References^Nick Wilson, Jennifer A. Summers, Driss Ait Ouakrim, Janet Hoek, et al. (2021). Improving on estimates of the potential relative harm to health from using modern ENDS (vaping) compared to tobacco smoking. BMC Public Health, vol. 21 (1). doi:10.1186/s12889-021-12103-x.^Janice Diong Li Nga, S. Lokman Hakim, Sobia Bilal. (2020). Comparison of End Tidal Carbon Monoxide Levels between Conventional Cigarette, Electronic Cigarette and Heated Tobacco Product among Asiatic Smokers. Substance Use & Misuse, vol. 55 (12), 1943-1948. doi:10.1080/10826084.2020.1781180.a, bDouglas Oliveri, Qiwei Liang, Mohamadi Sarkar. (2019). Real-World Evidence of Differences in Biomarkers of Exposure to Select Harmful and Potentially Harmful Constituents and Biomarkers of Potential Harm Between Adult E-Vapor Users and Adult Cigarette Smokers. doi:10.1093/ntr/ntz185.a, b, cDorothy K Hatsukami, Ellen Meier, Bruce R Lindgren, Amanda Anderson, et al. (2019). A Randomized Clinical Trial Examining the Effects of Instructions for Electronic Cigarette Use on Smoking-Related Behaviors and Biomarkers of Exposure. doi:10.1093/ntr/ntz233.^Rachel Boykan, Catherine R. Messina, Gabriela Chateau, Allison Eliscu, et al. (2019). Self-Reported Use of Tobacco, E-cigarettes, and Marijuana Versus Urinary Biomarkers. Pediatrics, vol. 143 (5), e20183531. doi:10.1542/peds.2018-3531.a, bJoanna Jay, Erika L Pfaunmiller, Norman J Huang, Gal Cohen, et al. (2019). Five-Day Changes in Biomarkers of Exposure Among Adult Smokers After Completely Switching From Combustible Cigarettes to a Nicotine-Salt Pod System. doi:10.1093/ntr/ntz206.a, bK. Udeni Alwis, B. Rey deCastro, John C. Morrow, Benjamin C. Blount. (2015). Acrolein Exposure in U.S. Tobacco Smokers and Non-Tobacco Users: NHANES 2005–2006. Environmental Health Perspectives, vol. 123 (12), 1302-1308. doi:10.1289/ehp.1409251.^G Scherer, M Urban, H-W Hagedorn, S Feng, et al. (2007). Determination of two mercapturic acids related to crotonaldehyde in human urine: influence of smoking. Hum Exp Toxicol, vol. 26 (1), 37-47. doi:10.1177/0960327107073829.^S. S. Hecht. (2005). Longitudinal Study of Urinary Phenanthrene Metabolite Ratios: Effect of Smoking on the Diol Epoxide Pathway. Cancer Epidemiology Biomarkers & Prevention, vol. 14 (12), 2969-2974. doi:10.1158/1055-9965.epi-05-0396.^Neal L Benowitz, John T Bernert, Jonathan Foulds, Stephen S Hecht, et al. (2019). Biochemical Verification of Tobacco Use and Abstinence: 2019 Update. doi:10.1093/ntr/ntz132.a, b, cCommittee on the Review of the Health Effects of Electronic Nicotine Delivery Systems, Board on Population Health and Public Health Practice, Health and Medicine Division, National Academies of Sciences, Engineering, and Medicine. (2018). Public Health Consequences of E-Cigarettes. doi:10.17226/24952.^Son Y, Bhattara C, Samburova V, Khlystov A.. (2020). Carbonyls and Carbon Monoxide Emissions from Electronic Cigarettes Affected by Device Type and Use Patterns. Int. J. Environ. Res. Public Health, vol. 17(8) .^Maciej Lukasz Goniewicz, Jakub Knysak, Michal Gawron, Leon Kosmider, et al. (2013). Levels of selected carcinogens and toxicants in vapour from electronic cigarettes. Tob Control, vol. 23 (2), 133-139. doi:10.1136/tobaccocontrol-2012-050859.^Jennifer A Summers, Driss Ait Ouakrim, Nick Wilson, Tony Blakely. (2021). Updated Health and Cost Impacts of Electronic Nicotine Delivery Systems, Using Recent Estimates of Relative Harm for Vaping Compared to Smoking. doi:10.1093/ntr/ntab178.

This definition of addiction is well specified. In particular,  the inclusion of a requirement for serious net harm within the definition and the rationale for this provided in the comment are valuable clarifications of language that is often used casually and inconsistently. This creates an operationally useful definition.Definition (addiction)A mental disposition towards repeated episodes of abnormally high levels of motivation to engage in a behaviour, acquired as a result of engaging in the behaviour, where the behaviour results in risk or occurrence of serious net harm.CommentThis entity focuses on abnormal motivation to engage in a behaviour and includes serious net harm as a feature. The reason is to limit the class to things that merit a treatment and public health response. It is a quantitative entity and a fuzzy set because there can be varying thresholds set for degree of harm and strength of motivation. As a result, it is essential to operationalise the term for it to be meaningfulI have one suggestion to improve the entry as it appears in AddictO: addiction - that it to remove the word Dependence under the heading “Synonyms”.  This is confusing because the definition of dependence, AddictO: substance dependence is usefully distinct from the definition of addiction, meaning that they should not be considered synonyms in an ontology. Definition (substance dependence)A bodily disposition which is realised as impaired functioning following reduction or termination of use of a psychoactive substance. This definition highlights the role of negative reinforcement through withdrawal but does not require the serious net harm in the addiction definition. In other words, it provides a useful two-tiered structure of meaning. All addictions involve dependence, but not all dependence is an addiction.  The difference is the extent of serious net harm.This is relevant in the case of nicotine where there are varying degrees of dependence according the the pharmacokinetics of possible through the delivery system and orders of magnitude difference in harm according to whether combustion of tobacco and inhalation of smoke are involved.

This definition misses the main point.  The definition of an FDA defined-tobacco product is a legal definition. It should be defined only by reference to the law that forms the definition. This may change over time through changes in legislation and case law.   The definition of an “FDA-defined tobacco product” must, ontologically, be identical to the definition used by FDA.  This is a follows:    An FDA defined tobacco product is any  product that meets the definition of a tobacco products set out in the United States Food, Drug and Cosmetic Act and as interpreted by the courts. Tobacco product is defined at Section 201 of the Federal Food, Drug, and Cosmetic Act (21 U.S.C. 321) in subsection (rr).The term 'tobacco product' means any product made or derived from tobacco that is intended for human consumption, including any component, part, or accessory of a tobacco product (except for raw materials other than tobacco used in manufacturing a component, part, or accessory of a tobacco product).The term 'tobacco product' does not mean an article that is a drug under subsection (g)(1), a device under subsection (h), or a combination product described in section 503(g).See: 21 USC 321(rr) Relevant case law includes but is not limited to: FDA v. Brown & Williamson.Supreme Court  (98-1152) 529 U.S. 120 (2000)Sottera v. FDA United States Court of Appeals, District of Columbia Circuit. No. 10–5032. (2010)

This definition misses the main point.  The definition of an FDA defined-tobacco product is a legal definition. It should be defined only by reference to the law that forms the definition. This may change over time through changes in legislation and case law.   The definition of an “FDA-defined tobacco product” must, ontologically, be identical to the definition used by FDA.  This is a follows:    An FDA defined tobacco product is any  product that meets the definition of a tobacco products set out in the United States Food, Drug and Cosmetic Act and as interpreted by the courts. Tobacco product is defined at Section 201 of the Federal Food, Drug, and Cosmetic Act (21 U.S.C. 321) in subsection (rr).The term 'tobacco product' means any product made or derived from tobacco that is intended for human consumption, including any component, part, or accessory of a tobacco product (except for raw materials other than tobacco used in manufacturing a component, part, or accessory of a tobacco product).The term 'tobacco product' does not mean an article that is a drug under subsection (g)(1), a device under subsection (h), or a combination product described in section 503(g).See: 21 USC 321(rr) Relevant case law includes but is not limited to: FDA v. Brown & Williamson.Supreme Court  (98-1152) 529 U.S. 120 (2000)Sottera v. FDA United States Court of Appeals, District of Columbia Circuit. No. 10–5032. (2010) Note that all countries have legal definitions of tobacco products for trade, administrative and excise duty purposes.  There is no reason to follow these definitions in science and it is a peculiarity of the United States that researchers use legal definitions that may be unhelpful scientifically.

I have four points about the definition: “A product that contains or consists of tobacco and is used by people to ingest tobacco constituents.”“or consists of” is redundant.“ingest” usually means to eat or absorb via the digestive tract and would be normally seen as different to absorption through inhalation  “Absorb” would be better as it allows for absorption in the oral cavity, lungs, throat or by any route. In the case of smoking, the user is not absorbing tobacco constituents but many de novo products of combustion that are not present in tobacco itself.For clarity, the definition should address the confusion about extracts made from tobacco leaf (for example, extracts of nicotine or natural extracts of tobacco used in tobacco flavoured vaping products, cosmetics or soaps).  It would be best if ‘tobacco’ was clarified to be restricted to parts of the tobacco plant in raw and cured form.  A suggested alternative definition: “A product that contains whole tobacco in raw or cured form and is used by people to absorb tobacco constituents or products of tobacco combustion. It does not include products made with extracts from the tobacco plant.”