Does AI Give Good Labeling Advice?

Several years ago, after a paper by OpenAI said that ChatGPT could hallucinate, I wrote an article about using ChatGPT for labeling advice. My finding then was that, yes, ChatGPT could (and did) hallucinate. The information given about how to label soap and cosmetics was generally correct, but also contained some glaring inaccuracies. Every question I asked has something wrong in the answer.

Now it’s two years later, and AI is supposed to have come a long way. So I decided to check again. Not just ChatGPT, but also Claude.ai, Google Gemini, Perplexity.ai, and Grok.

Comparison of AI programs

I tried to compare the programs evenly, using an incognito window in the browser. But I was limited to the number of questions before I had to log in. I used my Google login, and I think that may have skewed the answers from Claude, Gemini and Perplexity a little bit, and from ChatGPT a lot.

I found it very interesting that when I was NOT logged in, the references given (when given) included quite a few of my own website pages, or articles that I had written for other sites. But when I was logged in, I got different sources for the information.

ChatGPT

I use ChatGPT regularly and have been training it. Once I logged in, it recognized me and responded differently. I like ChatGPT – especially the fact that I can set up my own little GPT, upload files, and have it answer from those files first. I’ve uploaded all the regulations and the content of my books, Navigating the Rules and Regs and Soap and Cosmetic Labeling. As a result, when I want to check something or get a draft article, it’s usually pretty accurate.

I like the format of the results (although that may also be partly a result of my training). It doesn’t give sources unless asked, though, so that’s a drawback.

It did have a fair amount of detail and explanation. When I asked several questions in incognito mode, I still got some incorrect information, such as:

Chat GPT Said	Correct Data (see footnotes for sources)
Product Identity must be in bold type.	Product identity must be clear and conspicuous, not necessarily a “bold” font.¹
Information panel “the label panel immediately to the right of the principal display panel”	Information panels are all the parts of the label that aren’t the principal display panel (Back and side panels are generally called information panels.)²
Ingredient declaration should be parallel to the bottom of the package	No specification exists that it must be placed in a certain direction.
Ingredient declaration must use INCI Names	Partially true; for botanical ingredients, the ingredient should use the English common name first, not the Latin name (which is the international format).³
On your ingredient list, colors are named as “FD&C [Color] No. [#]” (e.g., FD&C Red No. 40).	True, but also: “Certified colors also may be identified in cosmetic ingredient declarations by color and number alone, without a prefix (such as Yellow 5).”⁴
If your product may contain one of several colors, you can use the “±” listing or “may contain”	The use of “may contain” is very limited. The ± (plus equal symbol) is not approved for use in the US.⁵

Claude.ai

I hadn’t used Claude before this experiment, so it was new to me. Personally, I don’t like the layout and formatting as much, but that’s a matter of personal preference, not a general drawback. And I just used the default style; it might be better with a different style selected.

Again, no sources referenced, but that may have been my settings.

I found the explanations to be less conversational and more of a series of bullet points drawn from different websites. Or, in one case, it appeared to simply summarize the information from one of the FDA’s website pages. Even so, it has some incorrect information.

Claude.ai Said	Correct Data (see footnotes for sources)
Common types of color additives: Natural colorants … beetroot	Beetroot is not approved as a color additive in cosmetics (it is an approved food).⁶
Identity statement: Common or usual name of the product	True, but it can also be a descriptive name, a fanciful name, an illustration, or a vignette if it communicates what the product is.⁷
Net Contents – Metric units must appear first	Not true. There are no specifications for placement of the metric net contents for cosmetics.
Ingredient declaration must use INCI Names	Partially true; for botanical ingredients, the ingredient should use the English common name first, not the Latin name (which is the international format).⁸
Information panel “the label panel immediately to the right of the principal display panel”	Information panels are all the parts of the label that aren’t the principal display panel (Back and side panels are generally called information panels.)⁹

Google Gemini

Gemini had detailed information (with a few inaccuracies) and links to sources. It’s odd that the sources are sort of mish-mashed. An article about label requirements might be cited for a line of text here or there, when actually the article was pretty correct for all of the data. I get the impression that the Gemini is trying to get “balanced” information by finding several different sources even though the information is the same and overlaps.

The inclusion (when I was incognito) or exclusion (when logged in) of source material I had written was very obvious.

Personally, I like the format, but I was a little put off by having different sources almost line-by-line. On the plus side, there are sources cited, so that’s good.

There were fewer inaccuracies by Google Gemini, but there still were some:

Google Gemini Said	Correct Data (see footnotes for sources)
The names of ingredients should generally follow the International Nomenclature of Cosmetic Ingredients (INCI) system	Partially true; for botanical ingredients, the ingredient should use the English common name first, not the Latin name (which is the international format).¹⁰
A warning for products with a safety that has not been substantiated.	With MoCRA, all cosmetics must have safety substantiation, so this warning is no longer valid.¹¹

Perplexity.ai

I hadn’t ever heard of perplexity.ai until I did a search to find different AIs to use in this experiment.

I like the format and layout. The data was presented in a more conversational manner, which (at least for me) makes it easier to read and grasp. However, in that format, it’s also a little harder to find the specifics.

Perplexity was good with references, and I like that it actually showed the name of the reference, so I didn’t have to mouse over it to see what it was.

This AI was the only one to mention fragrance allergen disclosures and changes to the labeling for professional products. However, despite citing multiple sources to answer my questions, it still contained wrong information:

Perplexity.ai Said	Correct Data (see footnotes for sources)
Soap with fragrance or skin softeners is a cosmetic.	Adding fragrance or skin softeners to a soap doesn’t automatically disqualify it from the exemption criteria which are:¹² a) the bulk of the non-volatile portion is the alkali salt of fatty acids (fragrances are volatile); and b) it is only labeled and marketed as soap.
Net contents … must use type no less than 1/16 inch in height (or 1/32 inch if the label area is under 12 square inches).	These specifications are for the ingredient declaration,¹³ not the net contents, whose size is determined by the size of the principal display panel.¹⁴
Ingredients must be listed using the International Nomenclature of Cosmetic Ingredients (INCI) name	Partially true; for botanical ingredients, the ingredient should use the English common name first, not the Latin name (which is the international format).¹⁵

Grok

Finally, I submitted the same questions to Grok. It was my first time using Grok, and I liked the layout and the way the information was presented. In the “Auto” mode, it was mostly quick bullet points and didn’t include citations.

Just for some comparison, I also asked the same questions in “Expert” mode. Expert mode got the changes and requirements for professional use products and the requirement for contact information for adverse event reporting. It also checked the information against MoCRA, noting any changes. Hiowever, I found that in Expert Mode, the data was more concise and not as detailed. But it did have citations!

In either mode I still got some inaccurate information:

Grok Said	Correct Data (see footnotes for sources)
The identity must be in bold type.	Product identity must be clear and conspicuous, not necessarily a “bold” font.¹⁶
Business name and address must be at least 1/16 inch in height	Not a specification in the regulations for business name and address.¹⁷
Ingredient declaration must appear on an Information Panel	It usually does, but it is not required. It may be placed on the principal display panel (although it usually is not, just for marketing purposes).
Unsubstantiated safety warning is cited.	With MoCRA, all cosmetics must have safety substantiation, so this warning is no longer valid.¹⁸
Professional-Use Products… are exempt from some labeling requirements, such as the ingredient declaration	With MoCRA, all professional use products must have full label information, plus additional statements.¹⁹

Final Thoughts

As good as AI is for some things, it still isn’t completely accurate when it comes to label requirements. Close, yes. Mostly right, yes. Completely 100% trustworty, no.

For nearly 20 years, I’ve been studying and following the laws, rules, regulations, and guides (and everything else I could get my hands on) in order to really understand all the nuances of cosmetic labeling. It’s tricky – especially when you get things like, “This is always true, except when that meets the definition of some other regulation or law” and you end up down a rabbit hole.

It’s definitely been a process to distill it down to the truth. I’ve come across SO many websites that have almost correct information. Since all the AI models scour the internet for information, maybe that’s part of the problem.

Pretty soon we’ll have articles being written by AI being referenced as the source for other articles written by AI. One error at the start just perpetuates throughout. Like “the information panel is directly to the right of the principal display panel.” Say, what? No! But two of the AIs made that exact same statement.

If your name is on the label of a cosmetic product, my advice is:

Get personally familiar with the rules and then make sure that they are followed for your product.
Research reliable data sources such as the FDA website, professional associations (usually pretty good), or get one of my books (Navigating the Rules and Regs or Soap and Cosmetic Labeling) that explains the regs in plain English.
Have your information double-checked by someone who knows, who can answer your questions, and refer you to the actual source.

I’m available – but if not me, then someone you trust and know is informed, accurate, and reliable. After all, it’s YOUR name on the label!

33 FR 9481 “Regulations for the Enforcement of the Federal Food, Drug and Cosmetic Act and the Fair Packaging and Labeling Act; Label Statements for Drugs, Devices and Cosmetics.” 1968 ↩︎
FDA. Cosmetics Labeling Guide. ↩︎
FDA. Cosmetic Ingredient Names ↩︎
FDA. Cosmetic Ingredient Names ↩︎
21 CFR 7 01.3(g) ↩︎
FDA. Color Additives Permitted for Use in Cosmetics and 21 CFR 73 ↩︎
21 CFR 701.11 ↩︎
FDA. Cosmetic Ingredient Name ↩︎
FDA. Cosmetics Labeling Guide ↩︎
FDA. Cosmetic Ingredient Name ↩︎
21 USC 364d. Safety Substantiation ↩︎
21 CFR 701.20 ↩︎
21 CFR 701.3(i) ↩︎
21 CFR 701.13(i) ↩︎
FDA. Cosmetic Ingredient Name ↩︎
33 FR 9481 “Regulations for the Enforcement of the Federal Food, Drug and Cosmetic Act and the Fair Packaging and Labeling Act; Label Statements for Drugs, Devices and Cosmetics.” 1968 ↩︎
21 CFR 701.12 ↩︎
21 USC 364d. Safety Substantiation ↩︎
21 USC 364e(c) ↩︎